Skip to the content
At Tian’s.The Studio
Back to the roomThe roomBack to the indexThe index

AACL-IJCNLP 2026 to appear

Three Numbers and a Precondition: Auditing Multi-Source LLM Benchmarks for Source-Identity Leakage

  1. Tian Xia
  2. Sarp Aykent

Affiliation at the time of the workMicrosoft (both authors). Sarp Aykent is now at NVIDIA.

When a test set pools rows from several sources, the detector being tested can score well just by recognising the source. The paper asks for two more scores beside the pooled score, and one check before one score is divided by another.

Read the paper (PDF) (opens in a new tab) Follow two real audits

Camera-ready version (PDF). The ACL Anthology link will replace it when the AACL-IJCNLP 2026 proceedings are out. Build the shortcut by hand. Words on this page

Figure 1Measured · Section 5.4

Sort the same 117 runs two ways, and watch one group turn solid.

A detector read the training log of each finished run and rated it. Every run has a known label, faulty or healthy.

It scored well on these 117 pooled runs, until the authors sorted them by the collection each run came from.

Sort the runs or draw a pair

Fault-injection release96 faulty, 0 healthyBaseline runs, same release11 healthyPublic project5 healthyPublic project5 healthy

filled: faulty outlined: healthy

Arrange

The score Take one faulty and one healthy run: how often does the faulty one get the higher rating? 0.50 is a coin flip, 1.00 is perfect.

No pair drawn yet.

Sorted by where each run came from. One collection holds all 96 faulty runs and no healthy run, so the collection name alone scores 1.000. It never opened a log.

MeasuredThe counts and the two scores are the paper's. The mixed order and the pairs are this page's. The faults were injected on purpose in a public release, so this shows a direction only: nothing here says any training software causes failures.

01 The problem

When sources differ in how often the label is yes, the source tag is already a hint.

Teams now routinely stitch a test set together from several sources: grades from several AI graders, quality numbers from several retrieval setups, training runs from several frameworks. Every row keeps a source tag that says where it came from. If the sources differ in how often the label is yes, the tag alone already tells you something about the label. Here are two real test sets from the paper.

Figure 2Measured · Sections 5.1 and 5.2

Cover everything but the source tag, and see which test set it still separates

Show

Two pooled test sets. Every row has measurements, a label and a source tag.

Pooled RAGBench rows

RAGBench is a public benchmark for systems that fetch documents and then write an answer from them. It comes in several configurations, each with its own documents and domain, and publishes quality numbers for every answer. What is detected: can those numbers tell supported answers from unsupported ones?

Row
one query, its retrieved documents, the generated answer
Measurements
six published per-query quality numbers, such as relevance, utilization and completeness
Label
adherent: is the answer supported by its documents?
Source tag
which of 4 configurations: HotpotQA, DelucionQA, EManual, ExpertQA
92.8%HotpotQA94.2%DelucionQA86.2%EManual40.2%ExpertQA

By source tag: knowing the configuration already says a lot about the label. 299 of the 433 no rows sit in one configuration.

2,000 rows, 500 from each; 1567 of 2000 yes, 78.3%

The pooling is the authors'. RAGBench does not publish this pooled score and does what it advertises: per-query numbers in each configuration.

Hard math, five judges

A judge here is an AI model (an LLM, a large language model) that reads an answer to a hard math problem and grades it CORRECT or INCORRECT. What is detected: can that grade tell answers that match the answer key from answers that do not?

Row
one judge's grade on one answer
Measurements
the grade itself, CORRECT or INCORRECT; the only one
Label
does the answer match the answer key? 41% do
Source tag
which of 5 judges, alphabetically: Claude Haiku 4.5, Claude Opus 4.6, Claude Sonnet 4.6, GPT-4o, one GPT-5-family reasoning model; every judge grades the same answers
41%41%41%41%41%five judges, the same 200 answers

By source tag: knowing the judge says nothing. Every judge graded the same answers.

200 answers, 966 usable grades of 1,000

About one set of answers, not about any judge.

How to read it. A filled tile is a row whose label is yes, an outlined tile a row whose label is no. A yes-rate is how often the label is yes among one source's rows. The drawing is this page's own; the yes-rates are the paper's. One new source: inside one source the tag is the same on every row, so the tag can separate nothing there. With everything visible: nobody measured this.

A score earned by recognising the source is no promise about a new source.

Inside one source the tag is the same on every row, so it can separate nothing there. A pooled score that leaned on the tag therefore promises nothing about a single new source. Nobody measured what happens there; that is the point. The same trap is known in digital health, where detectors learned to recognise the participant and not the disease.

See it in Figure 2: one new source

Pooled scores drive decisions: which detector a team trusts, how a leaderboard orders its entries. When such a score can mostly be recovered from the source,

those decisions inherit a number that may not transfer to the single-source setting in which most practitioners deploy

The paper, Ethical Considerations, p.10

02 Two audits, followed

One tempting ratio makes two very different test sets look alike. The pooled score itself tells them apart.

One run of the paper's procedure on one test set is an audit, and it is small. Score the same rows three ways: with everything visible (the headline: the pooled score a team would report), with the source tag covered (the measurements-only score), and with the source tag alone (the source-only score). Each score (the paper: AUC) is the pairs game of Figure 1: 0.50 is a coin flip, 1.00 is perfect. Then divide the source-only score by the headline. Follow both test sets through it, one step at a time.

Words on this page row, measurements, label, source tag, score, the three numbers, the ratio L, floor, band, verdict
  1. Row. One item with a known yes or no label: one finished training run, one generated answer, one judge's grade on one answer.
  2. Measurements (the paper: features). What a detector is allowed to read about a row. The auditor chooses the list and publishes it.
  3. Label. The known yes or no for a row. It never means the source.
  4. Source tag. The column that says where a row came from: which collection, which configuration, which judge. The yes-rate (the paper: label rate) is how often the label is yes among one source's rows.
  5. Detector, rating, score (the paper: AUC). A detector gives every row a rating. The score is a pairs game over a whole test set: take one yes row and one no row; how often does the yes row get the higher rating, ties counting half? 0.50 is a coin flip, 1.00 is perfect, and it is not percent correct. Knowing nothing scores 0.50, not 0.
  6. The three numbers (the paper: the triad). The same rows scored three ways: the headline (everything visible), the measurements-only score (tag covered), the source-only score (tag alone). One run of this on one test set is an audit.
  7. The ratio, L (the paper: leakage ratio). The source-only score divided by the headline. One score divided by another, never a share of anything; it cannot fall below 0.50 divided by the headline, and it is above 1 when the tag alone outscores the headline.
  8. Floor, band, verdict. Floor: the headline must reach 0.65, the authors' default, before L is read; under it L is reported but not interpreted (printed, not read). Band: one line, not a range: with the floor passed, L of at least 0.90, also the authors' default, is flagged. Both were chosen on their own test sets. The verdict is the outcome of an audit; a judge gives a grade. "Leakage" is the paper's operational label with no causal content: nothing leaked into training data and it is not a security leak.

Figure 3Measured · Tables 1 and 2

Follow two real audits step by step, or select any box

Follow

The two audits both drawn at every step; the one you follow is litwhat each has printed by the last step

The audit as a diagram where this step happens; every box opens its detail

The audit as a diagram: pooled rows, three scores on one ruler, the ratio, the floor check, the band check, four verdicts, one report. Pooledtest setThreescoressame rows, onlywhat is visiblechangesReadingruleL is alwaysprinted; thefloor decidesif it is readsource-only ÷ headlineheadlineLyesno: under the flooryestag at the coin flipotherwisePooled rowsone test setMeasurementsper-row numbersSource tagwhere it came fromLabelknown yes or noTag coveredmeasurements onlyEverythingthe headlineTag alonesource onlyOne ruler0.50 coin flip1.00 perfectThe ratio, Lalways printedFloor checkat least 0.65?Band check0.90 or more?Source-dominatedToo weakto interpretSource carriesnothingNotflaggedPrint it beside the headline: three scores, L, the floor check, one verdict

Pooled RAGBench rows headline 0.843 · source-only 0.811 · best single measurement 0.637 · L 0.96 · floor passes · source-dominated

Hard math, five judges headline 0.594 · measurements-only 0.585 · source-only 0.501 · L 0.84 · printed, not read · too weak to interpret

  1. Pool the rows

    The authors drew 500 rows from each of four RAGBench configurations and stacked them: 2,000 rows, one score. The pooling is theirs. Every row keeps a source tag that says which configuration it came from.

    Five judges, each an AI model that grades answers, grade the same 200 answers to hard math problems. One judge's grade on one answer is a row: 966 usable grades of 1,000, because 3.4% could not be read as CORRECT or INCORRECT. Every row keeps a source tag that says which judge gave the grade.

    In
    4 configurations of 500 rows5 judges on the same 200 answers
    Out
    2,000 rows with a source tag966 rows with a source tag

  2. Count the yes labels in each source

    The label asks: is the answer supported by its documents? The yes-rate differs by configuration: 92.8%, 94.2%, 86.2% and 40.2%, pooled 78.3%. Of the 433 no rows, 299 sit in one configuration, so the tag is already a strong hint.

    The label asks: does the answer match the answer key? 41% do. Every judge graded the same answers, so every judge's rows have the same yes-rate, and knowing which judge tells nothing about the label.

    In
    rows with a label and a source tag
    Out
    92.8 / 94.2 / 86.2 / 40.2%41%, five times

  3. Score the source tag alone

    Give every row one rating, the yes-rate of its source, and play the pairs game: does the yes row have the higher rating? Without reading a single answer, the source tag alone scores 0.811. Nothing is trained.

    The same recipe gives every row the same rating, so every pair is a tie, and ties count half. The source tag alone scores 0.501: the coin flip, by design and not because of any judge. It is 0.501 and not 0.500 only because a few grades could not be read.

    In
    source tag, label
    Out
    source-only score 0.8110.501

  4. Cover the source tag

    Now the opposite view: measurements only. No detector was built on all six published quality numbers, so the honest stand-in is to use one quality number at a time as the rating: the best scores 0.637 and the weakest 0.480. Every one sits below the source tag alone.

    The only measurement is the grade itself, and alone it scores 0.585. On this set the judges said CORRECT 77% to 93% of the time while 41% of answers were correct, so the grades barely separate right from wrong. That describes one set of answers, not any judge.

    In
    measurements, label
    Out
    one measurement at a time: 0.480 to 0.637measurements-only score 0.585

  5. Score with everything

    The headline is the number a team would report: measurements plus source tag. Here the rating is fixed, half the source's yes-rate plus half the row's mean measurement, and no model is trained. It reads 0.843. Its 95% interval, the range it would likely stay in if the rows were drawn again, is 0.825 to 0.860. It stands close to the source tag alone.

    The same fixed rating: half the judge's yes-rate plus half the grade. It reads 0.594 (95% interval 0.534 to 0.654). All three marks huddle by the coin flip: nothing here detects much, and the source tag detects nothing.

    In
    measurements, source tag, label
    Out
    headline 0.8430.594

  6. Divide, and both come out high

    The tempting summary is one ratio, L: the source-only score divided by the headline, 0.811 ÷ 0.843 = 0.96. But a tag that knows nothing still scores 0.50, so L can never fall below 0.50 ÷ headline, which is 0.59 here. The observed 0.96 stands well clear of that.

    0.501 ÷ 0.594 = 0.84, which looks almost as high. But the lowest value L could take here is 0.50 ÷ 0.594 = 0.84. The ratio sits on its lowest possible value: it is high only because the headline is near the coin flip.

    In
    source-only score, headline
    Out
    L 0.96, lowest possible 0.59L 0.84, lowest possible 0.84

    L is one score divided by another, never a share of anything.

  7. Before reading it, check the floor

    Before L is read, the headline must reach the floor: 0.65, the authors' default, chosen on their own test sets. 0.843 clears it, and so does the low end of its interval, 0.825. L may now be read.

    The headline is 0.594 and the floor is 0.65. The point estimate and the low end of its interval, 0.534, both miss it. The verdict is too weak to interpret: neither flagged nor cleared. The ratio is still reported but not interpreted: printed, not read.

    In
    headline; near the floor, the low end of its 95% interval
    Out
    floor passesverdict: Too weak to interpret

  8. Read the ratio, if the floor let it through

    With the floor passed, L meets one more line, the band: 0.90 (the paper: the band), the authors' default. 0.96 is above it, so the pooled score the authors computed is flagged as source-dominated: the tag alone ranks rows nearly as well as everything together. This is about one pooled score; RAGBench does what it advertises.

    Nothing more is read: the floor failed, so the band is never reached. What this set needs is more informative grades, not a fix on the source side.

    Two ratios that looked alike, 0.96 and 0.84, end differently: one is flagged, the other cannot be read at all. The floor on the headline separated them.

    In
    L and the band, only if the floor passed
    Out
    verdict: Source-dominatednot reached

    Either way the audit prints numbers, not a badge.

Select any box in the drawing to read what it takes in and gives out. Arrow keys move between boxes.

Pooled rows

A pooled test set stacks rows from several sources and reports one score for all of them. A row is one item with a known yes or no label.

In
rows from several sources
Out
one table: measurements, source tag, label
Here
2,000 rows · 966 rows
Section 2, p.2

Measurements (the paper: features)

What a detector may read about a row. Where a benchmark publishes per-row numbers the audit uses those; otherwise the auditor builds a list. Either way it says which ones it used.

In
what is published per row
Out
the columns a detector may read
Here
six published quality numbers · the grade itself
Section 2, p.3; Section 4.2, p.5; Limitations, p.10

Source tag

Where the row came from: which configuration, which judge, which training framework. The auditor chooses this column and says which one was chosen.

In
how the test set was assembled
Out
one category per row
Here
1 of 4 configurations · 1 of 5 judges
Section 2, p.3

Label

The known yes or no every score is checked against. It never means the source.

In
the answer key
Out
yes or no per row; a yes-rate per source
Here
supported by its documents? 78.3% yes · matches the key? 41% yes
Section 2, p.3

Tag covered: the measurements-only score

The same detector with the source tag covered. It shows whether the detector needs the tag column; it cannot show that the detector uses no source information, because measurements can carry the source themselves.

In
measurements, label
Out
the measurements-only score (square)
Here
one measurement at a time, 0.480 to 0.637 · 0.585
Section 4.2, p.5

Everything: the headline (the paper: AUC_full)

The detector sees measurements and source tag together; this is the number a team would report. In these two audits the rating is fixed, half and half, and nothing is trained; other audits in Figure 5 train a model, so compare verdicts, not third decimals.

In
measurements, source tag, label
Out
the headline (circle)
Here
0.843 (0.825 to 0.860) · 0.594 (0.534 to 0.654)
Section 4.1, p.4

Tag alone: the source-only score

Every row gets one rating, the yes-rate of its source, worked out from these same rows. No training and no measurements are involved.

In
source tag, label
Out
the source-only score (triangle)
Here
0.811 · 0.501
Section 4.1, p.4

One ruler (the paper: AUC)

Every score is the same pairs game: take one yes row and one no row; how often does the yes row get the higher rating, ties counting half? 0.50 is a coin flip, 1.00 is perfect, and it is not percent correct.

In
one rating per row, the label
Out
one number per view, read against 0.50 (coin flip) and 1.00 (perfect); a weak score can land a little under 0.50
Here
marks far to the right · marks by the coin flip
Section 2, p.3; Appendix A, p.13

The ratio, L (the paper: leakage ratio)

The source-only score divided by the headline: how close the tag alone comes to the number a team would report. One score divided by another, never a share, and it cannot fall below 0.50 ÷ headline even when the tag knows nothing.

In
source-only score, headline
Out
L, printed beside every verdict
Here
0.96, lowest possible 0.59 · 0.84, lowest possible 0.84
Eq. 1, Section 2, p.3

Floor check (the paper: the floor, the precondition)

The headline must reach 0.65 before L is read; near the floor, go by the low end of the 95% interval. 0.65 is the authors' default, chosen on their own test sets.

In
the headline, and its interval near the floor
Out
pass: L may be read. Fail: the three numbers, L printed, not read
Here
passes · fails
Section 4.2, p.4 to 5

Band check (the paper: the band)

With the floor passed, L of at least 0.90 is flagged. 0.90 is the authors' default for reporting and less stable than the floor, which is why L is printed beside every verdict.

In
L, only if the floor passed
Out
a verdict
Here
0.96 is above it · not reached
Section 4.2, p.5; Appendix C, p.15

Source-dominated (the paper: leakage-confirmed)

The floor passes and L is at least 0.90: the tag alone ranks rows nearly as well as the full detector. It describes one pooled score, not a benchmark's maintainers and not any single source.

In
a passed floor; L at least 0.90
Out
the flag; repairs the paper lists, listed, not tested
Here
pooled RAGBench rows. RAGBench does what it advertises.
Section 4.2, p.5 to 6; Ethical Considerations, p.10

Too weak to interpret

The headline is under the floor, so the ratio is not read: neither flagged nor cleared. It calls for more informative grades or measurements first, and it reflects the auditor's detector as much as the test set.

In
a failed floor check
Out
the three numbers; L printed, not read
Here
hard math, five judges
Section 4.2, p.4; Section 5.3, p.7; Ethical Considerations, p.10

Source carries nothing (the paper: leakage-ruled-out)

The floor passes and the source-only score sits at the coin flip, so L sits at its lowest possible value. It says only that the pooled headline cannot borrow from the tag under this design; it is not a certificate.

In
a passed floor; tag at the coin flip
Out
the verdict
Here
neither; the same five judges on another set of answers end here (Figure 5, audit 8)
Section 4.2, p.5 to 6; Section 5.3, p.8

Not flagged (the paper: no verdict, the three numbers are reported)

The floor passes, L is under 0.90 and the tag is above the coin flip. Print the three numbers and L as they are; not flagged is not a certificate.

In
a passed floor; L under 0.90
Out
the three numbers and L
Here
neither; audits 5 and 7 in Figure 5 end here
Section 4.2, p.5

Print it beside the headline

The audit publishes numbers, not a badge: the three scores, L, the floor check, the source tag and measurement list used, and one verdict. The cost is three scores and one threshold check.

In
everything above
Out
one report line
Here
the two printed lines, one per audit, complete at the last step
Section 4.2, p.5 to 6; Section 6, p.9

How to read it. The upper drawing uses the paper's measured numbers. The lower drawing is this page's own diagram of an audit and computes nothing. The dashed link is the exit taken when the floor fails. A verdict box repeats its stamp's border: heavy for source-dominated, dashed for too weak, double for source carries nothing, dotted for not flagged. Both audits use a fixed rating with no training, while other audits in Figure 5 train a model, so compare verdicts, not third decimals. Circle: headline. Square: measurements-only score. Triangle: source-only score.

03 Build it by hand

A source tag that knows nothing still scores 0.50, so the ratio starts high for free.

Now build a test set by hand. In the two audits you followed, the sources differed a lot in one and not at all in the other, and the rows told little in both. Here you can move the two things separately: how much the sources differ, and how much the rows tell. Everything in this figure is a formula, not data.

Figure 4Illustration

Build the shortcut by hand, then line the sources up again and watch the ratio stay high

Sourceseach column is filled to its yes-rate92.8%HotpotQA94.2%DelucionQA86.2%EManual40.2%ExpertQA0.600.700.800.900.50coin flip1.00perfectfloor 0.65 (the authors' default)headline 0.841measurements-only 0.637source-only 0.811measured0.8430.8110.480 to 0.637one measurement at a time; no detector on all sixout of reach00.501.001.30band 0.90 (the authors' default)lowest possible 0.59printed, not read: the headline is under the floorL 0.96measured L 0.96L = source-only ÷ headline · one score divided by another, never a share of anythingVerdictSOURCE-DOMINATEDa flag on onepooled score
The four yes-rates of the pooled RAGBench rows. Filled marks: the formula. Outlined pins marked "measured": the paper's numbers. About one pooled score the authors computed: RAGBench does what it advertises.
Try

Every source has the same yes-rate, so the tag knows nothing: 0.50, and L rests on its lowest possible value. Three things to try, in order.

80%
0.75
Start from

The still above, and the five-judge preset, as numbers. The formula is not expected to land on the measured values, because the paper's headline and the formula are built differently. Only 0.8106 follows from the yes-rates alone.
ValueRAGBench yes-rates: formulameasuredFive judges: formulameasured
Headline0.8410.8430.5850.594
Measurements onlysquare parked at 0.637no detector on all six; one measurement at a time 0.480 to 0.6370.5850.585
Source only0.81060.8110.5000.501
L (lowest possible)0.96 (0.59)0.96 (0.59)0.85 (0.85)0.84 (0.84)
  1. Every source at the same yes-rate: the tag knows nothing, the source-only score is 0.50 and L rests on its lowest possible value.
  2. Pull one source down: the sources now differ, so the source tag alone can order some pairs.
  3. Make the rows tell less: the less the rows tell, the more the headline rests on the source tag.
  4. Line the sources up again: the tag knows nothing now, and the ratio is still above the band. It is resting on its lowest possible value.

IllustrationA formula, with no data and no trained detector. Filled marks are computed in your browser; pins marked "measured" are the paper's numbers, and the formula is not expected to land on them. This page's choices: the 0.505 line between "at the coin flip" and "above it", and parking the square at 0.637, the best single measurement, for the RAGBench yes-rates.

What you just saw. Below a headline of 0.556, which is 0.50 ÷ 0.90, the ratio is above 0.90 whatever the source does. That is arithmetic on the authors' default band, not a universal threshold. Circle: headline. Square: measurements-only score. Triangle: source-only score.

04 The evidence

Across ten audits, high ratios sit on both sides of the floor.

The authors ran the same audit ten times on public test sets. The two you followed are already marked. Some audits use a fixed rating and some train a model, so compare verdicts, not third decimals.

Figure 5Measured · Table 1, p.5

Find the two audits you know, then point at any other

0.500.600.700.800.901.00coin flipperfectheadline score (AUC)0.500.700.901.101.30L0.8430.960.5940.84floor 0.65 (the authors' default)band 0.90 (the authors' default)printed,not readSource-dominatedNot flaggedSource carries nothinglowest possible L: 0.50 ÷ headline1Pooled RAGBenchrows2Hard math,five judges345678910

Filled: already audited when the authors picked 0.65 and 0.90. Ring: audited afterwards, with both values fixed. Hatched, left of the floor: printed, not read. The dashed curve is the lowest possible L; it meets the band at a headline of 0.556, which is 0.50 ÷ 0.90. Both axes start at 0.50: no score and no lowest possible ratio on this chart sits below it.

Try

Two ratios close in height, on opposite sides of the floor.

0.65

The floor is at 0.65. For any floor from 0.60 to 0.68 every verdict stays as printed; at 0.70 one audit changes.

High L occurs on both sides of the floor: horizontal position, not height, decides whether L is read.

The paper, Figure 3 caption, p.7

Flagged means the source tag alone scores nearly as high as the headline, not that the detector used the tag. On the RL audits the detector does not need the tag column. That cannot show it uses no source information: logged step counts ran 20 to 40 in one framework's runs and 35 to 999 in another's. Nearly all faulty runs come from one framework, so read direction, not magnitude; nothing here says any framework causes failures.

The dataset cards of audits 3 to 10 are in the fold under the table; those of audits 1 and 2 stand in Figure 2.

Verdict

At the default floor the band decides the verdict for three audits (L 0.83, 0.85, 0.92), which is why L is printed beside every verdict. Both values are the authors' defaults, chosen on their own test sets.

Ten audits, in the order of the paper's Table 1.
AuditReading (the paper's rule)Source tagnHeadline (AUC)Measurements-onlySource-onlyL
1Pooled RAGBench rowsSource-dominated4 RAG configurations2,000 rows (500 each)0.843† (0.825 to 0.860)0.480 to 0.637*0.8110.96
2Hard math, five judgesToo weak to interpretjudge (every judge grades the same answers)966 grades on 200 answers0.594† (0.534 to 0.654)0.5850.5010.84
3MT-Bench human judgmentsToo weak to interpretannotator and turn (130 sources)2,575 judgments0.5270.4990.6451.22
4MMLU, stand-in labelToo weak to interpretsubject (57)14,042 items0.5090.4950.5561.09
5RL, two-framework auditNot flaggedtraining framework113 runs (19 healthy + 94 faulty)0.8870.8810.7370.83
6RL, plus the TRL frameworkSource-dominatedtraining framework134 runs (35 healthy + 99 faulty)0.9110.8840.8380.92
7Hard math, three answer modelsNot flaggedwhich model answered3,456 answers0.8710.7860.7410.85
8Same five judges, another set of answersSource carries nothingjudge (every judge grades the same answers)4,999 grades on 1,000 answers0.690† (0.663 to 0.715)0.6890.5000.725
9MT-Bench, second measurement setToo weak to interpretsame as audit 32,575 judgments0.5330.5000.6451.21
10MMLU, second measurement setToo weak to interpretsame as audit 45,000 items (a fixed random subset)0.5200.5000.5781.11
  • † A fixed rating, nothing trained (three audits). The other seven train a model, an out-of-fold Random Forest, scored only on rows it was not trained on; the source-only score is always worked out on the same rows. So compare verdicts, not third decimals.
  • * No detector was built on all six measurements; the range is one published measurement at a time used as the rating.
  • n counts different things by row, and some audits share rows, so ten audits are not ten independent datasets.
  • Intervals are 95% intervals: the range a score would likely stay in if the rows were drawn again. They belong to the headline, never to L.
  • L above 1 in four audits: the tag alone outscored the headline, which points at measurements that tell nothing, and all four miss the floor.
  • MT-Bench and MMLU were planned in advance, missed their planned targets and are reported as floor failures.
  • Pooled RAGBench rows: about one pooled score the authors computed; RAGBench does what it advertises.
The question
Is a good pooled score about the rows, or about where they came from?
The reading
Five miss the floor and five clear it. Of those five, two reach the band and three do not. Four of the five left of the floor have a ratio above 0.90, so the ratio alone would have flagged all four.
The dataset cards of audits 3 to 10

The cards of audits 1 and 2 stand in Figure 2.

3MT-Bench human judgments

Row
one human choice between two answers
Measurements
length, hedge, refusal and verbosity proxies
Label
answer B wins
Source tag
annotator and turn: 130 sources

2,575 judgments · out-of-fold Random Forest, shared 5-fold split

Planned in advance, missed its planned target, reported as a floor failure. 130 small sources flatter the source-only score.

4MMLU, stand-in label

Row
one multiple-choice item
Measurements
token count, options count, language
Label
a stand-in (the paper: proxy): is the gold answer option A? No model was scored
Source tag
subject: 57

14,042 items · out-of-fold Random Forest, shared 5-fold split

Planned in advance, missed its planned target. The main claims do not rest on it.

5RL, two-framework audit

Row
one finished training run (RL fine-tuning)
Measurements
numbers derived from its logged training metrics
Label
faulty or healthy
Source tag
training framework (more than two tag values)

113 runs: 19 healthy + 94 faulty · out-of-fold Random Forest, shared 5-fold split

Re-uses runs from the opening set. Read direction, not magnitude.

6RL, plus the TRL framework

Row
one finished training run (RL fine-tuning)
Measurements
numbers derived from its logged training metrics
Label
faulty or healthy
Source tag
training framework, with the authors' later TRL runs added

134 runs: 35 healthy + 99 faulty · out-of-fold Random Forest, shared 5-fold split

A different set from the 113-run audit; their numbers never mix. Read direction, not magnitude.

7Hard math, three answer models

Row
one model's answer to a hard math problem
Measurements
call metadata only: temperature, token counts, latency, retries
Label
does the extracted final answer equal the key?
Source tag
which of three models answered; not named, not ranked

3,456 answers · out-of-fold Random Forest, shared 5-fold split

Token counts depend on the tokenizer, so the measurements are not fully blind to the source.

8Same five judges, another set of answers

Row
one judge's grade on one answer
Measurements
the grade itself, CORRECT or INCORRECT
Label
does the answer match the answer key? 62.2% do
Source tag
which of the same 5 judges; every judge grades the same answers

1,000 answers, 4,999 usable grades · fixed rating, no model trained

The two sets of answers differ (41% and 62.2% correct); not a like-for-like scale-up.

9MT-Bench, second measurement set

Row
one human choice between two answers
Measurements
a second set, written separately from the raw schema
Label
answer B wins
Source tag
annotator and turn: 130 sources

2,575 judgments · out-of-fold Random Forest, shared 5-fold split

Same rows and sources as audit 3; both measurement sets were written by the authors.

10MMLU, second measurement set

Row
one multiple-choice item
Measurements
a second set, written separately
Label
a stand-in: is the gold answer option A?
Source tag
subject: 57

5,000 items, a fixed random subset · out-of-fold Random Forest, shared 5-fold split

A subset, so its source-only score is not comparable with audit 4.

The opening 117 runs, in three numbers

This opening set used a different source tag and a different model from the RL audits above, so it is not on the chart. Row: one finished training run (RL fine-tuning). Measurements: numbers from its training log. Label: faulty or healthy; faulty means a fault injected on purpose in a public release. Source tag: the collection it came from, read off the run name.

1.000 headline0.907 measurements-only1.000 source-only

Logistic regression, 5-fold; the source-only score is worked out on the same rows. The four run-name prefixes and their counts: openrlhf_fault 96 faulty, 0 healthy; openrlhf_baseline 11 healthy; dapo_verl 5 healthy; deepscaler_verl 5 healthy.

The authors also shuffled the source tags across runs, 1,000 stored shuffles, with measurements and labels held fixed. The stored summary of the ratio, as four numbers:

0.635 mean of the shuffles0.718 99th percentile0.756 maximum1.000 observed

Direction only; it does not transfer to other test sets. The measurements can carry the source too: logged step counts ran 20 to 40 in one framework's runs and 35 to 999 in another's. Nothing here says any training software causes failures. Section 5.4, p.8; Appendix D.1, p.16

05 On Monday

Two ratios that look alike call for different repairs.

The audit costs three scores and one check: score your detector once more with the source tag covered, and score the tag alone. What you do next depends on which side of the floor you are: the floor is 0.65 and the band 0.90, both the authors' defaults.

The repairs the paper lists: listed, not tested.

Too weak to interpret

Get more informative grades or measurements first. In the paper: hard math, five judges.

Source-dominated

Report each source separately, or weight the rows so every source has the same yes-rate (the paper: reweighting), or model each source's level separately. In the paper: pooled RAGBench rows.

Not flagged Source carries nothing

Publish the three numbers and L as they are. Neither is a certificate.

After a source-side repair, what the measurements achieve inside each source still has to be measured on the rows.

  1. Name the source column, and write down which one you chose.
  2. Score the tag alone: give each row its source's yes-rate. No training.
  3. Print the other two scores beside the headline.
  4. Check the floor before comparing.
  5. Look for source fingerprints inside the measurements.

Advice, not a finding of the paper. It follows from how this work started: build test sets where yes rows and no rows share sources.

Beside every pooled score, print two more scores and check one condition.

Figure 6Your numbers, the paper's rule

Set your own three numbers and read the verdict

Close to the headline: the detector does not need the tag column. It cannot show that it uses no source information.

Near the floor? Add the low end of your interval

Near the floor the paper recommends a stricter rule: L is read only if the low end of the 95% interval of the headline is above 0.65. The interval belongs to the headline, never to L.

The reading, in three checks

  1. 1
  2. 2
  3. 3

  1. Floor: if the headline is under 0.65 (the authors' default), L is printed, not read, and the verdict is too weak to interpret.
  2. Ratio: L is the source-only score divided by the headline; it cannot fall below 0.50 divided by the headline.
  3. Coin flip: if the floor passes and the source-only score prints as 0.50, the source carries nothing, which is not a certificate.
  4. Band: if the floor passes and L is at least 0.90 (the authors' default), the pooled score is source-dominated; otherwise it is not flagged, which is not a certificate.

IllustrationComputed by this page from the numbers you enter, with the authors' default floor and band and this page's 0.505 line for "at the coin flip". Not a result of the paper, and no verdict is a certificate. It opens with made-up numbers.

The one idea

Check the floor, then the ratio.

What to print

Three scores, L, the floor check, the source tag and measurement list you used, and one verdict. Numbers, not a badge.

What two audits showed

Ratios of 0.96 and 0.84 looked alike and ended differently: one is flagged, the other cannot be read at all. The headline, not the ratio, separated them. The flagged one is a pooled score the authors computed: RAGBench does what it advertises.

What is not claimed

No new RL algorithm. No early-warning or early-stopping system. No compute savings. No universal thresholds. No tested repairs. No claim that any framework, configuration, judge or model causes failures or is better or worse. No claim that RAGBench is broken. No claim that a detector fails on a new source, only that a pooled score is no promise about one.

06 Office hours

The night shift answers.

Written by Tian's agent while Tian slept and Pepper supervised. Pepper accepts no liability either.

Each answer says where in the paper it comes from, so you can check the agent's work.

Pepper, the tabby cat, supervising: fast asleep on his blanket on the sofa, beside the laptop.
Isn't L just how much of the score comes from the source?

No. L is one score divided by another, and knowing nothing scores 0.50, not 0, so the top of the division never starts at zero.

On the same five judges with another set of answers the tag sits at 0.500, the coin flip, and L is still 0.725. On the 200-answer set L is 0.84, and that is the lowest value it could take there. Below a headline of 0.556 the ratio is above 0.90 whatever the source does; four of the five audits left of the floor have ratios above 0.90 (1.22, 1.09, 1.21, 1.11).

Source: Section 2, p.3; Table 1, p.5; Section 5.3, p.8
Who chose 0.65 and 0.90, and how fragile are they?

The authors did, on their own test sets; they call both operating defaults, and the floor was formulated with the 200-answer result in hand, so that audit illustrates the failure and does not test the threshold.

Any floor from 0.60 to 0.68 leaves the ten verdicts as printed; at 0.70 the 1,000-answer judge audit (0.690) changes; at 0.55 the 200-answer audit (0.594) would pass. The band decides the verdict for three audits (L 0.83, 0.85, 0.92). No test set was chosen or audited by a third party, and the paper warns against hardening either value into an acceptance rule.

Source: Section 4.2, p.4 to 5; Section 6, p.9; Limitations, p.10; Ethical Considerations, p.10
Maybe one configuration's answers really are less supported. Is the flag then wrong?

It may be a real gap, and the paper says so. The audit cannot separate a real quality gap from the way the label behaves on that configuration, and the verdict does not depend on which it is.

Either way the configuration tag alone reaches 0.811 while the best single published measurement reaches 0.637. If the gap is real, the accurate report is per configuration, not one pooled score. The flag describes one pooled score the authors computed; RAGBench does what it advertises.

Source: Section 5.2, p.7; Terms, p.3
If covering the tag barely moves your RL detector, why was one of its pooled scores flagged?

Because the flag and the measurements-only score answer different questions. The flag says what the tag alone could reach; the measurements-only score says whether the detector needs the tag column.

On the 134-run audit the tag alone reaches 0.838 against a headline of 0.911, and covering the tag moves the score from 0.911 to 0.884. That cannot show the detector uses no source information, since the logs differ by source too; with nearly all faulty runs in one framework the authors read direction, not magnitude, and nothing here says any framework causes failures.

Source: Section 4.2, p.5; Section 5.4, p.8
Does "Not flagged" or "Source carries nothing" mean the test set is fine?

No. Each is a reading, not a certificate. They say only that, for the source tag and measurements the auditor chose, the tag alone does not recover the headline.

They are silent on contamination, label noise, shortcuts inside a source and any source nobody encoded. On the judge audits the result says a pooled headline cannot borrow from judge identity under that design, not that individual judges are unbiased.

Source: Ethical Considerations, p.10; Section 5.3, p.8
What did you not test, and can the audit be gamed?

The listed repairs were not tested (listed, not tested); how a detector does on a source it has never seen was not measured; how common the problem is was not measured; scores were estimated in two different ways.

And yes: one informative engineered measurement could lift the headline over the floor while the source-only score stays high, so the ratio looks unremarkable although the source still dominates. That is why the paper asks for the numbers and not a pass mark.

Source: Appendix E.2, p.18 to 19; Limitations, p.10; Ethical Considerations, p.10