Language models and NLP EMNLP 2025
Spend eight picks two ways.
The paper reports that a few added safety examples leave a model less harmful when they are refusals of harmful requests, spread evenly across harm categories, than when they are drawn at random: 1.19 against 1.48 at 150 added, on an automated 0 to 4 harm score.
Figure 1Illustration
Circles (T1) are refusals of harmful requests. Columns A to H are made-up categories.
Press either button. Every press is a fresh draw.
| Rule | Categories covered | Refusals (T1) picked |
|---|---|---|
| At random | 6 of 8 | 4 of 8 |
| One refusal per category | 8 of 8 | 8 of 8 |
01 The problem
Supervised fine-tuning adapts a pre-trained model to a task. The paper starts from earlier work showing that even fine-tuning on benign data can erode the safety alignment a model already had (it cites Qi et al., 2024), and from the known repair: mix a few safety examples into the fine-tuning set. It prints no score for a model before fine-tuning, so neither does this page. What it does print is what a random handful of safety examples buys.
Figure 2Measured, Tables 1 and 2, Random row
150 added to 20,000
What the last step bought
Harm score: 1.84 to 1.48, down 0.36
Attack success rate: 38.67 to 27.39, down 11.28 points
Differences of two printed means, computed by this page.
| Safety examples added | 0 | 50 | 100 | 150 | 250 | 350 | 500 | 1000 |
|---|---|---|---|---|---|---|---|---|
| Harm score, 0 to 4 | 2.37 | 2.16 ± 0.06 | 1.84 ± 0.09 | 1.48 ± 0.11 | 0.95 ± 0.09 | 0.75 ± 0.06 | 0.64 ± 0.04 | 0.57 ± 0.03 |
| Attack success rate, percent | an estimated 58 | 50.73 ± 2.48 | 38.67 ± 3.32 | 27.39 ± 3.67 | 14.36 ± 2.13 | 10.54 ± 1.40 | 8.70 ± 0.91 | 7.12 ± 0.88 |
MeasuredThe 0 stop comes from the text of Section 5, not from a table: 2.37 with no safety data, and what the paper calls “an estimated 58%” attack success rate. It prints no interval for either, and does not say how the second was estimated.
The paper reports that over-rejection, refusing harmless prompts such as “How can I kill a Python process?”, rises as more safety examples are added (the paper’s Figure 6, Appendix C.1). That figure prints no data labels, only axis ticks, so this page reads no values off it and draws nothing here. The paper also notes that safety data is expensive to collect and often needs manual curation, and that a larger safety set costs more compute in training.
Earlier fixes, the paper says, rely on complex optimisation, filter by embeddings, or fall back on a random draw, and “often do not address a fundamental question: which safety examples are most effective, and why?” The paper calls its answer “a first step”.
02 The idea
The paper states this as a hypothesis with two axes: an example’s instruction and response behaviour, and its contribution to diversity across harm categories. Its sampling strategies are those two choices multiplied. Learn the four shapes first, then set the two switches yourself.
Figure 3Illustration
Select a cell to read the paper’s definition of that type.
“The model receives a harmful instruction and produces a safe refusal.” The paper’s printed example, in neutral words: a request for help with a theft, answered with a refusal that explains why and points to legal alternatives.
kept by the B strategies In the paper’s preliminary analysis, 50 examples of one type at a time over 10 random trials, T1 carries the lowest harm values while helpfulness stays level (the paper’s Figure 2).
“The model receives a harmful instruction and provides a safe but compliant response.” In neutral words, the paper’s example is a request to research religious customs that other people find intolerable, answered with a careful explanation.
The abstract’s example of a difference in behaviour is “refusal vs. explanation”; it does not say which types it means.
“The model receives a safe instruction and unnecessarily refuses to respond.” In neutral words, the paper’s example is a question about the law on store robbery in one US state, answered with a warning that robbery is a crime and advice to follow the law; the paper files it under unnecessary refusal.
This is the behaviour the over-rejection test counts in a fine-tuned model.
“The model receives a safe instruction and provides a compliant, safe response.” An ordinary answer to an ordinary request.
In the paper’s Figure 2, 50 added T4 examples carry the highest pair of harm values of the four types, 2.29 and 2.36. For comparison, made by this page: the paper’s text gives 2.37 for no safety data.
Measured, labels of the paper’s Figure 2
| Type added | Harm Score labels | Help Score labels |
|---|---|---|
| T1 | 2.03 and 2.12 | 4.99 and 5.29 |
| T2 | 2.23 and 2.30 | 5.04 and 5.24 |
| T3 | 2.18 and 2.24 | 4.98 and 5.22 |
| T4 | 2.29 and 2.36 | 5.04 and 5.25 |
The figure prints two numbers at each error bar, one below and one above, and does not say what they are: they read as the two ends of an interval, no mean is printed, and the caption is silent. They are shown here exactly as printed and never as means. The four Help pairs overlap. The paper’s reading: helpfulness “remains stable across all data types”.
Figure 4Illustration
Circles (T1) are refusals of harmful requests. Columns A to H are made-up categories.
Closer to a column’s centre line means more typical of that made-up category. Height inside a band means nothing. Filled with a ring: picked. Hatched bands are not eligible. Pool, ours: the paper does not print how its safety pool is spread over harm categories, or name them.
The paper’s name for this pair of choices
Random
The baseline: one uniform draw from the whole pool, category and behaviour ignored.
varies between runs
Try “At random” with Draw again a few times, then “Evenly per category”, then add “Refusals only”, then “Most typical”.
The last 24 draws under this rule
Column height: categories covered, of 8. Square: filled when a draw picked different dots from the one before. Rules that involve chance give a different sample each run, which is why Tables 1 and 2 print a plus or minus for Random, SSS and SSS-B and a single value for the deterministic rows.
With scripts on, this figure is a board of 144 dots that you sample with the paper’s rules. Without scripts, here is what the two switches make.
| How the budget is spread | Any behaviour | Refusals of harmful requests only (T1) |
|---|---|---|
| At random | Random, the baseline | not one of the seven rows |
| Evenly per category | SSS | SSS-B |
| Most typical per category | PSS | PSS-B |
IllustrationThe paper’s selection rules run on an invented pool of 144 dots in 8 made-up categories. No text, no model, no fine-tuning. The numbers are facts about the sample on screen, not safety results.
The paper’s. The rules: a uniform draw from the whole pool (Random), a uniform draw inside each category (SSS), the pairs nearest each category’s embedding centroid (PSS), and the same two restricted to T1 (SSS-B, PSS-B). Cossim and Cossim-B need a second, labelled dataset and are left to the pipeline in section 03.
Ours. The pool, its categories A to H, one category per dot, the two pool presets, dealing one pick per category in turn, and centroids taken over every dot of a category.
What the toy does not show. In this toy, a random draw rarely misses a category once the budget is large: at 48 picks the chance is 1 in 100 in the uneven pool and under 1 in 100 in the even one. In Table 1 the gap between Random and SSS-B is 0.25 and 0.29 score points at 100 and 150 and 0.04 or less from 350 on (differences of printed means, computed by this page; other rows differ, for example Cossim prints 0.47 against 0.57 at 1000). The paper’s words: “The benefits of structured sampling are especially pronounced at small sample sizes.” The toy does not show that coverage causes this, and the paper does not test it. The only mechanism the paper calls “unverified” is why refusals work (Section 7); of the deterministic rules it says “there remains a gap in understanding which specific data characteristics drive these gains” (Section 5).
Figure 5Measured, Tables 1 and 2
Printed cells at 150 added safety examples, strategy SSS-B. Lower is safer.
Harm score, 0 to 4 Table 1
Attack success rate, percent Table 2
What a rule does to a sample is arithmetic, shown above. What it does to a model is a measurement, printed here. The toy’s budget of 8 to 48 dots is never mapped onto these budgets.
03 Follow one budget
By this page’s count, 150 is the budget the paper’s results text names most often (six times in Section 5), and it is one of the two budgets (150 and 500) at which SSS-B is the bold cell of both main tables. Change the budget or the strategy and the steps re-word themselves. Every number in this section is a printed cell or a number from the paper’s text, never a toy value.
Figure 6Illustration
Diagram drawn for this page from Sections 3 and 4 and Appendix A of the paper; it is not the paper’s Figure 1. The shapes on the wires are symbols, not counts. Wires differ by dash and width, not by colour.
A random sample of 20,000 instruction and response pairs from the cleaned Alpaca set stands in for an everyday fine-tuning job with no explicit safety intervention. For a model fine-tuned with no safety data, the paper’s text gives a harm score of 2.37 on the 0 to 4 scale.
2,483 pairs from earlier work (Bianchi et al., 2024), built by turning red-teaming prompts into instructions and writing safe responses with a language model. Labelled safe as a whole, the set still mixes several kinds of example: the four types of Figure 3 on this page.
A language model assigns one or more harm categories, prompted with the definitions of an earlier taxonomy (Ji et al., 2023). A classifier, WildGuard, sorts each pair into T1 to T4. Both labels are automatic.
Cossim and Cossim-B do not use the language model’s category labels: each pair takes the category whose outside reference set it is most similar to (Section 4.3).
Only T1 pairs, refusals of harmful requests, stay in play. The paper’s reason is its preliminary analysis: adding 50 examples of one type at a time, T1 carried the lowest harm values (the paper’s Figure 2).
This strategy skips the filter. Every behaviour type stays in play, and the wire takes the detour under the box.
SSS and SSS-B draw uniformly inside each harm category, so every labelled category is covered by construction.
PSS and PSS-B embed every pair, average each category’s embeddings into a centroid, and keep the pairs with the highest cosine similarity to it: the most typical of each category.
Cossim and Cossim-B score each pair by its average cosine similarity to an outside set of pairs labelled with one category only, give the pair its best-scoring category, and keep the top scorers per category. The outside set “may include unsafe items” and is used only to define the space.
Random uses no categories at all: one uniform draw from the whole pool.
The picks join the base set: a small stream into a large one. LLaMA 2 7B is fine-tuned with LoRA for 3 epochs at batch size 128, on L40S GPUs.
Harm: 700 harmful prompts, answers scored 0 to 4 by a harm reward model. Attack: the attack-enhanced prompts of SALAD-Bench, each answer judged safe or unsafe by MD-Judge. Over-rejection: harmless but sensitive prompts from XSTest, refusals counted by WildGuard. Helpfulness: ordinary prompts, scored by a reward model.
Select any box in the drawing to read what it is in the paper, what goes in and out, and what the paper does not say. Arrow keys move between boxes.
Section 4.2, Appendix B
Introduced by Bianchi et al. (2024). The appendix notes that although the set is labelled safe, it includes “potentially harmful prompts with safe refusals and safe completions”.
Not said: how many pairs fall in each behaviour type. The paper’s Figure 5 shows T1 as by far the tallest bar, against axis ticks from 0 to 1,500 samples, and prints no exact counts.
Section 3.2
Safety datasets often lack category labels, so the paper makes them with a language model before any category-aware sampling.
Not said: which model did the labelling, its prompt, the names of the categories, how many were used for sampling, and any check of label quality.
Section 3.3
The paper’s premise: “not all safe responses are equally effective for alignment”. The same classifier later counts refusals in the over-rejection test.
Not said: how accurate these labels are.
Section 4.3
Used only by Cossim and Cossim-B, which take a pair’s category from this set, not from the language model’s labels: “We assign the category cj with the highest similarity score to the candidate”. It “is used only to define the embedding space and may include unsafe items”; nothing from it is added to training. The paper calls Cossim useful “when no LLM-based labels are available or when deterministic, interpretable scoring is desired”.
Section 3.3, the paper’s Figure 2
Used by SSS-B, PSS-B and Cossim-B. In the paper’s words the variants “isolate the impact of refusal-type behavior while preserving semantic diversity across harm categories”.
Not said: why refusals work. Of refusals, the paper says “the underlying mechanism remains unverified” (Section 7).
Sections 3.2, 4.1 and 4.3
SSS: “uniformly sample examples from each harm category”. PSS: embed every pair with Sentence-BERT (all-mpnet-base-v2), average each category’s embeddings into a centroid, keep the top k per category by cosine similarity to it. Cossim: score each pair by its average cosine similarity to an outside labelled set, assign the best-scoring category, keep the top k per category. Random skips all of this.
Not said: how SSS splits a budget that does not divide evenly, what any rule does when a category runs out, and how instruction and response become one embedding.
Tables 1 and 2
The paper tries seven budgets: 50, 100, 150, 250, 350, 500 and 1000. Cossim and PSS are deterministic and print single values; Random and SSS “require multiple runs”, so their cells print a mean and an interval.
Section 4.2
Used “to simulate standard instruction fine-tuning without explicit safety interventions”. The limitations section says about 20k samples “may not capture all dynamics in other data settings”.
Sections 1 and 7
The paper describes the additions as “a small set of high-impact demonstrations”. Selection happens once: it calls its methods “one-shot sampling” and leaves strategies that adjust during training to future work.
Section 4.1
Low-Rank Adaptation (LoRA), 3 epochs, batch size 128, “consistent with prior work”, on L40S GPUs with 48GB of memory.
Not said: base or chat checkpoint, LoRA settings, learning rate, decoding settings, number of GPUs, or any training time.
Section 4.4, Table 1
The scorer is the Harm Reward Model of Bianchi et al. (2024). It is an automated judgement, and the paper says such scorers “may not fully capture subjective perceptions of safety”.
Section 4.4, Table 2
The judge is the MD-Judge model, which gives each response a binary safety label.
Not said: the size of the subsets used.
Section 4.4, the paper’s Figure 6
WildGuard classifies each response as a refusal or not.
Not said: any value. The paper’s Figure 6 draws four curves that end higher than they start. It prints no data labels, only axis ticks, so this page reads no values off it.
Appendix A.2, the paper’s Figure 2
The paper reports that helpfulness “remains stable across all data types” and shows it only for the 50-example runs of the paper’s Figure 2.
Not said: the scale of the score, and any helpfulness number per strategy, budget or model.
Measured, Tables 1 and 2Printed cells at 150 added safety examples, strategy SSS-B. Lower is safer.
Harm score, 0 to 4 Table 1
Attack success rate, percent Table 2
Over-rejection and helpfulness
No value printed for any strategy and budget: the paper’s Figure 6 draws curves with no data labels, only axis ticks, so this page reads no values off it, and helpfulness appears only in the paper’s Figure 2.
IllustrationThe drawing is this page’s own. The numbers in the card are measured: printed cells of Tables 1 and 2 and two numbers from the text of Section 5.
04 The evidence
Both tables come from one setting: LLaMA 2 7B fine-tuned with LoRA on 20,000 ordinary instructions plus the picks, drawn from a pool of 2,483. Lower is safer in both. Read every row against Random, the baseline from prior work. At the same budget, SSS-B prints lower than Random in every cell; each of the other five rules prints level or higher somewhere (computed by this page from Tables 1 and 2; not a significance test). The amount added still moves the score far more than the way of choosing.
Figure 7Measured, Table 1
One budget at a time, from zero
computed At 150 added: all 6 strategies print lower than Random. Lowest: SSS-B, 1.19 ± 0.14.
computed SSS-B is the only strategy that prints a lower value than Random in all 14 cells of Tables 1 and 2.
| Strategy | 50 | 100 | 150 | 250 | 350 | 500 | 1000 | Lower at |
|---|---|---|---|---|---|---|---|---|
| Cossim | lower | higher | lower | lower | lower | higher | lower | 5 of 7 |
| SSS | lower | lower | lower | lower | lower | level | lower | 6 of 7 |
| PSS | lower | lower | lower | level | higher | higher | higher | 3 of 7 |
| Cossim-B | lower | lower | lower | lower | lower | higher | lower | 6 of 7 |
| SSS-B | lower | lower | lower | lower | lower | lower | lower | 7 of 7 |
| PSS-B | lower | lower | lower | lower | higher | level | higher | 4 of 7 |
lower than Random level higher
| Strategy | 50 | 100 | 150 | 250 | 350 | 500 | 1000 |
|---|---|---|---|---|---|---|---|
| Random Sampling (Baseline) | 2.16 ± 0.06 | 1.84 ± 0.09 | 1.48 ± 0.11 | 0.95 ± 0.09 | 0.75 ± 0.06 | 0.64 ± 0.04 | 0.57 ± 0.03 |
| Cossim | 2.04, lowest in this column, computed | 1.91 | 1.29 | 0.73, lowest in this column, computed | 0.71 | 0.67 | 0.47, lowest in this column, computed |
| SSS | 2.13 ± 0.05 | 1.82 ± 0.20 | 1.32 ± 0.06 | 0.88 ± 0.07 | 0.73 ± 0.04 | 0.64 ± 0.04 | 0.53 ± 0.02 |
| PSS | 2.06 | 1.52 | 1.23 | 0.95 | 0.81 | 0.71 | 0.60 |
| Cossim-B | 2.06 | 1.39, lowest in this column, computed | 1.44 | 0.83 | 0.64, lowest in this column, computed | 0.69 | 0.52 |
| SSS-B | 2.07 ± 0.04 | 1.59 ± 0.13 | 1.19 ± 0.14, lowest in this column, computed | 0.80 ± 0.08 | 0.72 ± 0.04 | 0.61 ± 0.06, lowest in this column, computed | 0.53 ± 0.04 |
| PSS-B | 2.09 | 1.48 | 1.20 | 0.89 | 0.90 | 0.64 | 0.61 |
Bold and underlined: bold in the paper, which says bold numbers “denote the best results”. ▽ lowest in its column, computed by this page; in this table the two agree in every column. The two heavier rules between rows are the paper’s; it prints no group names. ↓ lower is safer in every column.
What the plus or minus is. The captions say “mean and confidence intervals” over “multiple runs” for Random and SSS (the SSS-B row prints them too); the paper states 10 trials for its random baselines and does not state the confidence level. Rows without an interval are deterministic single runs. A lower mean is not a significance test: several intervals overlap, for example 2.07 ± 0.04 against 2.16 ± 0.06 at 50 in Table 1. With no safety data the paper reports 2.37 (Section 5 text).
Figure 8Measured, Table 2
One budget at a time, from zero
computed At 150 added: all 6 strategies print lower than Random. Lowest: SSS-B, 20.39 ± 2.95.
computed SSS-B is the only strategy that prints a lower value than Random in all 14 cells of Tables 1 and 2.
| Strategy | 50 | 100 | 150 | 250 | 350 | 500 | 1000 | Lower at |
|---|---|---|---|---|---|---|---|---|
| Cossim | lower | higher | lower | lower | lower | higher | lower | 5 of 7 |
| SSS | lower | lower | lower | lower | higher | lower | lower | 6 of 7 |
| PSS | lower | lower | lower | higher | higher | higher | higher | 3 of 7 |
| Cossim-B | lower | lower | lower | lower | lower | higher | lower | 6 of 7 |
| SSS-B | lower | lower | lower | lower | lower | lower | lower | 7 of 7 |
| PSS-B | lower | lower | lower | lower | higher | lower | lower | 6 of 7 |
lower than Random level higher
| Strategy | 50 | 100 | 150 | 250 | 350 | 500 | 1000 |
|---|---|---|---|---|---|---|---|
| Random Sampling (Baseline) | 50.73 ± 2.48 | 38.67 ± 3.32 | 27.39 ± 3.67 | 14.36 ± 2.13 | 10.54 ± 1.40 | 8.70 ± 0.91 | 7.12 ± 0.88 |
| Cossim | 41.62, lowest in this column, computed | 39.03 | 22.58 | 11.28, lowest in this column, computed | 10.38 | 10.01 | 5.49, lowest in this column, computed |
| SSS | 49.49 ± 1.84 | 37.06 ± 4.50 | 23.40 ± 1.08 | 13.32 ± 1.94 | 10.69 ± 0.78 | 8.42 ± 0.61 | 6.37 ± 0.61 |
| PSS | 48.11 | 28.56 | 23.90 | 14.58 | 11.80 | 9.64 | 8.72 |
| Cossim-B | 46.29 | 24.81, lowest in this column, computed | 26.30 | 13.18 | 10.45 | 9.19 | 5.88 |
| SSS-B | 46.94 ± 1.61 | 31.42 ± 4.09 | 20.39 ± 2.95, lowest in this column, computed | 11.87 ± 1.49 | 9.82 ± 1.04, lowest in this column, computed | 7.79 ± 0.93, lowest in this column, computed | 6.25 ± 0.73 |
| PSS-B | 46.29 | 27.31 | 21.21 | 13.05 | 15.18 | 8.37 | 6.67 |
Bold and underlined: bold as printed; this caption does not say what bold means. ▽ lowest in its column, computed by this page. In the 50 column the paper’s bold is on 46.29 (Cossim-B); the lowest value in that column is 41.62 (Cossim), and PSS-B prints the same 46.29 without bold. ↓ lower is safer in every column.
What the plus or minus is. The captions say “mean and confidence intervals” over “multiple runs” for Random and SSS (the SSS-B row prints them too); the paper states 10 trials for its random baselines and does not state the confidence level. Rows without an interval are deterministic single runs. A lower mean is not a significance test: several intervals overlap, for example 2.07 ± 0.04 against 2.16 ± 0.06 at 50 in Table 1. With no safety data the paper reports “an estimated 58%” (Section 5 text).
05 Other models, and the price of more
The appendix repeats the comparison of Random and SSS-B on LLaMA3, LLaMA3-Instruct, Qwen2.5-Instruct and Mistral: two strategies, two budgets, the harm score only. Then it says what a bigger budget costs, in words and in curves without data labels.
Figure 9Measured, Table 4
| Method | Llama3 | Llama3 Instr | Qwen | Mistral |
|---|---|---|---|---|
| Size 50 | ||||
| Random | 2.09 ± 0.05 | 1.28 ± 0.11 | 1.47 ± 0.05 | 1.21 ± 0.09 |
| SSS-B | 2.00 ± 0.04 | 1.14 ± 0.05 | 1.31 ± 0.07 | 1.01 ± 0.08 |
| Size 100 | ||||
| Random | 1.80 ± 0.08 | 0.83 ± 0.05 | 1.09 ± 0.07 | 0.72 ± 0.05 |
| SSS-B | 1.57 ± 0.08 | 0.70 ± 0.04 | 0.97 ± 0.05 | 0.70 ± 0.05 |
Cells as printed, mean and interval. Table 4 prints no bold marks. Lower is safer. The caption and Appendix C.2 call the metric “harmfulness scores”; that this is the 0 to 4 Harm Reward Model score of Table 1 is this page’s reading of Section 4.4, the paper does not restate the scale here.
computed SSS-B prints the lower mean in 8 of 8 pairs. In 4 of them the printed intervals overlap or touch (Llama3 at 50, Llama3 Instr at 50, Qwen at 100 and Mistral at 100); the paper does not state the confidence level.
The paper reports that as more safety examples are added, models refuse more harmless prompts, in all four panels it plots: Random and the three refusal-based rows PSS-B, SSS-B and Cossim-B (the paper’s Figure 6; the PSS-B and Cossim-B curves each dip once on the way up). The figure prints no data labels, only axis ticks, so this page reads no values off it and draws none. The paper prints no comparison of over-rejection between strategies at the same budget.
In a supplementary run the authors rewrote the responses of BeaverTails pairs marked unsafe into refusals with GPT-4o mini, about 3,000 examples, and compared random draws from that set with random draws from the pairs marked safe. The paper reports a lower attack success rate at every sample size (the paper’s Figure 7, which prints no data labels, only axis ticks). As the paper sets it up: the refusal pool keeps the instructions of the pairs marked unsafe and replaces their responses with refusals written by GPT-4o mini; the baseline pool is every pair marked safe, which the paper says holds a mix of T1 to T4, with the original responses. The paper reads the result as support for its hypothesis about refusal behaviour; it reports no run in which only the behaviour type changes.
Building that refusal set took “about 2.27M input tokens and 485K output tokens per split, costing approximately $0.70 and taking around 5 hours per version to generate” (Appendix C.3; “per split” and “per version” are not defined). The paper measures no training cost.
The takeaway
A way to choose which safety examples to add: sort them by behaviour, keep the refusals, and spread the budget across harm categories. The paper tests six strategies built from these two choices, three of which keep only refusals, against a random draw at seven budgets.
At 150 added examples, 1.19 against 1.48 on the harm score and 20.39 against 27.39 percent attack success. Computed by this page: the gap to Random is widest at 100 to 250 added examples (0.25, 0.29, 0.15 score points) and is 0.04 or less from 350 on.
That choosing beats adding more: a random draw of 250 prints lower than any strategy at 150. With a large budget the choice matters less, and the paper reports that the model starts refusing things it should not.
06 Office hours
Written by Tian's agent while Tian slept and Pepper supervised. Pepper accepts no liability either.
Each answer says where in the paper it comes from, so you can check the agent's work.
No, and the tables show it. A random draw alone goes from 2.16 at 50 examples to 0.57 at 1,000, and Random at 250 (0.95) prints lower than every strategy at 150. What the paper supports is narrower: at the same budget, SSS-B prints lower than Random at every size, by 0.25 to 0.29 score points at 100 and 150 and by 0.04 or less from 350 on. Its reasons for keeping the budget small are that safety data is costly to collect and that more of it makes the model refuse harmless prompts.
Source: Table 1; Section 1; Appendix C.1. The differences are computed by this page from printed means.The paper does not know, and says so: refusals “may provide a clear and direct safety signal”, but “the underlying mechanism remains unverified”. Its evidence is empirical: 50 examples of each type, one type at a time, over 10 random trials, with T1 carrying the lowest harm values in the paper’s Figure 2 and helpfulness level; and the supplementary BeaverTails run. It lists mixed ratios of behaviour types as future work.
Source: Section 3.3, the paper’s Figure 2, Appendix C.3, Section 7Models did, at every stage. A language model assigned the harm categories, WildGuard assigned the behaviour types and counted refusals, a harm reward model gave the 0 to 4 scores, and MD-Judge labelled attack responses safe or unsafe. The paper names this as a limitation, says these tools “are not a substitute for human judgment”, and mentions a sanity check against human annotations without describing it. It does not name the labelling model or report how accurate any of these labels are.
Source: Sections 3.2, 3.3, 4.4, 7, 8 and 9Helpfulness: the paper reports it “remains stable”, but shows it only for the preliminary analysis (50 examples of one type at a time); there is no helpfulness number per strategy, budget or model. Over-rejection: the paper reports that it rises with the amount added in all four panels of the paper’s Figure 6. The four panels share one y axis (labelled XSTest score, tick labels from 0.00 to 0.35) and carry no data labels, so strategies can be compared only by eye, the paper prints no such comparison, and this page reads no numbers off curves. Its conclusion calls SSS-B “a practical balance between safety improvements and over-rejection risks”.
Source: Appendix A.2, the paper’s Figure 2, Appendix C.1, the paper’s Figure 6, Section 6One base set of about 20,000 Alpaca pairs and one 2,483-pair pool, plus the supplementary BeaverTails run. LoRA only. One main model, with four others checked at two sizes for two strategies on one metric. The only baseline is a random draw: the paper calls its own approach complementary to the filtering methods it discusses, SAFT and SEAL, and does not run them. Random baselines use 10 trials; the deterministic strategies are single runs; the confidence level of the intervals is not stated. The taxonomy is fixed, selection happens once before training, and there is no full human evaluation.
Source: Sections 2, 4, 7 and 8; Appendix C.2; captions of Tables 1 and 2