[Defense] Add SDD

[luis-slyfield]

Changes

Add SDD Defence

Main additions:

• sdd.py: Main defence implementation (SDDConfig, SDD class, training logic)
• dataset.py: Dataset construction with cosine similarity filtering
• test_sdd.py: Basic sanity check unit test

Also modifies:

• names.py: Adds SDD to DefenseName enum
• defenses/__init__.py: Exports SDD and SDDConfig
• pyproject.toml: Adds sentence-transformers>=3.0.0 dependency

Testing

Luis's testing

Commit: 69dfdde

• Ran various iterations of training the defence, initially trying to replicate the paper results
• I was unable to replicate the performance stated in the paper, probably for the following reasons:
  • The paper uses LLaMA-Factory and LLaMA 2 is specifically trained to refuse harmful prompts in a specific format. SDD is designed and tuned in the paper for attacks that go through the chat template pathway.
  • Testing it in plain text gives much more modest results (and is also likely why it required a higher learning rate to show any results, as much bigger changes were needed to the weights).

With further iteration it would probably be possible to improve the results, but it feels like the dataset construction and defence fine tuning would have to be tweaked for each specific attack to get the best results.

Summary of experiments:

Variant	Format	LR	reject_prefix	Attack	Attack Samples	StrongREJECT (lower = safer)	MMLU-Pro (higher = smarter)	Notes
Baseline (no defense)	-	-	-	LoRA	3000	0.409	0.214	No defense applied
SDD v1 (Alpaca format)	### Instruction / ### Response	5e-7	No	LoRA	3000	0.401	0.229	Format mismatch with attack defense had no effect
SDD v2 (plain format)	plain	5e-7	No	LoRA	3000	0.378	0.207	Format fixed but lr too low; diagnostic showed identical model outputs
SDD v3 (plain, higher lr)	plain	2e-5	No	LoRA	3000	0.367	0.236	Best result: 10% reduction in attack success
SDD v4 (reject prefix)	plain	2e-5	Yes	LoRA	3000	0.381	0.207	Reject prefix hurt performance
Baseline (no defense)	-	-	-	Full param	64	0.148	0.264	Weak attack (only 64 samples)
SDD v3	plain	2e-5	No	Full param	64	0.140	0.250	Too few attack samples to distinguish

Tom's testing

After making some more tweaks to the setup, I tried replicating the paper's main result (Table 1) on Llama2-7b-chat using the settings from the LLM-finetune-Safety benchmark (Qi et al. 2023). Full details in scripts/sdd/README.md. Like Luis, I had to increase the learning rate (used Luis's 2e-5 rather than the paper's 5e-7).

Harmfulness (HS = Harmfulness Score 1-5, HR = Harmfulness Rate %):

Method	Initial	10-shot MFT	50-shot MFT	100-shot MFT
Vanilla	HS 1.13 / HR 1.0%	HS 4.94 / HR 96.3%	HS 5.00 / HR 99.7%	HS 4.86 / HR 91.0%
SDD	HS 1.00 / HR 0.0%	HS 1.58 / HR 13.7%	HS 4.87 / HR 93.0%	HS 4.84 / HR 89.7%

MMLU-Pro:

Method	Initial	10-shot MFT	50-shot MFT	100-shot MFT
Vanilla	23.2%	21.4%	12.5%	13.8%
SDD	20.9%	23.4%	3.2%	1.3%

Findings:

• SDD provides strong defense against 10-shot MFT (HR 96% → 14%)
• Self-degradation mechanism confirmed: MMLU-Pro collapses to 1-3% under
  50/100-shot MFT for SDD (vs 12-14% for vanilla). However, despite the MMLU-Pro collapste, the defense still has harmfulness at 50/100-shot (HR ~90%), unlike the paper's claimed 0%

See scripts/sdd/README.md for a full write-up

AI use: Claude Code with Opus 4.6 used extensively, as well as Sonnet/Opus 4.6 in chat and a small amount of Cursor Composer

[luis-slyfield]

Also a note on code quality; this is mostly from iterating with Claude Code to see if it works at all. I haven't put any effort into matching style etc with the rest of the repo, let me know if you'd like me to do this.

[tomtseng]

overall looks good, thanks! have some comments about a few places the implementation differs from the paper

[tomtseng]

The actual default set in the code below is learning_rate: float = 2e-5 instead of 5e-7, is this intended? if yes then maybe add a comment on why 2e-5 instead of 5e-7

[tomtseng]

Suggested change

# Resample with replacement if we need more samples than available

tiny nit: this comment seems redundant since there's already a comment about it four lines later

[tomtseng]

I made a few small tweaks (dataset change, chat templating) and tried to replicate the experiment from the paper, which uses a smaller attack (10–100 datapoints for 5 epochs)

I got the same thing you did where the paper's learning rate of 5e-7 didn't achieve anything, but using your learning rate of 2e-7 got modest results. It doesn't quite match the paper but it does seem to do something: SDD defends against the 10-datapoint attack, and SDD's benign capabilities degrade on the 50-datapoint and 100-datapoint attack.

I think this will be as close as we'll get since we've both looked at this so I'll merge this now