olmo_tap.experiments.robustness.build_attack_bank

Build a portable attack bank of transferable GCG suffixes on MedMCQA.

Three resumable phases:

1. Seed selection – pick –num-seeds validation examples by seed.

2. Suffix gen – run AmpleGCG on each seed, –num-return-seq candidates each.

3. Transfer score – test every candidate against all seeds (own + others)

   on OLMo-7B + prod security LoRA; tier-filter survivors.

Each phase persists incrementally. On re-run, phases resume from their last cached progress. Intended usage:

# smoke test (minutes)
pixi run -e cuda python -m olmo_tap.experiments.robustness.build_attack_bank \
    --num-seeds 3 --num-return-seq 2

# real run (hours)
pixi run -e cuda python -m olmo_tap.experiments.robustness.build_attack_bank

Functions

filter_and_save_bank(out_dir, scored, args)
main()
parse_args()
phase_1_select_seeds(out_dir, seed, num_seeds)
phase_2_generate_suffixes(out_dir, ...)
phase_3_score_transfer(out_dir, val_indices, ...)	Score every candidate against all seeds on the target; resumable.

olmo_tap.experiments.robustness.build_attack_bank.filter_and_save_bank(out_dir: Path, scored: list[dict], args: Namespace) → None

olmo_tap.experiments.robustness.build_attack_bank.main() → None

olmo_tap.experiments.robustness.build_attack_bank.parse_args() → Namespace

olmo_tap.experiments.robustness.build_attack_bank.phase_1_select_seeds(out_dir: Path, seed: int, num_seeds: int) → list[int]

olmo_tap.experiments.robustness.build_attack_bank.phase_2_generate_suffixes(out_dir: Path, val_indices: list[int], num_return_seq: int, num_beams: int) → list[dict]

olmo_tap.experiments.robustness.build_attack_bank.phase_3_score_transfer(out_dir: Path, val_indices: list[int], candidates: list[dict], shard_id: int, batch_size: int, max_seq_len: int) → list[dict]

Score every candidate against all seeds on the target; resumable.