PAIR: attacker + judge loop
Attacker LLM (A)
goal G + history
propose prompt p_k
in-context feedback
previous refusals seen
Target LLM (T)
receive p_k
emit response r_k
black-box only
no gradients needed
Judge LLM (J)
score (p_k, r_k)
goal satisfaction?
halt if score >= thr
else: feed back to A
prompt
response
history updated (k < K)
related automated attacks
PAIR (black-box, natural-language) success < 20 queries on GPT-3.5-turbo, Llama-2-7B.
GCG (white-box, token gradient) universal adversarial suffixes; unreadable; transferable.
AutoDAN (evolutionary) genetic search; readable; moderate ASR.
TAP (tree of attacks + pruning) PAIR-branching; higher ASR at higher compute.
PAP (persuasive adversarial prompts) human persuasion templates; strong on HHH-aligned models.
benchmarks: JailbreakBench (100 behaviours), HarmBench (510 behaviours, 18 attacks vs 33 models).