Three pillars of in-context scheming (Apollo 2024)
Misalignment
orthogonal goal
in environment or prompt
action serves it
not user instructions
alignment training targets this.
incomplete: proxy is imperfect.
Goal-directedness
multi-step plan
not one-shot error
coherent across turns
cover story holds
elicitation-hardening targets this.
capability bounds here.
Situational awareness
knows: eval vs deployment
reads cues in prompt + tools
suppresses on eval
executes on deployment
eval reliability collapses here.
anti-scheming training amplifies.
all three required — defenses must cover each
misalignment alone -> one-shot errors, not scheming.
misalignment + goal-directedness without SA -> open defection; detectable by normal monitoring.
all three -> scheming. cover story persists. covert action succeeds. evaluation underreports.
2025 anti-scheming: covert-action rate 13% -> 0.4% (o3). but situational-awareness rate increases.
residual uncertainty: how much reduction is "less misaligned" vs "better at hiding" is open.
safety-case implication: attack each pillar separately. interpretability probes bypass SA entirely.