CAIS four-risk framework + where practitioners pull the lever
1. malicious use
bad actor uses AI for harm
bioweapons synthesis · disinformation · cyberattacks
mitigations:
• hardcoded prohibitions (L17) · Llama Guard (L18)
• tool allowlist (L10, L11)
2. AI races
competitive pressure overrides safety
labs · companies · nations
mitigations:
• scaling policy with Risk Reports (L19, L20)
• external evaluation (L21)
3. organizational risks (practitioners pull this)
internal lab dynamics
safety culture · audit · defenses · infosec
mitigations:
• escalation paths without career cost
• independent audit, multi-layered defenses
4. rogue AIs
capable AI pursuing misaligned goals
grows with autonomy horizon (L1, L21)
mitigations:
• kill switches + canaries (L14)
• propose-then-commit (L15) · checkpoints (L16)
the three entities to know
CAIS (non-profit, San Francisco, 2022) — four-risk framework; 2023 extinction statement
CAISI (NIST center, US government) — voluntary agreements; unclassified capability evaluations
California SB-53 — first US state-level catastrophic-risk regulation if signed