LLM security — four layers, zero-trust by default
secrets (vault)
· HashiCorp Vault
· AWS Secrets Manager
· Azure Key Vault
· GCP Secret Manager
IAM role + short-lived
OIDC for CI/CD
rotation ≤ 90 days
PII scrubbing
· Presidio / spaCy NER
· regex (SSN, email, phone)
· consistent tokenization
· Mesh approach
same value → same placeholder
preserves semantics for LLM
reverse mapping optional
network egress
whitelist only:
api.openai.com
api.anthropic.com
vault endpoints
vector DB
DNS allowlist only
prevents tunneling exfil
audit log
append-only
per call:
· user / tenant
· prompt/resp hash
· tokens + cost
· guardrail trips
SOC2: 1y
HIPAA: 6y
gateway pattern (Phase 17 · 19)
app → gateway → model provider
gateway pulls credentials from vault at runtime
rotate in vault → apps pick up in minutes; no redeploys
the 2026 incident driver
Vercel supply-chain attack: compromised CI/CD credentials
exfiltrated env vars across thousands of customer deployments
CI/CD creds are prod-equivalent; store in vault, scope narrowly, rotate aggressively