regulated-domain RAG — ingestion, guard, cache, observe
ingestion
docling / Unstructured
ColPali for visual docs
role + jurisdiction tags
summaries + labels
pgvector + pgvectorscale
Tantivy BM25 side-index
payload: role, jurisdiction
eval
200-Q golden set
RAGAS faithfulness
citation correctness
Phoenix weekly drift
red-team suite
(PAIR + PII + off-domain)
alert: nDCG drop > 5%
conversation (LangGraph)
filter role + jurisdiction
hybrid retrieve + RRF
rerank bge-v2-gemma-2b
assemble cache-stable prompt
Claude Sonnet 4.7 synth
Llama Guard 4 output
Presidio PII scrub
citation enforcement
prompt-caching target: 60-80%
cache key = system+policy+context
Claude 4.5+ prompt-cache API
guardrails + observe
Llama Guard 4 input
jailbreak + PII probes
NeMo Guardrails v0.12
off-domain rails
Langfuse self-hosted
annotation queue
Arize Phoenix drift
weekly retrieval quality
cost report
prompt-cache hit rate
tokens / query breakdown
$ / query by stage
rerank / synth / guard
hit rate target: 60-80%
3-5x cost reduction
when prefix is stable
Harvey / Glean / Mendable 2026 shape