Managed LLM platforms — three strategies
AWS Bedrock — marketplace
Claude · Llama · Titan · Cohere
Mistral · Stability · AI21
one IAM, one CloudWatch surface
Application Inference Profiles
cleanest FinOps attribution
tag team / product / feature
per-profile cost without post-proc
TTFT P50 ≈ 75 ms
on-demand shared capacity
PT available $21-$50/hr
bet: optionality over any one model
frontier rotates — keep the door open
BAA in most regions, VPC endpoints
guardrails built-in
Azure OpenAI — exclusive + PTU
GPT-4 / 4o / 5 / o-series
DALL-E, Whisper, fine-tuning
no non-OpenAI models
Provisioned Throughput Units
dedicated capacity, hourly billed
up to 70% savings at high util
break-even ≈ 40-60% utilization
TTFT P50 ≈ 50 ms (PTU)
25 ms better than Bedrock
gap shrinks on on-demand
bet: OpenAI stays the frontier
regulated-industry default
HIPAA, SOC 2, ISO 27001
EU data residency available
Vertex AI — Gemini-first
Gemini 2.5 Pro / Flash / Nano
1M-token context window
Model Garden for 3rd-party
BigQuery billing export
project-per-team + labels
arbitrary SQL on cost data
more work, more flexibility
TTFT P50 ≈ 60 ms
on-demand, shared
provisioned per-SKU, less public
bet: multimodal long-context
1M context is the differentiator
Model Garden hedges third-party
GCP compliance inherits
two-provider minimum — Claude from one, GPT from another, gateway failover between
single-vendor lock-in is the expensive mistake in 2026 — frontier rotates monthly