A Mamba prefix resumes only at a snapshot
A cached conversation drawn as a token ribbon from 0 to 2,048 tokens, with Mamba state snapshots every 256 tokens. A new request matches the first 1,800 tokens. Full-attention KV exists for all 1,800 tokens, but the recurrent state exists only at snapshots, so the request resumes from the snapshot at 1,792 tokens and prefills tokens 1,792 to 1,800 again.
CACHED CONVERSATION · 2,048 TOKENS
0
512
1,024
1,536
2,048
teal ticks: Mamba state snapshots, every 256 tokens
NEW REQUEST · MATCHES 1,800 TOKENS
reused: 1,792 tokens, up to the last snapshot
prefilled again: 8 tokens from 1,792 to 1,800
Full-attention KV exists for every token. The recurrent state
exists only where a snapshot was saved.