Request rows point into one slot pool
Two ReqToTokenPool rows, request A and request B, each list KV slot indices. Both rows start with the same slots 1 to 4 for the shared system prompt, then point to their own slots for their questions. The KV pool strip below shows slots 1 to 4 stored once and used by both rows, and the free list hands out new slots.
REQTOTOKENPOOL ROWS
request A
1
2
3
4
5
6
request B
1
2
3
4
7
8
cells hold slot indices
positions 0 to 5
KV POOL SLOTS
0
1
2
3
4
5
6
7
8
9
slot 0: padding, never given out
slots 1 to 4: the shared system prompt, stored once and read by both rows
slots 5 to 8: each request’s own question tokens
slot 9: on the TokenToKVPoolAllocator free list