ColPali / ColQwen late-interaction retrieval
query side
tokenize query
embed each token
n query tokens -> n vectors
MaxSim per query token
for each q_i:
max over doc patches (q_i . p_j)
sum all q_i max scores
-> page score
O(|q| * |patches|) per page
but vectorized on GPU
multi-vector index stores
patches + MaxSim ranks
Vespa / Qdrant multi-vector / AstraDB
document side
render page (180 DPI)
ColQwen2.5-v0.2
patch tokenizer + LM head
patch embeddings
~2048 patches per page
dim 128 each
no pooling: preserve locality
DocPruner: keep 50%
accuracy drop < 0.5%
storage halved
per-patch bounding box kept
for evidence highlighting
Illuin Tech ColPali + 2026 DocPruner
synthesis + eval
top-5 pages
Qwen3-VL-30B
vision + text context
cited regions
viewer overlay
evaluation
ViDoRe v3 (retrieval)
M3DocVQA (multi-page QA)
content-class x approach
matrix: text / table / chart /
handwriting / equation
vs OCR-then-text baseline
publish side-by-side delta