three embedding modes — same passage, three vectors
"Apple released the first iPhone on June 29, 2007."
dense
one vector, fixed size
[0.23, -0.11, ..., 0.08]
dim = 1024 (BGE-M3)
cosine similarity ranks
passages by meaning
Matryoshka option:
truncate to 256 dim
≈ 1% accuracy loss
6x storage savings
sparse (SPLADE-style)
one weight per vocab token
{iPhone: 2.1, Apple: 1.8, ...}
mostly zeros, |vocab| slots
learned BM25 —
term weights from transformer
strength:
keyword-heavy queries,
named entities,
rare terms
multi-vector (ColBERT)
one vector per token
[v_Apple, v_released, ...]
(n_tokens, 128)
MaxSim: for each q-token
take best d-token match
tradeoff:
larger index,
better recall on long
and specialized queries
BGE-M3 emits all three from one forward pass — fuse with weighted sum or RRF