LLaVA-OneVision — unified token budget and three-stage curriculum
shared visual-token budget ~4000 across three scenarios
single-image
AnyRes 9 tiles + thumbnail
each tile 384, 2x pool
182 tokens per tile
total 1820 tokens
perception + OCR + detail
multi-image
6 images at 384
no pooling, 729 per image
cross-image reasoning
total 4374 tokens
compare, count, diff
video
32 frames at 384
3x pool -> 81 per frame
temporal grounding
total 2592 tokens
events, action recog
three-stage curriculum + emergent skills
Stage SI
single-image only
AnyRes high-res
perception base
MMMU + OCR + detail
reverse order (video first)
loses 2-4 MMMU
per OneVision ablation
Stage OV
50% single
30% multi-image
20% video
unified budget
cross-scenario alignment
structural reasoning
Stage TT
task-specific mix
e.g., agent / doc
/ robotics / video
deployment tune
optional, preserves base
emergent
multi-camera
reasoning
set-of-mark
prompt
iPhone screen
agent
none trained