Patch-n'-pack: variable-resolution images into one sequence
three images, three shapes
receipt
600x1500
chart 1280x720
16:9
screen
phone
native resolution, patch 14
patchify each
packed sequence
concatenate all images' patches
receipt tokens (n_0)
chart tokens (n_1)
phone tokens (n_2)
cu_seqlens = [0, n_0, n_0+n_1, N]
FlashAttn varlen: no padding, no dense mask
N = n_0 + n_1 + n_2 (zero waste)
block-diag
block-diagonal mask
each image attends only within itself
four strategies for any-resolution vision
square resize
resize to 336x336
fixed 576 tokens
loses OCR, squishes
text, wastes pad
baseline, pre-2024
LLaVA-1.5 used this
AnyRes tiling
tile MxN + thumbnail
frozen encoder at 336
fidelity + global ctx
expensive past 3x3
LLaVA-NeXT (2024)
M-RoPE + native
3D pos (t, r, c)
arbitrary HxWxT
no position table
min/max pixel caps
Qwen2-VL / Qwen3-VL
NaFlex single ckpt
pick 256/729/1024
per-task at inference
patch-n'-pack train
fractional pos embed
SigLIP 2 (2025)
2026 open-VLM default