encoder hidden states (kept, not discarded)
h1
h2
h3
h4
h5
Je
ne
suis
pas
chat
s_{t-1}
query
score + softmax
α_{t,i} = softmax(
score(s_{t-1}, h_i)
)
context c_t
= Σ α_{t,i} · h_i
weighted average
of encoder states
decoder step
outputs next token
no more fixed-size bottleneck. context reshapes every step.