Deterministic inference repeats, and the gap stays
Logprob gap per step for four 12-step runs with seed 1. All four stay between 0.0145 and 0.0160. The two Triton runs start at the same value, 0.015404, and then drift apart.
0.0145
0.0150
0.0155
0.0160
0
3
6
9
11
step
logprob gap
step 0: both Triton runs gave 0.015404
default
deterministic, Triton
same, second run
deterministic, FlashInfer
The engine repeats itself. It still does not compute like the trainer.