More answers per prompt, same learning in 24 steps
Reward gain over the first 24 steps, the mean of steps 16 to 23 minus the mean of steps 0 to 7, for 4, 8, and 16 answers per prompt. 4 answers: +0.053. 8 answers, three seeds: +0.050, +0.038, +0.098. 16 answers: +0.097.
0.00
0.12
reward gain over 24 steps
+0.053
4, one seed
+0.050
8, seed 1234
+0.038
8, seed 2
+0.098
8, seed 3
+0.097
16, one seed
answers per prompt
16 answers stayed inside the range of the three seeds at 8.