Math Data Hurt Coding
Under a fixed training budget, DataSmith searched for training data that could improve
both math (GSM8K) and coding (HumanEval). Early in the search, simply adjusting the mix
of math and coding data improved both benchmarks. But as performance plateaued,
a tug-of-war emerged. Several changes that improved math hurt coding, even though the
coding data itself had not changed at all.
Obvious Fixes Failed
At first, DataSmith tried the natural responses, such as generating synthetic data to
target coding failures, improving coding data quality, and diversifying the math data
sources. However, when one benchmark improved, the other usually regressed.
The obvious fixes failed to produce gains on both at the same time.
The Contrarian Bet
After performance plateaued for several rounds, DataSmith recognized that its search had
narrowed around the same assumptions. It spawned a fresh strategist agent to reexamine
the failed experiments. Rather than assuming that coding regression meant the model
needed more or better coding data, the strategist proposed a different explanation.
Off-the-shelf math data might differ too much from the model’s own outputs, creating
a distributional shift that also disrupted coding.
That led to a counterintuitive proposal. To improve coding, add more on-policy math data.
That led to a counterintuitive proposal. To improve coding, add more on-policy math data.
Idea to Data
To validate this idea, the Autosynth agent was dispatched to serve the last champion
checkpoint and generate on-policy math data for the GSM8K training split. It sampled
seven rollouts per problem and kept at most two diverse solutions whose final answers
matched the gold label. The coding data was held fixed, isolating the
effect of the new math data on HumanEval performance.
600 problems→
7 solutions each→
verify answers→
diversity filter→
1,117 examples
Breaking the Ceiling
The resulting experiment is the orange point in the figure. It was the first run to
clearly break above the tradeoff seen in the earlier search. GSM8K held while improving
slightly, from 84.3% to 84.6%, while
HumanEval jumped from 81.1% to 84.8%. By improving coding without
sacrificing math, the run broke the cross-task tradeoff and became the new champion.
Math · GSM8K
84.31%→84.61%
+0.30 percentage points
Coding · HumanEval
81.10%→84.76%
+3.66 percentage points