Scale (Model Params @ Pretraining Tokens)
30B-A3B @ 1T
12B-A1.4B @ 0.44T
4B-A0.6B @ 0.19T
Benchmark group
Highlights
Baseline
Curated
CURATED − BASELINE
(POINTS)
0
100
Benchmark score
Enable JavaScript to select a scale and display benchmark scores.