Skip to main content
The Benchmarks page (/benchmarks) aggregates all scores produced by sf bench:
StarForge benchmarks page

Benchmarks board. Empty until you run sf bench.

  • Rows = models / runs, columns = benchmarks (GSM8K / MMLU / C-Eval…). Each cell is score and sample count.
  • Best score in a column is green.
  • Filters: base model, method, runner, time range.
  • Click a score to open the eval job (logs and runner params).

​
Where the data comes from

sf bench evaluation jobs run lm-eval / evalscope inside the training container; scores are ingested idempotently (re-running the same run + benchmark overwrites the old score, never producing duplicate rows). Both runners obey the same score contract, so the same benchmark is comparable across runners.
Make it a habit to benchmark the base model before training, so the board has a reference frame for improvement margins. See Standard Benchmarks for usage.