StarForge

One control plane for the whole LLM post-training lifecycle: environments, training, evaluation, serving and deployment — across any framework and any compute backend.

StarForge is the control plane for everything around a post-training job. It governs who may run what, on which approved runtime and data, on which GPUs, under which quota — and carries the result back: metrics, logs, evaluations, artifacts, a model version, a deployment, and the production traffic that becomes the next version’s training data.

It is deliberately neutral about two things. Which framework trains the model — NeMo-RL, verl, TRL, OpenRLHF, or your own — and which backend runs it: Docker, a bare-metal agent, KubeRay or Slurm. One JobSpec expresses the intent; the platform carries it to any of them.

What it does not do is own what runs inside the job. That belongs to the training framework and to your code.

uv tool install starforge-core
sf init my-lab && cd my-lab
sf new my-grpo --method nemo-rl/grpo
sf submit my-grpo --profile h200:8

Four commands from an empty directory to a GRPO run on a GPU cluster, with the loss curve streaming into a web console. Your laptop never holds a cluster credential.

Quickstart

Install, log in, submit, and watch the first data point land on the chart.

15 minutes

Core concepts

Experiment, run, recipe, profile, executor — the nouns everything else uses.

Read first

What it does

StarForge is a control plane for the whole post-training lifecycle. It parses, validates, stores, schedules and distributes work — and deliberately does not own what runs inside a job. Training frameworks own that.

Submit	Declare an experiment in YAML, validate it on your laptop, send it to a cluster you never log in to
Run	NeMo-RL, verl, TRL, OpenRLHF or your own trainer, on Docker, a bare-metal agent, KubeRay or Slurm
Watch	Loss, reward, KL, GPU utilisation, live logs, validation samples, and a diagnosis when a curve goes wrong
Score	GSM8K, MMLU, C-Eval and others through standard harnesses — or your own benchmark pack, or an LLM judge scoring against a rubric your team wrote
Serve	Chat with a checkpoint in the Playground, or promote a version to a managed deployment with a stable endpoint
Govern	Quotas, scheduling windows, an image allowlist, audit, and datasets a trainer may use without being able to read

Pick your path

I want to train a model

Quickstart, then submitting, sweeps and pipelines.

I want to extend the platform

Nine extension points: recipes, benchmarks, environments, rubrics, plugins.

I have to run this thing

Four executors, 233 settings, auth, storage, upgrades.

A run, end to end

1. Scaffold

   sf init creates a project. sf new copies a method template into experiments/<name>/ and pins the exact recipe and framework version in recipe.lock.json.

2. Validate

   sf validate checks types, ranges and batch-size divisibility against the method declaration — on your machine, in seconds, before anything is queued.

3. Submit

   sf submit packages the working tree, refuses to include anything that looks like a secret, and hands the job to the server. Admission checks quota, the recipe handshake, and the image allowlist, then queues it.

4. Observe

   The job reports lifecycle marks, metrics, samples, logs and artifacts back through ingest. The console charts them live.

5. Decide

   Score it with sf bench, compare it against previous runs, chat with it in the Playground, export it to HuggingFace format, or register it as a model version and deploy it.

What runs where

	Your laptop	The control plane	The cluster
Config validation	✓	✓
Cluster credentials		✓
Object-storage keys		✓
Training code			✓
Plugin code			✓
Dataset contents			✓

The control plane never executes third-party code, and a trainer never receives the raw contents of a protected dataset. Both are structural, not policy: see extension points.

Two repositories

Repository	What it is
starforge	The platform: starforge-console (control plane) and starforge-core (the sf CLI and SDK)
starforge-tutorial	An example project. Your own lab comes from sf init, not from cloning this

Submissions need a clean git tree, or --allow-dirty. It is the cheapest way to make a result traceable to an exact commit six months later, when it matters and nobody remembers.