> ## Documentation Index
> Fetch the complete documentation index at: https://starforge.mintlify.site/llms.txt
> Use this file to discover all available pages before exploring further.

# Create and tune an experiment

> sf new scaffolding, layered config.yaml inheritance, local validation

An experiment is a self-contained directory: `experiments/<name>/`, containing `config.yaml` (the tuning entry point), `README.md`, and `recipe.lock.json` (the method lock). Create a repository first with [`sf init`](/en/cli/init), then run the commands below inside that repository. Disk layout and upload excludes: [Lab layout](/en/concepts/lab-layout). Inheritance details: [config.yaml](/en/guides/config).

## Create an experiment

<Tabs>
  <Tab title="From the method catalog">
    ```bash theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
    sf new my-grpo --method nemo-rl/grpo
    sf new my-verl --method verl/grpo --framework-version 0.9.0
    sf new my-kto  --method trl/kto
    ```

    The scaffold comes from the recipe's official template: config base, README tuning notes, and lock file all in place at once.
  </Tab>

  <Tab title="Fork an existing experiment">
    ```bash theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
    sf new my-grpo-v2 --from my-grpo
    ```

    Inherits the source experiment's full configuration and lock (`--method` is ignored). Ideal for incremental changes on top of an already-converged configuration.
  </Tab>
</Tabs>

## config.yaml: write only the diff

An experiment config inherits from official bases via `defaults` and only writes the keys you want to change:

```yaml experiments/my-grpo/config.yaml theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
defaults:
  - ../../configs/base/grpo_math_1B.yaml     # official base
  - ../../configs/models/qwen3.5-9b.yaml     # model fragment

# ── tuning section ──
policy:
  optimizer:
    kwargs:
      lr: 2.0e-6
grpo:
  num_generations: 8
  kl_coef: 0.01
```

<Note>
  Hardware and distributed details (parallelism, GPU memory tuning, NCCL) are **not written in the experiment config** — they are delivered from the server-side registry via `--profile` at submission time. Switching GPU types for the same experiment requires no config change.
</Note>

The top of the scaffold contains a "tuning cheat sheet" comment block listing the method's most commonly changed keys (learning rate, batch, sequence length, validation interval, etc.) with empirical ranges.

## Local validation

```bash theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
sf validate my-grpo
```

The validator checks against the recipe declaration:

* **Key validity**: struct mode; misspelled config keys fail immediately;
* **Types and ranges**: learning rate, batch, etc. are checked against declared types and value ranges;
* **Batch size relationships**: methods like GRPO validate the divisibility relationship between rollout batch and train batch (divisible but unequal lengths produce an off-policy warning; not divisible is an error);
* **Data declarations**: required `--model` / `--train-data` for verl / TRL are caught early if missing.

`sf submit` automatically runs the same validation; `--no-validate` skips it (not recommended).

## Browse methods and hyperparameters

```bash theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
sf methods                # all methods: one-line summary + supported framework versions
sf methods nemo-rl/grpo   # tunable hyperparameters for that method: type, default, range, description
sf ls                     # list all experiment directories in this repository
```

The console **Methods** page (`/methods`) has a visual version of the same catalog.

## Comparing runs in the console

The jobs area on a project detail page is a dense comparison matrix:

* Job name and status stay fixed while metric/config columns scroll horizontally;
* Rows and columns are both virtualized, so hundreds of runs × hundreds of columns render only nearby cells;
* All latest metrics and config values that vary across runs are visible by default; constant config columns are available from the Columns panel;
* Search jobs, filter status, search/hide/pin columns, and select 2–4 runs for review comparison.

## What else can live in an experiment directory

| Content                          | Purpose                                                                                                                                |
| -------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------- |
| `train.sh`                       | Custom only: training entrypoint. Logs follow stdout; curves use `starforge.report`. See [Custom training](/en/guides/custom-training) |
| Custom environment / reward code | Modules in `common/environments/` and `common/rewards/` referenced by the config, uploaded with the job package                        |
| `plugins.lock.json`              | References platform plugins (`sf plugin install --exp`); injected by the platform at submission                                        |
| `cluster` annotation file        | Legacy: default profile record; explicit `--profile` at submission is now recommended                                                  |