> ## Documentation Index
> Fetch the complete documentation index at: https://starforge.mintlify.site/llms.txt
> Use this file to discover all available pages before exploring further.

# Train with TRL

> Accelerate + TRL trainers, required model/data, OPSD teacher, image

```bash theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
sf new my-sft --method trl/sft
sf submit my-sft --profile h200:4 \
  --model Qwen/Qwen2.5-7B --train-data data/train.parquet
```

Hugging Face TRL **1.10.0** as its own runtime, not routed through `custom`. Methods: `trl/sft`,
`dpo`, `grpo`, `rm`, `kto`, `rloo`, `opsd`.

Observability is `platform`. TRL's own metric names (`loss`, `eval_loss`, `learning_rate`) are
aliased onto the console's series, so the charts fill in without you renaming anything.

## Create

```bash theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
sf new my-sft --method trl/sft
sf new my-kto --method trl/kto
sf new my-opsd --method trl/opsd
```

Entrypoint is `train.py` in the experiment directory (`kind: experiment`). Edit that file only if you know you need a fork; hyperparameters belong in `config.yaml` / `--set`.

## Submit: model and data required

Same idea as verl: `--model` and `--train-data` (and usually `--validation-data`) are required unless the config already has them in the bound fields.

```bash theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
sf submit my-sft --profile h200:8 \
  --model Qwen/Qwen3.5-9B \
  --train-data data/train.parquet \
  --validation-data data/val.parquet
```

`trl/opsd` needs a same-vocab teacher (`teacher_model` in the recipe params). Missing it fails validation.

Gated Hub repos: link HuggingFace on the console so submit preflight can check access. [HuggingFace](/en/integrations/huggingface).

## Launch

Adapter uses Accelerate (`adapter_options.launcher: accelerate`). GPU count still comes from `--profile` / `FORGE_CLUSTER_GPUS_PER_NODE`.

## After training

Checkpoints match HuggingFace layout (`checkpoint-*`, `final_model`). Export/eval lifecycle is declared on the recipe (`starforge.lifecycle.export_complete`, experiment `eval.py`).

```bash theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
sf export my-sft --checkpoint <path> --checkpoint-format huggingface
```