> ## Documentation Index
> Fetch the complete documentation index at: https://starforge.mintlify.site/llms.txt
> Use this file to discover all available pages before exploring further.

# Train with NeMo-RL

> sf new, config inheritance, submit, topology, and NeMo-RL-only flags

```bash theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
sf new my-grpo --method nemo-rl/grpo
sf validate my-grpo
sf submit my-grpo --profile h200:8
```

NeMo-RL is the default catalog for a full post-training stack, currently **0.7.0**. Methods:
`nemo-rl/sft`, `dpo`, `grpo`, `grpo-lora`, `ppo`, `rm`, `distillation`, `maxrl`, `opsd`.

Observability is `platform`, which means two things you do not have to do: pass
`--observability-url`, and call `starforge.report` yourself. The adapter and the framework bridge
handle both.

## Create and tune

```bash theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
sf new my-grpo --method nemo-rl/grpo
# edit experiments/my-grpo/config.yaml  (defaults + diffs only)
sf validate my-grpo
sf methods nemo-rl/grpo
```

Model and datasets usually live in `configs/base/` + `configs/models/` fragments. Submit often needs no `--model` / `--train-data`.

Config details: [config.yaml](/en/guides/config).

## Submit

```bash theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
git add -A && git commit -m "grpo lr"
sf submit my-grpo --profile h200:8
```

`--profile` sets billed topology. The adapter writes `cluster.num_nodes` and `cluster.gpus_per_node` from `FORGE_CLUSTER_*` and points `checkpointing.checkpoint_dir` / `logger.log_dir` at `FORGE_OUT_DIR`. Do not fight those keys in the experiment file; they are overwritten.

Heterogeneous pools (train vs rollout on different GPU series) only on `kuberay` / `slurm`:

```bash theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
sf submit my-grpo --profile train=h200:8 --profile rollout=h100:2
```

When a separate rollout pool exists, the adapter sets `policy.generation.colocated.enabled=false` and fills colocated resource counts from `FORGE_POOL_TOPOLOGY`. You do not maintain a separate noncolocated overlay YAML for that.

`local` and `agent` do not run multi-pool jobs.

## Entrypoint

Recipe default is the versioned NeMo-RL module/file. If the recipe declares `experiment_override` and that file exists in the experiment directory, that file wins (custom env / dataset experiments that ship `run.py`). `spec.source.entrypoint` in the JobSpec is rejected; you cannot override the recipe from submit JSON.

The container needs `NEMO_RL_DIR` (server `FORGE_NEMO_RL_DIR`, typically `/opt/nemo-rl` in the NGC image). Missing it fails compile: the server cannot proxy NeMo-RL jobs.

## After training

```bash theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
sf export my-grpo --checkpoint <path> --checkpoint-format nemo-dcp
# formats: nemo-dcp | nemo-megatron
sf eval my-grpo --model /path/or/hub/id --eval-config configs/eval/gsm8k.yaml
sf bench run --model …          # standard benchmarks, not the same as sf eval
```

`--then export --then eval` on submit runs those after `SUCCEEDED`.

## Tool environments

GRPO/PPO can point at modules under `common/environments/`. See [Agent environments](/en/guides/agent-envs).