> ## Documentation Index
> Fetch the complete documentation index at: https://starforge.mintlify.site/llms.txt
> Use this file to discover all available pages before exploring further.

# sf submit

> Complete parameter reference for submitting training jobs

```bash theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
sf submit <EXP> [options]
```

Config and hyperparameters are validated automatically before submission; see [Submitting Training](/en/guides/submit) for the workflow walkthrough.

## Resources

| Option                               | Description                                                                                                                                                                                                   |
| ------------------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `--profile [ROLE=]name[:total-gpus]` | Target hardware and resources in a single argument: `h200` (registry default shape), `h200:4`, `h200:16` (2 full nodes). Repeatable to split pools by role: `--profile train=h200:8 --profile rollout=h100:2` |

The project name comes from `starforge.yaml` at the repository root and is attached automatically. Do not pass `--project`.

## Method and Hyperparameters

| Option                | Description                                                                                                 |
| --------------------- | ----------------------------------------------------------------------------------------------------------- |
| `-m, --method`        | Method id; if omitted, read from the experiment's `recipe.lock.json`                                        |
| `-s, --set KEY=VALUE` | Override hyperparameters, repeatable; types and ranges are validated locally against the method declaration |
| `--upgrade-recipe`    | Upgrade the experiment lock to the current catalog before submitting                                        |
| `--framework-version` | Switch framework version together with `--upgrade-recipe`                                                   |

## Model and Data

| Option                                         | Description                                                                                                                             |
| ---------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------- |
| `--model`                                      | Base model path or Hub id; **required for verl / TRL**                                                                                  |
| `--train-data` / `--validation-data`           | Data paths; required for verl / TRL. When referencing a platform dataset, use the relative file name within the dataset                 |
| `--train-dataset` / `--validation-dataset`     | Platform dataset reference `<owner>/<name>[@version]`; recommended to put in the config, with the CLI used only for temporary overrides |
| `--init-from run/<RUN_ID>/checkpoint[@step=N]` | Start training from a previous stage's artifact (pipelines)                                                                             |

## Framework and Observability (custom only)

| Option                | Description                                                                                                                             |
| --------------------- | --------------------------------------------------------------------------------------------------------------------------------------- |
| `--image`             | Override the OCI image for this job; optional for first-class frameworks, required for custom. Tags work; prefer a digest in production |
| `--observability-url` | External observability recipes only; forbidden for platform recipes                                                                     |

## Flow Control

| Option          | Description                                                                                   |
| --------------- | --------------------------------------------------------------------------------------------- |
| `--then ACTION` | Run automatically after successful training (`export` / `eval`), repeatable                   |
| `--allow-dirty` | Allow uncommitted changes in the working tree (rejected by default to guarantee traceability) |
| `--no-validate` | Skip pre-submission validation                                                                |

## Examples

```bash theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
# NeMo-RL GRPO, 8 GPUs
sf submit my-grpo --profile h200:8

# verl 0.9 + platform dataset + temporary hyperparameter override
sf submit my-verl --profile h200:4 \
  --model Qwen/Qwen3.5-9B \
  --train-dataset alice/gsm8k-zh@v2 --train-data train.parquet \
  -s actor_rollout_ref.actor.optim.lr=1e-6

# Heterogeneous multi-pool + automatic export after training
sf submit my-grpo --profile train=h200:8 --profile rollout=h100:2 --then export

# Custom
sf submit my-custom --profile h100:1 \
  --image registry.example.com/train@sha256:abc…
# Catalog default is still external observability: add --observability-url
# Curves in the console: see Custom images
```

## `sf submit`

Submit a training job

```bash theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
sf submit <EXP> [options]
```

| Argument | Description                            |
| -------- | -------------------------------------- |
| `EXP`    | Experiment name or path (**required**) |

| Option                 | Description                                                                                                                                                                                                                                                                                                                                                     |
| ---------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `--profile`            | Target hardware and resources in one argument: h200 (the registry default shape), h200:4 (4 GPUs), h200:16 (2 full nodes). Repeat it to split pools per role (the heterogeneous extension point): --profile train=h200:8 --profile rollout=h100:2. Comma-separate roles that share a pool: --profile actor,rollout=h200:8 --profile teacher=h200:8 (repeatable) |
| `--method`, `-m`       | Method identifier \<framework>/\<method> (e.g. nemo-rl/grpo, verl/grpo); read from the experiment directory's recipe.lock.json when omitted                                                                                                                                                                                                                     |
| `--set`, `-s`          | Override a hyperparameter, repeatable. Types and ranges are checked locally against the method declaration, so a typo fails immediately (repeatable)                                                                                                                                                                                                            |
| `--init-from`          | Start from the previous stage's artifact (the SFT -> DPO -> GRPO pipeline)                                                                                                                                                                                                                                                                                      |
| `--model`              | Base model path or Hub id; required for verl/TRL                                                                                                                                                                                                                                                                                                                |
| `--train-data`         | Training data path; required for verl/TRL. With a platform dataset declared (in the config or via --train-dataset), write the file name relative to the dataset (e.g. train.parquet) and the job side resolves it into the cache directory                                                                                                                      |
| `--validation-data`    | Validation data path; required for verl/TRL, used like --train-data                                                                                                                                                                                                                                                                                             |
| `--train-dataset`      | Platform dataset reference: pulled into the shared cache when the job starts, with \<NAME>\_DATA\_DIR injected. Prefer declaring it as data.train.dataset in the experiment config; this flag is only a temporary override                                                                                                                                      |
| `--validation-dataset` | Platform dataset reference for the validation set; matches data.validation.dataset in the config                                                                                                                                                                                                                                                                |
| `--output-dataset`     | Publish what this run produces as this dataset version. The job writes files into FORGE\_DATASET\_OUT\_DIR and the platform publishes them when the run succeeds -- the job is never given a credential for the registry, and the published dataset inherits the classification of the material the run read. Used by a preprocessing run over a corpus         |
| `--environment`        | Agent RL environment reference. Delivered like a corpus and for a stronger reason: a private environment is non-exportable, so the platform resolves the version into the job runtime and mounts it read-only. Run `sf env ls` to see the environments you can reference                                                                                        |
| `--image`              | Job image (a tag is enough, e.g. registry/nemo-rl-ext:v0.7.0). Required for custom; for other frameworks it overrides the default image the console configured for them                                                                                                                                                                                         |
| `--then`               | Run automatically after training succeeds (export/eval), repeatable (repeatable)                                                                                                                                                                                                                                                                                |
| `--observability-url`  | For an external observability recipe only; a platform recipe must not set it                                                                                                                                                                                                                                                                                    |
| `--framework-version`  | Only together with --upgrade-recipe; on its own it does not rewrite the lock file                                                                                                                                                                                                                                                                               |
| `--upgrade-recipe`     | Upgrade this experiment's lock to the current catalog before submitting, reusing sf recipe upgrade                                                                                                                                                                                                                                                              |
| `--allow-dirty`        | Allow uncommitted changes in the working tree (refused by default, to keep submissions traceable)                                                                                                                                                                                                                                                               |
| `--no-validate`        | Skip the pre-submission validation                                                                                                                                                                                                                                                                                                                              |
