sf submit

Complete parameter reference for submitting training jobs

sf submit <EXP> [options]

Config and hyperparameters are validated automatically before submission; see Submitting Training for the workflow walkthrough.

Resources

Option	Description
--profile [ROLE=]name[:total-gpus]	Target hardware and resources in a single argument: h200 (registry default shape), h200:4, h200:16 (2 full nodes). Repeatable to split pools by role: --profile train=h200:8 --profile rollout=h100:2

The project name comes from starforge.yaml at the repository root and is attached automatically. Do not pass --project.

Method and Hyperparameters

Option	Description
-m, --method	Method id; if omitted, read from the experiment’s recipe.lock.json
-s, --set KEY=VALUE	Override hyperparameters, repeatable; types and ranges are validated locally against the method declaration
--upgrade-recipe	Upgrade the experiment lock to the current catalog before submitting
--framework-version	Switch framework version together with --upgrade-recipe

Model and Data

Option	Description
--model	Base model path or Hub id; required for verl / TRL
--train-data / --validation-data	Data paths; required for verl / TRL. When referencing a platform dataset, use the relative file name within the dataset
--train-dataset / --validation-dataset	Platform dataset reference <owner>/<name>[@version]; recommended to put in the config, with the CLI used only for temporary overrides
--init-from run/<RUN_ID>/checkpoint[@step=N]	Start training from a previous stage’s artifact (pipelines)

Framework and Observability (custom only)

Option	Description
--image	Override the OCI image for this job; optional for first-class frameworks, required for custom. Tags work; prefer a digest in production
--observability-url	External observability recipes only; forbidden for platform recipes

Flow Control

Option	Description
--then ACTION	Run automatically after successful training (export / eval), repeatable
--allow-dirty	Allow uncommitted changes in the working tree (rejected by default to guarantee traceability)
--no-validate	Skip pre-submission validation

Examples

# NeMo-RL GRPO, 8 GPUs
sf submit my-grpo --profile h200:8

# verl 0.9 + platform dataset + temporary hyperparameter override
sf submit my-verl --profile h200:4 \
  --model Qwen/Qwen3.5-9B \
  --train-dataset alice/gsm8k-zh@v2 --train-data train.parquet \
  -s actor_rollout_ref.actor.optim.lr=1e-6

# Heterogeneous multi-pool + automatic export after training
sf submit my-grpo --profile train=h200:8 --profile rollout=h100:2 --then export

# Custom
sf submit my-custom --profile h100:1 \
  --image registry.example.com/train@sha256:abc…
# Catalog default is still external observability: add --observability-url
# Curves in the console: see Custom images

sf submit

Submit a training job

sf submit <EXP> [options]

Argument	Description
EXP	Experiment name or path (required)

Option	Description
--profile	Target hardware and resources in one argument: h200 (the registry default shape), h200:4 (4 GPUs), h200:16 (2 full nodes). Repeat it to split pools per role (the heterogeneous extension point): —profile train=h200:8 —profile rollout=h100:2. Comma-separate roles that share a pool: —profile actor,rollout=h200:8 —profile teacher=h200:8 (repeatable)
--method, -m	Method identifier <framework>/<method> (e.g. nemo-rl/grpo, verl/grpo); read from the experiment directory’s recipe.lock.json when omitted
--set, -s	Override a hyperparameter, repeatable. Types and ranges are checked locally against the method declaration, so a typo fails immediately (repeatable)
--init-from	Start from the previous stage’s artifact (the SFT -> DPO -> GRPO pipeline)
--model	Base model path or Hub id; required for verl/TRL
--train-data	Training data path; required for verl/TRL. With a platform dataset declared (in the config or via —train-dataset), write the file name relative to the dataset (e.g. train.parquet) and the job side resolves it into the cache directory
--validation-data	Validation data path; required for verl/TRL, used like —train-data
--train-dataset	Platform dataset reference: pulled into the shared cache when the job starts, with <NAME>_DATA_DIR injected. Prefer declaring it as data.train.dataset in the experiment config; this flag is only a temporary override
--validation-dataset	Platform dataset reference for the validation set; matches data.validation.dataset in the config
--output-dataset	Publish what this run produces as this dataset version. The job writes files into FORGE_DATASET_OUT_DIR and the platform publishes them when the run succeeds — the job is never given a credential for the registry, and the published dataset inherits the classification of the material the run read. Used by a preprocessing run over a corpus
--environment	Agent RL environment reference. Delivered like a corpus and for a stronger reason: a private environment is non-exportable, so the platform resolves the version into the job runtime and mounts it read-only. Run sf env ls to see the environments you can reference
--image	Job image (a tag is enough, e.g. registry/nemo-rl-ext:v0.7.0). Required for custom; for other frameworks it overrides the default image the console configured for them
--then	Run automatically after training succeeds (export/eval), repeatable (repeatable)
--observability-url	For an external observability recipe only; a platform recipe must not set it
--framework-version	Only together with —upgrade-recipe; on its own it does not rewrite the lock file
--upgrade-recipe	Upgrade this experiment’s lock to the current catalog before submitting, reusing sf recipe upgrade
--allow-dirty	Allow uncommitted changes in the working tree (refused by default, to keep submissions traceable)
--no-validate	Skip the pre-submission validation