> ## Documentation Index
> Fetch the complete documentation index at: https://starforge.mintlify.site/llms.txt
> Use this file to discover all available pages before exploring further.

# Upstream Version Adoption SOP

> How the platform keeps up when NeMo-RL / verl / TRL ship new versions

The platform uses an **exact-pinning** strategy for upstream frameworks: every published version = one variant in the recipe version matrix + one execution artifact in the runtime registry. Adopting a new version is a **declarative** process; in most cases no platform code changes.

## Standard steps

<Steps>
  <Step title="Build the execution artifact">
    Build the training image for the new version (K8s/Docker: OCI digest-pinned) or SIF/SQSH (Slurm), publish it to the runtime registry, and assign a `runtime_id`.
  </Step>

  <Step title="Declare the version variant">
    Add a new version entry under `runtime.versions` in the recipe YAML: `runtime_id`, dependencies, and the **version-bound differences** — entrypoint override (`entrypoint`), parameter path overrides (`path_overrides`), and certified observer patch versions (`observer_versions`).
  </Step>

  <Step title="Run adaptation tests">
    SDK unit tests cover the version matrix (entrypoint / parameter paths / observers asserted per version); full regression via `tests/`.
  </Step>

  <Step title="Canary release">
    Don't make the new version `default_version` at first: early adopters opt in explicitly with `sf new --framework-version X` / `sf recipe upgrade --framework-version X`. Switch the default once it's stable.
  </Step>

  <Step title="User upgrade">
    After the catalog is published, existing experiment locks show drift hints; users upgrade explicitly with `sf recipe upgrade` — there is no silent switching.
  </Step>
</Steps>

## Framework-specific notes

| Framework   | Common differences                                                                              |
| ----------- | ----------------------------------------------------------------------------------------------- |
| **NeMo-RL** | Official YAML base updates (sync `configs/base/`), Megatron/DTensor config key migrations       |
| **verl**    | Training entrypoint consolidation (e.g. the unified main\_ppo in 0.9), Ray version requirements |
| **TRL**     | Trainer API signature changes, Accelerate config default changes                                |

## What not to do

* ❌ Write `if version >= X` branches in adapter code — version differences belong in recipe declarations;
* ❌ Use latest / branch names as versions — only exact semver or full commits are accepted;
* ❌ Reuse an old `runtime_id` to point at a new image — artifacts are immutable; new image, new id.

<Note>
  The full evaluation of this mechanism (why pinning is not over-coupling) is in the repo's `docs/ops/upstream-version-adoption.md`.
</Note>