Extension points

Everything you can add to StarForge without changing the platform, and how to pick the right one.

sf plugin publish ./my-extension

Nine ways to extend the platform. Pick by answering one question first, because it decides everything else: does the platform execute your code, or only read your declaration?

The map

You want to add	Use	Platform runs your code?	Publish with
A training method	recipe pack	No	sf plugin publish
An evaluation set	benchmark pack	No	sf plugin publish
A scoring standard for a judge	rubric	No	Console, or PUT /api/rubrics
Diagnosis thresholds	playbook pack (admin)	No	sf plugin publish
Diagnosis or judge prompts	prompt pack (admin)	No	sf plugin publish
A runtime patch to a training loop	algorithm plugin	In your training container	sf plugin publish
An agent RL task suite	environment	—	sf env push
The code that drives a model through an environment	environment plugin (harness)	In your training container	sf plugin publish
A local data preprocessing step	data-prep plugin	On your machine	sf plugin publish

The one rule that shapes all of it

The control plane never loads third-party code. Not now, not behind a flag.

The console process holds the database connection, the JWT signing secret, and object-storage credentials. Running a plugin there would hand over the whole platform to whoever published it.

So code plugins only ever execute somewhere that is already yours — your training container, or your own laptop. To change control-plane behaviour, cross a process boundary instead: a webhook for a notification channel, an external MCP server for an agent tool.

That is also why there is no runs_in: console plugin kind, and why one will not be added.

Declarative packs cannot contain code

A pack the platform only reads (recipe, benchmark, playbook, prompt) is refused at publish time if it contains an executable file at its root — .py, .sh, .so, .js and about twenty other suffixes.

my-grpo-variant/
├── plugin.yaml         # kind: recipe
├── recipe.yaml         # the declaration the platform parses
├── README.md
└── template/           # the one place executables are allowed
    ├── config.yaml
    └── run.sh

template/ is the exception, and the reason is worth knowing: it is scaffolding that sf new copies into your repository, which you then run in your container. The platform never touches it.

This turns “the platform never executes declarative packs” from a promise into something a script checks. The error, if you trip it:

recipe is a declarative pack -- the platform parses it and never runs it, so it must
contain no executable files: helper.py. Scaffolding meant to be handed to users belongs
under template/; if the platform really has to execute code, use one of the
algorithm/environment/data-prep kinds instead.

Three gates on executable plugins

Every plugin whose code runs passes the same three digest checks. They exist so that what ran on the cluster is exactly what you published.

1. At publish

   The server unpacks your directory and recomputes the content digest, then compares it to what the client claimed. A mismatch is refused.

2. At submission

   The (id, version, digest) triple in the JobSpec is checked against the library record, and against whether an administrator has disabled the plugin.

3. At launch

   The launcher recomputes the digest of the directory that was actually injected into the job. If it differs from the one the JobSpec locked, the job refuses to start rather than producing a result nobody can reproduce.

The digest is taken over the directory contents — sorted relative paths plus file bytes — not over a tarball. A tarball hash would fold in mtimes and uids, so the same code packed twice would produce two digests and pinning would mean nothing.

__pycache__, .git, .venv, *.pyc and .DS_Store are excluded from the digest. They are environment by-products, not plugin content.

Two more checks on executable packages

Published-time structural validation, so a mistake fails on your laptop instead of forty minutes into a cluster allocation:

The entrypoint module must actually exist

entrypoint: patch:install requires patch.py or patch/__init__.py in the package. A typo is rejected at publish:

entrypoint module 'pathc' does not exist in the package (expected pathc.py or
pathc/__init__.py). A misspelled entry point would otherwise fail only when training
starts, so it is refused here.

Top-level names must not shadow real dependencies

Your plugin root joins sys.path, so a file called torch.py at the root would make every subsequent import torch in the training process load your file. The symptom would appear nowhere near the cause, so these names are reserved and refused:

starforge  nemo_rl  common  ray  torch  transformers  vllm
trl  verl  numpy  yaml  json  os  sys

Versions are immutable

Publishing 1.2.0 twice is a 409. Fix a bug by publishing 1.2.1.

This is the premise digest pinning rests on: if a version could be overwritten, a lockfile would name something that no longer exists. It also covers README.md — the readme is inside the digest, so changing the long description requires a new version. “The docs say one thing, the code does another” is the most common failure of plugin marketplaces, and this is the cheapest way to prevent it.

Naming and namespaces

Object	Reference form	Notes
Plugin	<owner>/<name>	Owner is stamped by the platform at publish, not claimed in the manifest
Published method	<framework>/<owner>.<name>	The extra segment makes a collision with a built-in method structurally impossible
Environment	<owner>/<name>@<version>
Rubric	<owner>/<name>

Governance

sf plugin disable alice/opsd-patch   # admin: no new job may reference it
sf plugin enable alice/opsd-patch

Three verbs with three different scopes — do not conflate them:

Verb	Who	Scope
Enable / disable	Admin	The whole deployment. Takes something down, keeps the history
Reference	You	One experiment, through plugins.lock.json
Sync	Automatic	Your local method library. Stateless — delete it and sync again

Disabling blocks new submissions only. Jobs already running or queued are unaffected.

Where to go next

Plugin packages

The manifest, the directory layout, publishing and installing.

Algorithm plugins

The entrypoint signature, and when your function is called.

Recipe packs

Add a training method without a platform release.

Benchmark packs

Declare what to run and which numbers to read.

Environments

Task suites for agent RL, and the four protocols.

Verifiers

The three ways to decide whether a task was completed.