Skip to main content
A Fleet is a named set of machines and the backend that runs work on them. It is a record you create, not a setting you export: one console carries several Fleets at once, and every job and deployment records the Fleet it was admitted against. This replaced a single deployment-wide backend variable. See ADR-0013 for why. FORGE_DEFAULT_FLEET_KIND survives it, and seeds the first Fleet on a console that has none — read once, at first start, and nowhere else.

​
What a Fleet carries

config holds the keys that differ between two Fleets of the same kind — the slurm REST endpoint, the KubeRay namespace. A key no setting reads is logged as a warning rather than ignored: a misspelled override is a setting an operator believes is in effect and is not.
A capability is a promise, not a preference. A Fleet without serve is not a Fleet that serves badly — a deployment naming it is refused.

​
Creating one

Fleets → New fleet. Pick the backend and what the Fleet is for; the delivery follows the backend unless you change it. Or through the API:
Administrators only. Registering machines is an operator’s job; picking a Fleet to submit to is a user’s. A console with no Fleet at all seeds one on first start from FORGE_DEFAULT_FLEET_KIND, so a zero-config deployment never has to make this call.

​
Registering a machine

Only node Fleets have machines to register — kuberay and slurm get theirs from the cluster manager, and local is the console’s own host. Mint a join token on the Fleet page — Add a machine. It is single-use, expires in 24 hours, and is stored only as a hash. The page shows one line to paste on the machine:
On a machine that already has the package:
The join token is the whole authorization. You need root on the machine, not an account on the console — which is what makes the one-liner safe to hand to whoever administers the box. Treat a token in a chat log as a standing key until it expires.
What comes back is the node’s identity and a long-lived credential, written under FORGE_NODE_STATE_DIR. The credential is returned once. Losing it means asking for another token and joining again as a new node, which is correct: a machine that lost its identity is not the same machine to a control plane reconciling containers against it. The node reports its own inventory at join — card name, count, driver, container runtime, and whether it can see the storage root. The console maps the card name onto a hardware series. A card the registry does not know is named in the join output so an administrator can add it; until then, jobs that pin a series are not placed there.

​
An address is not an identity

A node is identified by the id the console issued, never by host:port. A machine that moves stays the same machine. The consequence is that two records may legitimately answer at one address, so joining where another live node already answers is a warning rather than a refusal — printed by forgelet join, because the usual cause is a re-join that left the old record behind, and then the console counts one machine’s cards twice.

​
Liveness

One last_seen_at, read against three thresholds, because the cost of being wrong differs by what the machine is for: These are three consequences, not three states. A node is declared gone on the most generous of its Fleet’s capabilities; the tighter rules answer per capability. “Do not send traffic there now” and “do not place anything there at all” are different claims, and a Fleet that declares all three would otherwise stop taking training work after 45 seconds of silence. Nothing about being gone kills anything. An unreachable node is not a dead job: observe returns no signal for its jobs, reconciliation skips them, and state converges from the real container when the machine comes back. A shared-mount node that reports losing the storage root is cordoned, and un-cordoned when the mount returns. An operator’s own cordon is never lifted by a heartbeat — that one is a decision, and a heartbeat is an observation.

​
Drain, cordon, remove

None of them kills anything; that is the point. cordoned is for a machine that is coming back, draining for one that is leaving. Removing an active node is refused. Its identity is what makes its containers reconcilable, so drain first and let the work finish. force=true exists for a machine that is already gone; it is recorded in the audit log as forced, because what it leaves behind is a container nothing can reconcile. A Fleet is deletable only with no nodes and no live deployment pinned to it. A deployment never changes Fleet, so deleting the Fleet under it would leave a row whose backend resolves to nothing: unobservable, unstoppable, and still holding cards.

​
Choosing a Fleet

A deployment is the exception because its address outlives the decision and never moves, and nothing on a Fleet record says which department owns the machines or what data may touch them. The platform declines to answer a question it cannot answer. Placement refuses at admission with a reason — capability, visibility, or inventory fit. A job that no Fleet can hold is told so while the submitter is still looking at the response, not hours later at the top of a queue.
Inventory fit is a check, not a reservation. The allocator remains authoritative for which cards a job actually gets; this only stops a request for hardware the Fleet has never had from queueing forever.

​
Visibility

A project Fleet is invisible outside its project, and a job naming one is refused as not found rather than forbidden — a private Fleet’s name does not leak through an error message.