> ## Documentation Index > Fetch the complete documentation index at: https://starforge.mintlify.site/llms.txt > Use this file to discover all available pages before exploring further. # Admin Workers Health > 后台角色心跳（仅反映**本实例**）。 排查「作业一直卡在 QUEUED / RUNNING 不转终态 / 配额不释放」时先看这里： leader 锁在 Redis 降级时返回「谁都不是 leader」，四个 worker 会一起安静 停摆 —— 没有异常也没有堆栈，日志里只有一句和调度不相干的降级警告。 inprocess_workers=false 时本实例的列表本来就应该是空的：角色跑在独立的 server.worker 进程里，要看那边的心跳。 Unlike `workers`, `projections` is cross-instance: the numbers come from the database, so they hold whichever replica reconciles. A heartbeat only says a tick ran here; it cannot say the projection caught up. A reconciler that ticks on schedule but fails every listing keeps a healthy heartbeat while every row ages. ## OpenAPI ````yaml /api-reference/openapi.json get /api/admin/workers openapi: 3.1.0 info: title: StarForge Console description: >- The StarForge control plane. Everything the `sf` CLI and the web console do goes through this API, and so can your own tooling. Authenticate with a bearer token from `POST /api/auth/login` or a CLI device flow; see the Authentication page for how to get one and how long it lasts. version: 0.3.15 servers: - url: https://{host} description: Your StarForge deployment variables: host: default: starforge.your-company.com description: >- The domain your administrator gave you, without a scheme or trailing slash. security: [] tags: - name: auth description: >- Log in, exchange a CLI device code, and inspect the current identity. Everything else on this API needs a bearer token from here. - name: profile description: >- The signed-in user's own account: quota, tokens, preferences, and notification settings. - name: projects description: >- Projects group runs the way `starforge.yaml` names them. A run belongs to exactly one. - name: experiments description: >- Read the experiment definitions the console found in the configured repository. - name: submit description: >- Admit a JobSpec. This is what `sf submit` calls: the catalog handshake, quota check, and preflight all happen here, and a rejection names the gate that refused it. - name: jobs description: >- Everything about a job after it is admitted: status, logs, metrics, samples, artifacts, and the pause/resume/stop controls. - name: runs description: >- Finished work, addressed by run id. A run outlives the job that produced it. - name: ingest description: >- The endpoints training code reports to. `starforge.report` speaks this; you only call it directly when writing an adapter for a framework the catalog does not cover. - name: datasets description: >- Versioned dataset upload, listing, and metadata. Protected datasets expose identity and schema here but never their records. - name: volumes description: Governed directories of files a job may mount read-only. - name: environments description: >- Agent RL environments: their manifests, versions, and upload URLs. A taskset is never returned. - name: benchmarks description: >- The benchmark catalog, the score matrix across runs, and externally scored evaluations. - name: rubrics description: >- Written scoring standards, their revisions, and which runs cited which version. - name: judge description: >- The LLM-judge endpoint a training job calls to score a rollout. OpenAI-compatible. - name: models description: >- The model registry: register a version, promote it, archive it, read its card. - name: model-deployments description: >- Managed model versions serving application traffic: revisions, promotion, rollback, suspension, and deployment tokens. - name: inference description: >- OpenAI-compatible inference against a promoted deployment revision. This is the endpoint applications call. - name: playground description: >- Short-lived serving sessions for human evaluation. Distinct from a deployment: a session expires, a deployment does not. - name: reflow description: >- The governed path from a deployment's production traffic back to the training data of its next version. - name: annotate description: 'Preference annotation: pull a batch, push judgements, read progress.' - name: plugins description: Installed plugins and the extension shelf the console renders. - name: diagnosis description: >- Automated analysis of a finished or failed run, and the accumulated project memory it draws on. - name: approvals description: 'Approval requests: an escalation path, one level deep, with a record.' - name: billing description: >- What the GPU-hours cost. One price on top of the hours the usage page already shows. - name: teams description: 'Teams: the unit capacity is budgeted to. A department, not a tenant.' - name: agent description: >- Submit plans: a proposed submission a human approves or rejects before it becomes a job. - name: share description: >- Public, revocable read-only links to a job or a comparison. The `/api/share/{token}` routes need no bearer token, which is the point. - name: notifications description: The signed-in user's notification feed. - name: search description: Cross-surface search over jobs, runs, datasets, and models. - name: sandbox description: >- Execute model-generated code in a throwaway container with no GPU and no network. - name: uploads description: >- Resumable upload sessions used by dataset, environment, and plugin publishing. - name: integrations-hf description: Hugging Face account linking and repository push. - name: mcp description: Model Context Protocol access information and per-user tool settings. - name: mcp-oauth description: >- OAuth metadata, authorization, token exchange, and dynamic client registration for MCP clients. - name: cluster description: Live capacity and node state across the fleets. - name: fleets description: >- Registered execution backends and the machines in them. Reading is open to every user; creating a fleet, minting a join token and draining a node are admin-only. Joining is authorized by the join token alone. - name: admin description: >- User, role, quota, hardware, schedule, integration, and settings administration. Admin role required. - name: tasks description: >- Scheduled platform maintenance tasks: what they are, when they last ran, and running one now. - name: report description: The rendered daily report page. - name: health description: Liveness and version. Unauthenticated. paths: /api/admin/workers: get: tags: - admin summary: Admin Workers Health description: >- 后台角色心跳（仅反映**本实例**）。 排查「作业一直卡在 QUEUED / RUNNING 不转终态 / 配额不释放」时先看这里： leader 锁在 Redis 降级时返回「谁都不是 leader」，四个 worker 会一起安静 停摆 —— 没有异常也没有堆栈，日志里只有一句和调度不相干的降级警告。 inprocess_workers=false 时本实例的列表本来就应该是空的：角色跑在独立的 server.worker 进程里，要看那边的心跳。 Unlike `workers`, `projections` is cross-instance: the numbers come from the database, so they hold whichever replica reconciles. A heartbeat only says a tick ran here; it cannot say the projection caught up. A reconciler that ticks on schedule but fails every listing keeps a healthy heartbeat while every row ages. operationId: admin_workers_health_api_admin_workers_get responses: '200': description: Successful Response content: application/json: schema: $ref: '#/components/schemas/WorkersHealthOut' security: - HTTPBearer: [] components: schemas: WorkersHealthOut: properties: inprocess_workers: type: boolean title: Inprocess Workers redis_configured: type: boolean title: Redis Configured workers: items: $ref: '#/components/schemas/WorkerHealthOut' type: array title: Workers default: [] projections: items: $ref: '#/components/schemas/ProjectionHealthOut' type: array title: Projections default: [] type: object required: - inprocess_workers - redis_configured title: WorkersHealthOut description: |- 后台 worker 健康。 leader 锁在 Redis 降级时返回「谁都不是 leader」，四个 worker 会一起安静停摆： 没有异常、没有堆栈，出队循环只是每轮 continue。这个端点让那种停摆看得见。 WorkerHealthOut: properties: role: type: string title: Role is_leader: type: boolean title: Is Leader default: false seconds_since_success: anyOf: - type: number - type: 'null' title: Seconds Since Success consecutive_leader_misses: type: integer title: Consecutive Leader Misses default: 0 last_error: anyOf: - type: string - type: 'null' title: Last Error type: object required: - role title: WorkerHealthOut description: 单个后台角色在**本实例**上的心跳。 ProjectionHealthOut: properties: kind: type: string title: Kind entries: type: integer title: Entries default: 0 oldest_synced_at: anyOf: - type: string - type: 'null' title: Oldest Synced At seconds_since_oldest_sync: anyOf: - type: number - type: 'null' title: Seconds Since Oldest Sync interval_s: type: number title: Interval S default: 0 stale: type: boolean title: Stale default: false type: object required: - kind title: ProjectionHealthOut description: >- How far behind the object-store projection for one kind has fallen. The heartbeats above only say whether a tick ran on this instance. They cannot say whether the projection is actually current: a reconciler that ticks but fails every listing keeps a healthy heartbeat while every row goes stale, and the datasets page quietly serves an older world. securitySchemes: HTTPBearer: type: http scheme: bearer ````