> ## Documentation Index
> Fetch the complete documentation index at: https://starforge.mintlify.site/llms.txt
> Use this file to discover all available pages before exploring further.

# Chat with a checkpoint

> Spin up a vLLM service from training artifacts in one step, with web chat and A/B comparison

After training, start a vLLM OpenAI-compatible server from the artifact and chat in the browser. Two services can sit side by side (A/B). Supported on `local` and `kuberay` only.

## Start a service

<Tabs>
  <Tab title="CLI">
    ```bash theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
    sf serve start run:<RUN_ID> --gpus 1          # serve a training artifact (hf_export)
    sf serve start Qwen/Qwen3.5-9B --gpus 2       # any HF model (--gpus is the tensor parallelism degree)
    sf serve start /shared/models/my-model        # shared-disk path

    sf serve ls                    # my services (with endpoints and remaining TTL)
    sf serve extend <SERVE_ID> --ttl-hours 4      # extend
    sf serve stop <SERVE_ID>      # stop and release GPUs
    ```
  </Tab>

  <Tab title="Console">
    **Playground** page (`/playground`) → pick a model (training artifact dropdown / HF id input) → choose GPU count → start. The page shows service status, remaining TTL, and an extend button.
  </Tab>
</Tabs>

<Warning>
  Inference services consume real GPU quota. Services have an **idle TTL** (default set by server configuration) and stop automatically on expiry to release GPUs — this is protection against "started it and forgot to stop it" hogging GPUs, not a fault. If you need it for longer, `extend` it.
</Warning>

## Chat and A/B comparison

* **Chat**: streaming, temperature / max tokens, system prompt
* **A/B**: same prompt to two services (SFT vs GRPO artifact, for example)
* Endpoints are OpenAI-compatible; `sf serve ls` prints URL and token

## Execution backend semantics

| Backend | Service form                                                                                            |
| ------- | ------------------------------------------------------------------------------------------------------- |
| local   | Docker container (vLLM), port on the console host                                                       |
| agent   | Not supported                                                                                           |
| kuberay | Standalone vLLM Pod (not a RayJob — training backoff/TTL does not apply). Reached via the K8s API proxy |
| slurm   | Not supported (compute nodes usually have no inbound path from the console)                             |

## Permissions

The Playground is a configurable advanced feature: administrators can enable it per role / user. When not enabled, the page shows a guidance card instead of an error.