> ## Documentation Index
> Fetch the complete documentation index at: https://starforge.mintlify.site/llms.txt
> Use this file to discover all available pages before exploring further.

# sf annotate

> Manage preference annotation

```bash theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
sf annotate status                                   # is the integration configured
sf annotate push run-4f2a91 --limit 200              # send answers out for labelling
sf annotate pull run-4f2a91 --dataset alice/prefs    # bring judgements back as a dataset
```

Human preference annotation, through an Argilla instance the deployment configures. The loop is:
take a run's generated answers out for people to judge, then bring the judgements back as a
versioned dataset you can train DPO on.

The platform is the transport and the governance here, not the annotation tool. `sf annotate status`
tells you whether an Argilla instance is wired up at all — start there, because everything else
fails confusingly without one.

See [Annotation](/en/guides/annotation) for the workflow around these commands.

## `sf annotate`

Manage preference annotation

```bash theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
sf annotate <subcommand>
```

| Command              | Description                        |
| -------------------- | ---------------------------------- |
| `sf annotate pull`   | Pull annotations back as a dataset |
| `sf annotate push`   | Send answers out for annotation    |
| `sf annotate status` | Check the Argilla integration      |

### `sf annotate pull`

Pull annotations back as a dataset

```bash theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
sf annotate pull <DATASET> [options]
```

| Argument  | Description                         |
| --------- | ----------------------------------- |
| `DATASET` | Argilla dataset name (**required**) |

| Option            | Description                                                                            |
| ----------------- | -------------------------------------------------------------------------------------- |
| `--into`          | Land them as a new version of this platform dataset; omit to only print the statistics |
| `--version`, `-v` | Dataset version; defaults to the date                                                  |

### `sf annotate push`

Send answers out for annotation

```bash theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
sf annotate push <RUN_A> <RUN_B> [options]
```

| Argument | Description                                      |
| -------- | ------------------------------------------------ |
| `RUN_A`  | Run id of the first training run (**required**)  |
| `RUN_B`  | Run id of the second training run (**required**) |

| Option                  | Description                                    |
| ----------------------- | ---------------------------------------------- |
| `--dataset`, `-d`       | Argilla dataset name; synthesised when empty   |
| `--limit` `<int range>` | How many pairs to push at most (default `200`) |

### `sf annotate status`

Check the Argilla integration

```bash theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
sf annotate status
```

No options beyond `--help`.