docs/specs/2026-09-08-user-runners-design.md

  1# Runners attached to repositories
  2
  3Ref #184 (option B). A `gitbay-runner` anyone installs on their own machine
  4and attaches to their repositories on any instance, so an instance offers CI
  5without offering compute.
  6
  7## Problem
  8
  9CI on gitbay.org builds the forge's own repositories and nothing else: the one
 10runner shares the host with the forge and is scoped with `-repos`. A
 11`.gitbay/ci.yml` in anyone else's repository queues builds nothing claims.
 12Widening that runner's scope is the second machine and the tier-per-trust-level
 13design in #184, which costs compute and storage the operator pays for.
 14
 15The runner already polls over SSH from anywhere with a key of scope `runner`,
 16and `admin runners` already lists several. What is missing is the server-side
 17rule that says which builds a given key may claim. Today there is none:
 18
 19- `keys add --scope runner` is self-service for any account.
 20- `runner next` checks only the key's scope (`requireRunner`). With no
 21  repository arguments it claims the oldest pending build on the instance,
 22  whichever repository it belongs to, and the claim carries the repository's
 23  secrets when the build is trusted.
 24- `runner log` and `runner done` accept any build id.
 25
 26With `registration = "open"` a stranger can run a runner against gitbay.org
 27today and receive builds and secrets for repositories they cannot read. The
 28wiki says a runner account is admin by necessity; the code does not enforce
 29it.
 30
 31## Decision
 32
 33A runner key is attached to repositories by a repository admin, and claims
 34builds only for the repositories it is attached to. A user who wants builds
 35installs `gitbay-runner`, runs `gitbay-runner init`, attaches the printed
 36public key to their repository, and starts the service. Admin keys keep
 37today's behaviour. Untrusted builds (merge request heads from forks) are
 38excluded from every claim unless the runner asks for them.
 39
 40Decisions taken on the way, with the alternatives rejected:
 41
 42- **Repository-level attachment**, not "a user's runner builds the user's
 43  repositories" and not "repositories the user can admin". One attachment
 44  row answers "who executes this repository's code" exactly.
 45- **The runner prints its key and an admin attaches it**, not a
 46  registration token. No secret crosses the wire, and it is the deploy-key
 47  motion the forge already has.
 48- **An attachment table keyed by ssh key**, not a `runner:<repo>` scope on
 49  the key. Fingerprints are unique per instance, so a scope binds one key
 50  to one repository; a table lets one key serve many.
 51- **Untrusted builds skipped by default**, enforced by the server, with a
 52  runner flag to opt in. Not "build everything" and not "the runner refuses
 53  without podman", which would be the runner's word.
 54- **Homebrew formula in `krz/homebrew-tap` on gitbay.org**, built from
 55  source at the tag like `gitbay.rb`. Not a GitHub tap, not an installer
 56  script.
 57
 58Not in scope: per-account build limits, claim order, a second operator
 59machine. Those stay on #184.
 60
 61## Server
 62
 63### Data
 64
 65Migration 0050:
 66
 67```sql
 68CREATE TABLE runner_repos (
 69    key_id   INTEGER NOT NULL REFERENCES ssh_keys(id) ON DELETE CASCADE,
 70    repo_id  INTEGER NOT NULL REFERENCES repos(id) ON DELETE CASCADE,
 71    added_at TEXT NOT NULL DEFAULT (strftime('%Y-%m-%dT%H:%M:%fZ','now')),
 72    PRIMARY KEY (key_id, repo_id)
 73);
 74```
 75
 76`runner_seen` is rekeyed from `user_id` to `key_id` (SQLite: recreate the
 77table; existing rows are dropped, they are heartbeats). `user_id` stays as a
 78plain column for the listing. Two runners on one account are two rows.
 79
 80`control.Ctx` gains `KeyID int64`, the id of the key that authenticated the
 81session, set by `internal/sshd`. Web and API sessions leave it zero; every
 82command that reads it is `SSHOnly`.
 83
 84### Commands
 85
 86One new noun under `repo`. Each requires `policy.CanAdmin` on the repository.
 87
 88| Command | Flags | Notes |
 89|---|---|---|
 90| `repo runner add <owner/name> < key.pub` | `ReadsStdin` | Unknown fingerprint: added to the caller's account with scope `runner`, then attached. Known fingerprint: must already have scope `runner`, and must belong to the caller unless the caller is an instance admin, else exit 4. A full-scope or deploy key is never promoted. Attaching an already-attached key is exit 0 and idempotent. The key's account must be able to read the repository, or the runner cannot clone it. Output `{fingerprint, repo}`. |
 91| `repo runner list <owner/name>` | `ReadOnly` | Per key: fingerprint, algo, owner username, `added_at`, `last_seen`, and the build it holds (`build_number`, `build_job`, `started_at`) when it holds one. |
 92| `repo runner remove <owner/name> <fingerprint>` | | Drops the attachment. The key stays on the account; `keys remove` drops the key and cascades. Exit 3 when not attached. |
 93
 94### Claim rule
 95
 96`runner next [--untrusted] [<owner/name>...]`:
 97
 98- Scope `runner`: the candidate set is the key's attachments. No attachments
 99  claims nothing and returns "no pending builds". Each `<owner/name>` on the
100  command line must be among the attachments, else exit 4 naming it; the
101  named set narrows the candidates.
102- Admin key: unchanged. Any repository, narrowed by the arguments.
103- Without `--untrusted`, builds with `trusted = 0` are never claimed, for
104  every key. `store.ClaimBuild` gains the parameter. The bay1 unit passes
105  `-untrusted` to keep building fork heads in podman.
106- Secrets ride the claim as today (`b.Trusted`). An attached runner was
107  attached by a repository admin and is trusted with the repository's
108  secrets.
109- The orphan skip loop, the reachability check and the heartbeat are
110  unchanged. The heartbeat records `c.KeyID`.
111
112`runner log <id>` and `runner done <id> ...` on a scope-`runner` key: the
113build's repository must be attached to the key, else exit 4. Admin keys are
114unchanged.
115
116### Listings
117
118`admin runners` rows carry `fingerprint` beside `username`. `scope` becomes
119the attached repositories for a runner key, joined with commas; for an admin
120key it stays the repositories the runner asked for, or `any`. `dashboard` reads the same rows.
121
122## Runner
123
124### `gitbay-runner init`
125
126A subcommand, `gitbay-runner init [-remote git@host] [-isolation podman|none]`.
127It:
128
1291. Creates the config directory: `$XDG_CONFIG_HOME/gitbay-runner`, else
130   `~/.config/gitbay-runner`. Mode 0700.
1312. Generates `id_ed25519` and `id_ed25519.pub` there unless present. Mode
132   0600 on the private key. The runner never overwrites a key.
1333. Writes `config.toml` unless present:
134
135   ```toml
136   remote = "git@gitbay.org"
137   workdir = "/Users/x/Library/Caches/gitbay-runner"   # defaultWorkdir()
138   isolation = "podman"
139   untrusted = false
140   ```
141
142   `isolation` is `none` unless `-isolation podman -image <ref>` are both
143   given: the runner refuses podman without an image, and there is no
144   image to guess. With `none` it prints one line saying so: steps run as
145   this user, and untrusted builds are excluded by default so that means
146   your own commits.
1474. Prints the public key and the next step:
148
149   ```
150   gitbay repo runner add owner/name < /Users/x/.config/gitbay-runner/id_ed25519.pub
151   ```
152
153   and the URL to paste it at, `https://<host>/<owner>/<name>/settings`,
154   with `<host>` taken from the remote. Then `brew services start
155   krz/tap/gitbay-runner`, or the binary with no arguments.
156
157Re-running `init` is safe and prints the same key.
158
159### Config file
160
161The daemon reads `config.toml` from the config directory when it exists.
162Keys are the flag names (`remote`, `ssh-opts`, `clone-base`, `workdir`,
163`poll`, `timeout`, `repos`, `jobs`, `image`, `isolation`, `memory`, `cpus`,
164`untrusted`, `identity`). A flag given on the command line overrides the
165file. `-config <path>` names another file. No other configuration source.
166
167### Own identity
168
169`-identity <path>`, default the generated key when it exists, else empty.
170When set, ssh and git clone get `-i <path>` and `-o IdentitiesOnly=yes`, so
171a laptop's ambient full-scope key is never offered. Under podman the clone
172already happens outside the container; the identity stays outside with it.
173
174### `-untrusted`
175
176Adds `--untrusted` to `runner next`. Default off.
177
178### Packaging
179
180- `Formula/gitbay-runner.rb` in `krz/homebrew-tap`: source build at the
181  tag, `go build ./cmd/gitbay-runner`, a `service do` block running
182  `opt_bin/"gitbay-runner"` with no arguments, `keep_alive true`, logs under
183  `var/"log"`, and caveats naming the two steps. `test do` asserts
184  `-version`.
185- `gitbay.rb` in the same tap moves from v0.4.0 to the current tag in the
186  same commit.
187- `deploy/release.sh` builds `gitbay-runner` for the three targets alongside
188  `gitbay` and `gitbayd`.
189
190Unchanged: poll interval, workdir layout, the per-repository build home,
191SIGTERM drain, podman isolation, log cap.
192
193## Web
194
195The repository settings page gains a Runners section: the `repo runner list`
196rows with a remove button each, and a textarea that posts a public key. Both
197go through `settingsSubmit` into the control commands; the paste is stdin
198via `dispatchIntoStdin`. No new route, so no reserved name. The account page
199already lists keys with their scope; a runner key shows as `runner`.
200
201## Docs
202
203- `Users`: a "Your own runner" section under CI builds: install, `init`,
204  attach (CLI and web), start, what it builds (your commits, with secrets)
205  and what it does not (fork heads, unless `-untrusted`), several
206  repositories on one runner, several runners on one account.
207- `Admin` and `Threat-Model`: replace "a runner account is admin by
208  necessity" and "the scoping is what the runner asks for, not an ACL the
209  server holds" with the attachment rule. The bay1 example gains
210  `-untrusted`.
211- `Parity`: one row in the repository table, runners attach/list/detach,
212  yes on CLI and web, no on iOS.
213- `CI` and `FAQ`: one line each pointing at the Users section. The FAQ's
214  "CI builds only the repositories the operator names" becomes "and any
215  repository with a runner attached".
216
217## Tests
218
219- Store: `ClaimBuild` skips untrusted builds unless asked; a claim limited to
220  attached repositories; attachment rows go with the key and with the
221  repository; `runner_seen` per key.
222- Control: a scope-`runner` key with no attachment claims nothing; an
223  attached key claims its repository and not another user's pending build;
224  a named repository outside the attachments is exit 4; `runner log` and
225  `runner done` refused for an unattached build; `repo runner add` refuses
226  a full-scope key and a deploy key; add is idempotent. The existing
227  `TestStdinCommandsReadStdin`, `TestReadOnlyCommandsWriteNothing`, the CLI
228  table coverage test and the Parity test cover the new commands without
229  changes. The e2e tests that add a scope-`runner` key today
230  (`buildcancelweb_test.go`, `reap_test.go`, `runner_scope_test.go`) gain an
231  attach step, since an unattached key no longer claims.
232- Runner: `init` writes key and config with the right modes and never
233  overwrites; config values are overridden by flags; `-identity` reaches the
234  ssh and git command lines.
235- e2e, one test: `init` against the test instance, attach over SSH with the
236  printed key, start the runner with the generated config, a push builds
237  and succeeds, a merge request head from a fork stays pending; with
238  `-untrusted` it builds.
239
240## Rollout
241
2421. Server, store, control, web, docs, tests: one MR against `main`, with
243   migration 0050. Deployed, the change closes the claim hole for every
244   existing scope-`runner` key on the instance: none has attachments, so
245   none claims. That includes the bay1 runner, which polls as the
246   non-admin account `ci` with a scope-`runner` key. Right after the
247   deploy, attach it as the admin:
248
249   ```
250   ssh -p 2222 root@bay1 cat /var/lib/gitbay-runner/.ssh/id_ed25519.pub \
251     | gitbay repo runner add krz/gitbay
252   ssh -p 2222 root@bay1 cat /var/lib/gitbay-runner/.ssh/id_ed25519.pub \
253     | gitbay repo runner add cmc/ci-smoke
254   ```
255
256   Builds queued between the deploy and the attach wait; none is lost.
257   `-untrusted` goes into the unit in the same deploy.
2582. Runner: `init`, config file, `-identity`, `-untrusted`. Same MR or the
259   next; the server change does not depend on it.
2603. Tap: formula and the `gitbay.rb` bump, after the release that carries
261   the runner change is tagged.