docs/plans/2026-09-08-user-runners.md
2378 lines · 85415 bytes
1# Runners attached to repositories: implementation plan
2
3> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
4
5**Goal:** A `gitbay-runner` anyone installs, pairs with their repositories on any instance, and runs as a service; the server hands a runner key only the builds of repositories it is attached to.
6
7**Architecture:** One new table (`runner_repos`) maps an SSH key to repositories; `runner next` claims only from a key's attachments and skips untrusted builds unless asked; `repo runner add|list|remove` manage attachments and render on the settings page. The runner gains `init`, a config file, its own identity, and `-untrusted`.
8
9**Tech Stack:** Go, SQLite via hand-written SQL, `github.com/BurntSushi/toml` (already a dependency), Go templates, e2e tests against real ssh/git.
10
11**Spec:** `docs/specs/2026-09-08-user-runners-design.md`
12
13## Global Constraints
14
15- Commit messages: `<area>, <area>: <what>` on the first line, body with `Ref #184`. No attribution trailers of any kind (top rule of `~/CLAUDE.md`).
16- Never push to `main`. Work on branch `user-runners`; MR at the end.
17- Locally: `go build ./... && go vet ./...`, the unit tests of the touched packages, and at most the one e2e test being written. The full suite runs in CI on bay1.
18- Every control command parses argv through `parseFlags` (`internal/control/flags.go`). A command that reads stdin sets `ReadsStdin: true`. A read command sets `ReadOnly: true`.
19- Every new control command needs a `pass()` entry in `cmd/gitbay/main.go`; a coverage test fails otherwise.
20- Secrets never in argv, never logged. A public key is not a secret.
21- Templates: `str`/`field` are nil-safe helpers; the whole stylesheet is `internal/web/static/style.css`.
22- Migrations: next number is `0050`, both `.up.sql` and `.down.sql`. Foreign keys are on.
23- Comments and docs in plain English, no hype. Wiki is `.gitbay/wiki/*.org`.
24
25---
26
27### Task 1: Store: ClaimBuild skips untrusted builds unless asked
28
29**Files:**
30- Modify: `internal/store/builds.go:80-118` (`ClaimBuild`)
31- Modify: `internal/store/builds_test.go` (every `ClaimBuild(` call gains `, false`; one new test)
32- Modify: `internal/store/queues_test.go:27` (`ClaimBuild(nil, false)`)
33
34**Interfaces:**
35- Produces: `Store.ClaimBuild(repoIDs []int64, untrusted bool) (Build, bool, error)`. With `untrusted` false only rows with `trusted = 1` are candidates.
36
37- [ ] **Step 1: Write the failing test**
38
39Append to `internal/store/builds_test.go`:
40
41```go
42// A merge request head from a fork is untrusted. A claim skips it unless
43// the runner asked for untrusted builds, so a runner on someone's laptop
44// never executes a stranger's branch by default.
45func TestClaimBuildSkipsUntrustedUnlessAsked(t *testing.T) {
46 s := open(t)
47 if err := s.MigrateUp(); err != nil {
48 t.Fatal(err)
49 }
50 uid, err := s.CreateUser("cmc", true)
51 if err != nil {
52 t.Fatal(err)
53 }
54 repo, err := s.CreateRepo("user", uid, "app", "public")
55 if err != nil {
56 t.Fatal(err)
57 }
58 // Queued first, so an unfiltered claim would take it.
59 forkBuild, err := s.CreateBuild(repo, "unit", "abc123", "refs/merge-requests/1/head", `["true"]`, "", "", false)
60 if err != nil {
61 t.Fatal(err)
62 }
63 own, err := s.CreateBuild(repo, "unit", "def456", "main", `["true"]`, "", "", true)
64 if err != nil {
65 t.Fatal(err)
66 }
67 b, ok, err := s.ClaimBuild(nil, false)
68 if err != nil || !ok || b.Number != own {
69 t.Fatalf("trusted-only claim: err=%v ok=%v number=%d, want %d", err, ok, b.Number, own)
70 }
71 if _, ok, _ := s.ClaimBuild(nil, false); ok {
72 t.Fatal("trusted-only claim took the fork build")
73 }
74 b, ok, err = s.ClaimBuild(nil, true)
75 if err != nil || !ok || b.Number != forkBuild {
76 t.Fatalf("untrusted claim: err=%v ok=%v number=%d, want %d", err, ok, b.Number, forkBuild)
77 }
78}
79```
80
81- [ ] **Step 2: Run it to see it fail**
82
83Run: `go test ./internal/store -run TestClaimBuildSkipsUntrusted 2>&1 | head -5`
84Expected: compile error, too many arguments to `ClaimBuild`.
85
86- [ ] **Step 3: Change `ClaimBuild`**
87
88Replace the signature, doc comment and query construction in `internal/store/builds.go`:
89
90```go
91// ClaimBuild atomically hands the oldest pending build to a runner and
92// marks it running. A non-empty repoIDs restricts the claim to those
93// repositories. Untrusted builds — merge request heads from another
94// repository — are skipped unless untrusted is set: they run a stranger's
95// code, which only a runner that isolates should take.
96func (s *Store) ClaimBuild(repoIDs []int64, untrusted bool) (Build, bool, error) {
97 tx, err := s.DB.Begin()
98 if err != nil {
99 return Build{}, false, err
100 }
101 defer tx.Rollback()
102 query := "SELECT id FROM builds WHERE status = 'pending'"
103 args := []any{}
104 if !untrusted {
105 query += " AND trusted = 1"
106 }
107 if len(repoIDs) > 0 {
108 marks := strings.TrimSuffix(strings.Repeat("?,", len(repoIDs)), ",")
109 query += " AND repo_id IN (" + marks + ")"
110 for _, id := range repoIDs {
111 args = append(args, id)
112 }
113 }
114 query += " ORDER BY id LIMIT 1"
115 var id int64
116 err = tx.QueryRow(query, args...).Scan(&id)
117```
118
119The rest of the function is unchanged.
120
121- [ ] **Step 4: Update existing callers in store tests**
122
123In `internal/store/builds_test.go` and `internal/store/queues_test.go`, every `s.ClaimBuild(x)` becomes `s.ClaimBuild(x, false)`:
124
125```bash
126sed -i '' -E 's/ClaimBuild\((nil|\[\]int64\{[a-zA-Z]+\})\)/ClaimBuild(\1, false)/g' internal/store/builds_test.go internal/store/queues_test.go
127grep -n "ClaimBuild(" internal/store/*_test.go
128```
129
130Every hit must now show two arguments.
131
132- [ ] **Step 5: Run the store tests**
133
134Run: `go test ./internal/store 2>&1 | tail -3`
135Expected: PASS.
136
137- [ ] **Step 6: Commit**
138
139```bash
140git add internal/store/builds.go internal/store/builds_test.go internal/store/queues_test.go
141git commit -m "store: ClaimBuild skips untrusted builds unless asked
142
143Ref #184"
144```
145
146---
147
148### Task 2: Store: migration 0050 and runner attachments
149
150**Files:**
151- Create: `internal/store/migrations/0050_runner_repos.up.sql`
152- Create: `internal/store/migrations/0050_runner_repos.down.sql`
153- Modify: `internal/store/runners.go` (whole file)
154- Create: `internal/store/runners_test.go`
155
156**Interfaces:**
157- Consumes: `Store.AddSSHKey(userID int64, fingerprint, algo string, blob []byte, scope string) error`, `Store.SSHKeyByFingerprint(fp) (SSHKey, error)`, `Store.CreateUser(name string, admin bool) (int64, error)`, `Store.CreateRepo(kind string, ownerID int64, name, visibility string) (int64, error)`, `Store.CreateBuild(repoID int64, job, sha, ref, steps, image, tree string, trusted bool) (int64, error)`.
158- Produces:
159 - `type RepoRunner struct { Fingerprint, Algo, Username, AddedAt, LastSeen, BuildRepo string; BuildNumber int64; BuildJob, StartedAt string }`
160 - `Runner` gains `Fingerprint string` and `KeyID int64`.
161 - `Store.AttachRunner(keyID, repoID int64) error` (idempotent)
162 - `Store.DetachRunner(repoID int64, fingerprint string) error` (`ErrNotFound` when not attached)
163 - `Store.RunnerRepoIDs(keyID int64) ([]int64, error)`
164 - `Store.RunnerRepoPaths(keyID int64) ([]string, error)` (owner/name, sorted)
165 - `Store.RunnerAttached(keyID, repoID int64) (bool, error)`
166 - `Store.ListRepoRunners(repoID int64) ([]RepoRunner, error)`
167 - `Store.TouchRunner(keyID, userID int64, scope string, buildID int64) error`
168 - `Store.RunnerDone(keyID int64) error`
169 - `Store.ListRunners() ([]Runner, error)` unchanged signature.
170
171- [ ] **Step 1: Write the migration**
172
173`internal/store/migrations/0050_runner_repos.up.sql`:
174
175```sql
176-- A runner key is attached to the repositories it may claim builds for
177-- (#184). runner_seen is rekeyed by key so two runners on one account
178-- are two rows; what it held were heartbeats, so the rows are dropped.
179CREATE TABLE runner_repos (
180 key_id INTEGER NOT NULL REFERENCES ssh_keys(id) ON DELETE CASCADE,
181 repo_id INTEGER NOT NULL REFERENCES repos(id) ON DELETE CASCADE,
182 added_at TEXT NOT NULL DEFAULT (strftime('%Y-%m-%dT%H:%M:%fZ','now')),
183 PRIMARY KEY (key_id, repo_id)
184);
185CREATE INDEX runner_repos_repo ON runner_repos(repo_id);
186
187DROP TABLE runner_seen;
188CREATE TABLE runner_seen (
189 key_id INTEGER PRIMARY KEY REFERENCES ssh_keys(id) ON DELETE CASCADE,
190 user_id INTEGER NOT NULL REFERENCES users(id) ON DELETE CASCADE,
191 last_seen TEXT NOT NULL,
192 scope TEXT NOT NULL DEFAULT '',
193 build_id INTEGER REFERENCES builds(id) ON DELETE SET NULL
194);
195```
196
197`internal/store/migrations/0050_runner_repos.down.sql`:
198
199```sql
200DROP TABLE runner_repos;
201DROP TABLE runner_seen;
202CREATE TABLE runner_seen (
203 user_id INTEGER PRIMARY KEY REFERENCES users(id) ON DELETE CASCADE,
204 last_seen TEXT NOT NULL,
205 scope TEXT NOT NULL DEFAULT '',
206 build_id INTEGER REFERENCES builds(id) ON DELETE SET NULL
207);
208```
209
210- [ ] **Step 2: Write the failing store tests**
211
212`internal/store/runners_test.go`:
213
214```go
215package store
216
217import (
218 "errors"
219 "testing"
220)
221
222// runnerFixture is one user with a runner key and two repositories.
223func runnerFixture(t *testing.T) (s *Store, uid, keyID, repoA, repoB int64) {
224 t.Helper()
225 s = open(t)
226 if err := s.MigrateUp(); err != nil {
227 t.Fatal(err)
228 }
229 uid, err := s.CreateUser("alice", false)
230 if err != nil {
231 t.Fatal(err)
232 }
233 if err := s.AddSSHKey(uid, "SHA256:runnerkey", "ssh-ed25519", []byte("blob"), "runner"); err != nil {
234 t.Fatal(err)
235 }
236 k, err := s.SSHKeyByFingerprint("SHA256:runnerkey")
237 if err != nil {
238 t.Fatal(err)
239 }
240 repoA, err = s.CreateRepo("user", uid, "a", "public")
241 if err != nil {
242 t.Fatal(err)
243 }
244 repoB, err = s.CreateRepo("user", uid, "b", "public")
245 if err != nil {
246 t.Fatal(err)
247 }
248 return s, uid, k.ID, repoA, repoB
249}
250
251// Attaching twice is one row; detaching what is not attached is not found.
252func TestAttachRunnerIdempotentAndDetach(t *testing.T) {
253 s, _, keyID, repoA, repoB := runnerFixture(t)
254 for range 2 {
255 if err := s.AttachRunner(keyID, repoA); err != nil {
256 t.Fatal(err)
257 }
258 }
259 ids, err := s.RunnerRepoIDs(keyID)
260 if err != nil || len(ids) != 1 || ids[0] != repoA {
261 t.Fatalf("attached repos %v err=%v, want [%d]", ids, err, repoA)
262 }
263 if ok, _ := s.RunnerAttached(keyID, repoB); ok {
264 t.Fatal("attached to a repo it was never attached to")
265 }
266 if err := s.DetachRunner(repoB, "SHA256:runnerkey"); !errors.Is(err, ErrNotFound) {
267 t.Fatalf("detach of an unattached repo: %v, want ErrNotFound", err)
268 }
269 if err := s.DetachRunner(repoA, "SHA256:runnerkey"); err != nil {
270 t.Fatal(err)
271 }
272 if ok, _ := s.RunnerAttached(keyID, repoA); ok {
273 t.Fatal("still attached after detach")
274 }
275}
276
277// Removing the key or the repository removes the attachment with it.
278func TestRunnerAttachmentCascades(t *testing.T) {
279 s, uid, keyID, repoA, repoB := runnerFixture(t)
280 if err := s.AttachRunner(keyID, repoA); err != nil {
281 t.Fatal(err)
282 }
283 if err := s.AttachRunner(keyID, repoB); err != nil {
284 t.Fatal(err)
285 }
286 if _, err := s.DB.Exec("DELETE FROM repos WHERE id = ?", repoB); err != nil {
287 t.Fatal(err)
288 }
289 if ids, _ := s.RunnerRepoIDs(keyID); len(ids) != 1 {
290 t.Fatalf("after repo delete: %v, want one attachment", ids)
291 }
292 if err := s.RemoveSSHKey(uid, "SHA256:runnerkey"); err != nil {
293 t.Fatal(err)
294 }
295 var n int
296 if err := s.DB.QueryRow("SELECT count(*) FROM runner_repos").Scan(&n); err != nil || n != 0 {
297 t.Fatalf("after key delete: %d rows err=%v, want 0", n, err)
298 }
299}
300
301// The heartbeat is per key: two keys on one account are two rows, and a
302// repository's runner list shows each key's last poll and the build it holds.
303func TestRunnerSeenPerKeyAndRepoList(t *testing.T) {
304 s, uid, keyID, repoA, _ := runnerFixture(t)
305 if err := s.AddSSHKey(uid, "SHA256:second", "ssh-ed25519", []byte("blob2"), "runner"); err != nil {
306 t.Fatal(err)
307 }
308 k2, _ := s.SSHKeyByFingerprint("SHA256:second")
309 for _, id := range []int64{keyID, k2.ID} {
310 if err := s.AttachRunner(id, repoA); err != nil {
311 t.Fatal(err)
312 }
313 }
314 if _, err := s.CreateBuild(repoA, "unit", "abc123", "main", `["true"]`, "", "", true); err != nil {
315 t.Fatal(err)
316 }
317 b, ok, err := s.ClaimBuild(nil, false)
318 if err != nil || !ok {
319 t.Fatalf("claim: %v ok=%v", err, ok)
320 }
321 if err := s.TouchRunner(keyID, uid, "", b.ID); err != nil {
322 t.Fatal(err)
323 }
324 if err := s.TouchRunner(k2.ID, uid, "", 0); err != nil {
325 t.Fatal(err)
326 }
327 runners, err := s.ListRunners()
328 if err != nil || len(runners) != 2 {
329 t.Fatalf("ListRunners: %v err=%v, want two rows", runners, err)
330 }
331 list, err := s.ListRepoRunners(repoA)
332 if err != nil || len(list) != 2 {
333 t.Fatalf("ListRepoRunners: %v err=%v, want two rows", list, err)
334 }
335 var held, idle int
336 for _, r := range list {
337 if r.Username != "alice" || r.LastSeen == "" || r.AddedAt == "" {
338 t.Fatalf("row %+v lacks username, last_seen or added_at", r)
339 }
340 if r.BuildNumber == b.Number && r.BuildJob == "unit" && r.BuildRepo == "alice/a" {
341 held++
342 } else if r.BuildNumber == 0 {
343 idle++
344 }
345 }
346 if held != 1 || idle != 1 {
347 t.Fatalf("held=%d idle=%d, want 1 and 1: %+v", held, idle, list)
348 }
349 if err := s.RunnerDone(keyID); err != nil {
350 t.Fatal(err)
351 }
352 list, _ = s.ListRepoRunners(repoA)
353 for _, r := range list {
354 if r.BuildNumber != 0 {
355 t.Fatalf("build still held after RunnerDone: %+v", r)
356 }
357 }
358 paths, err := s.RunnerRepoPaths(keyID)
359 if err != nil || len(paths) != 1 || paths[0] != "alice/a" {
360 t.Fatalf("RunnerRepoPaths: %v err=%v", paths, err)
361 }
362}
363```
364
365- [ ] **Step 3: Run the tests to see them fail**
366
367Run: `go test ./internal/store -run 'TestAttachRunner|TestRunnerAttachment|TestRunnerSeen' 2>&1 | head -20`
368Expected: compile errors, `s.AttachRunner undefined` and friends.
369
370- [ ] **Step 4: Replace `internal/store/runners.go`**
371
372```go
373package store
374
375import "sort"
376
377// Runner is one runner key as the instance admin sees it.
378type Runner struct {
379 Username string `json:"username"`
380 Fingerprint string `json:"fingerprint"`
381 KeyID int64 `json:"-"`
382 LastSeen string `json:"last_seen"`
383 // Scope is what the runner asked for: comma-joined owner/name, ""
384 // for any. admin runners replaces it with the attachments for a
385 // runner key.
386 Scope string `json:"scope,omitempty"`
387 // The build it holds, if any.
388 BuildRepo string `json:"build_repo,omitempty"`
389 BuildNumber int64 `json:"build_number,omitempty"`
390 BuildJob string `json:"build_job,omitempty"`
391 StartedAt string `json:"started_at,omitempty"`
392}
393
394// RepoRunner is one key attached to a repository, as repo runner list
395// shows it.
396type RepoRunner struct {
397 Fingerprint string `json:"fingerprint"`
398 Algo string `json:"algo"`
399 Username string `json:"username"`
400 AddedAt string `json:"added_at"`
401 LastSeen string `json:"last_seen,omitempty"`
402 BuildRepo string `json:"build_repo,omitempty"`
403 BuildNumber int64 `json:"build_number,omitempty"`
404 BuildJob string `json:"build_job,omitempty"`
405 StartedAt string `json:"started_at,omitempty"`
406}
407
408// AttachRunner lets a key claim a repository's builds. Attaching twice is
409// one row.
410func (s *Store) AttachRunner(keyID, repoID int64) error {
411 _, err := s.DB.Exec("INSERT OR IGNORE INTO runner_repos (key_id, repo_id) VALUES (?, ?)", keyID, repoID)
412 return err
413}
414
415// DetachRunner removes one attachment by fingerprint. The key itself stays.
416func (s *Store) DetachRunner(repoID int64, fingerprint string) error {
417 res, err := s.DB.Exec(`DELETE FROM runner_repos WHERE repo_id = ?
418 AND key_id = (SELECT id FROM ssh_keys WHERE fingerprint = ?)`, repoID, fingerprint)
419 if err != nil {
420 return err
421 }
422 if n, _ := res.RowsAffected(); n == 0 {
423 return ErrNotFound
424 }
425 return nil
426}
427
428// RunnerRepoIDs is every repository a key is attached to.
429func (s *Store) RunnerRepoIDs(keyID int64) ([]int64, error) {
430 rows, err := s.DB.Query("SELECT repo_id FROM runner_repos WHERE key_id = ? ORDER BY repo_id", keyID)
431 if err != nil {
432 return nil, err
433 }
434 defer rows.Close()
435 var ids []int64
436 for rows.Next() {
437 var id int64
438 if err := rows.Scan(&id); err != nil {
439 return nil, err
440 }
441 ids = append(ids, id)
442 }
443 return ids, rows.Err()
444}
445
446// RunnerRepoPaths is RunnerRepoIDs as owner/name, sorted.
447func (s *Store) RunnerRepoPaths(keyID int64) ([]string, error) {
448 rows, err := s.DB.Query(`SELECT COALESCE(u.username, o.name) || '/' || r.name
449 FROM runner_repos rr JOIN repos r ON r.id = rr.repo_id
450 LEFT JOIN users u ON r.owner_kind = 'user' AND u.id = r.owner_id
451 LEFT JOIN orgs o ON r.owner_kind = 'org' AND o.id = r.owner_id
452 WHERE rr.key_id = ?`, keyID)
453 if err != nil {
454 return nil, err
455 }
456 defer rows.Close()
457 var paths []string
458 for rows.Next() {
459 var p string
460 if err := rows.Scan(&p); err != nil {
461 return nil, err
462 }
463 paths = append(paths, p)
464 }
465 sort.Strings(paths)
466 return paths, rows.Err()
467}
468
469// RunnerAttached reports whether a key may claim a repository's builds.
470func (s *Store) RunnerAttached(keyID, repoID int64) (bool, error) {
471 var n int
472 err := s.DB.QueryRow("SELECT count(*) FROM runner_repos WHERE key_id = ? AND repo_id = ?", keyID, repoID).Scan(&n)
473 return n > 0, err
474}
475
476// ListRepoRunners is every key attached to a repository with its last
477// poll and the build it holds, oldest attachment first.
478func (s *Store) ListRepoRunners(repoID int64) ([]RepoRunner, error) {
479 rows, err := s.DB.Query(`SELECT k.fingerprint, k.algo, u.username, rr.added_at,
480 COALESCE(rs.last_seen, ''),
481 COALESCE(COALESCE(bu.username, bo.name) || '/' || br.name, ''),
482 COALESCE(b.number, 0), COALESCE(b.job, ''), COALESCE(b.started_at, '')
483 FROM runner_repos rr
484 JOIN ssh_keys k ON k.id = rr.key_id
485 JOIN users u ON u.id = k.user_id
486 LEFT JOIN runner_seen rs ON rs.key_id = rr.key_id
487 LEFT JOIN builds b ON b.id = rs.build_id AND b.status = 'running'
488 LEFT JOIN repos br ON br.id = b.repo_id
489 LEFT JOIN users bu ON br.owner_kind = 'user' AND bu.id = br.owner_id
490 LEFT JOIN orgs bo ON br.owner_kind = 'org' AND bo.id = br.owner_id
491 WHERE rr.repo_id = ? ORDER BY rr.added_at, k.id`, repoID)
492 if err != nil {
493 return nil, err
494 }
495 defer rows.Close()
496 var out []RepoRunner
497 for rows.Next() {
498 var r RepoRunner
499 if err := rows.Scan(&r.Fingerprint, &r.Algo, &r.Username, &r.AddedAt, &r.LastSeen,
500 &r.BuildRepo, &r.BuildNumber, &r.BuildJob, &r.StartedAt); err != nil {
501 return nil, err
502 }
503 out = append(out, r)
504 }
505 return out, rows.Err()
506}
507
508// TouchRunner records a poll by one key: the time, the scope the runner
509// asked for, and the build it just claimed (0 for none).
510func (s *Store) TouchRunner(keyID, userID int64, scope string, buildID int64) error {
511 _, err := s.DB.Exec(`INSERT INTO runner_seen (key_id, user_id, last_seen, scope, build_id)
512 VALUES (?1, ?2, strftime('%Y-%m-%dT%H:%M:%fZ','now'), ?3, NULLIF(?4, 0))
513 ON CONFLICT (key_id) DO UPDATE SET
514 last_seen = excluded.last_seen, scope = excluded.scope,
515 build_id = COALESCE(excluded.build_id, runner_seen.build_id)`,
516 keyID, userID, scope, buildID)
517 return err
518}
519
520// RunnerDone records that the key reported and holds nothing now.
521func (s *Store) RunnerDone(keyID int64) error {
522 _, err := s.DB.Exec(`UPDATE runner_seen SET last_seen = strftime('%Y-%m-%dT%H:%M:%fZ','now'),
523 build_id = NULL WHERE key_id = ?`, keyID)
524 return err
525}
526
527// ListRunners lists every key that has ever polled as a runner, most
528// recently seen first.
529func (s *Store) ListRunners() ([]Runner, error) {
530 rows, err := s.DB.Query(`SELECT u.username, k.fingerprint, k.id, r.last_seen, r.scope,
531 COALESCE(COALESCE(bu.username, bo.name) || '/' || br.name, ''),
532 COALESCE(b.number, 0), COALESCE(b.job, ''), COALESCE(b.started_at, '')
533 FROM runner_seen r JOIN users u ON u.id = r.user_id
534 JOIN ssh_keys k ON k.id = r.key_id
535 LEFT JOIN builds b ON b.id = r.build_id AND b.status = 'running'
536 LEFT JOIN repos br ON br.id = b.repo_id
537 LEFT JOIN users bu ON br.owner_kind = 'user' AND bu.id = br.owner_id
538 LEFT JOIN orgs bo ON br.owner_kind = 'org' AND bo.id = br.owner_id
539 ORDER BY r.last_seen DESC`)
540 if err != nil {
541 return nil, err
542 }
543 defer rows.Close()
544 var out []Runner
545 for rows.Next() {
546 var r Runner
547 if err := rows.Scan(&r.Username, &r.Fingerprint, &r.KeyID, &r.LastSeen, &r.Scope,
548 &r.BuildRepo, &r.BuildNumber, &r.BuildJob, &r.StartedAt); err != nil {
549 return nil, err
550 }
551 out = append(out, r)
552 }
553 return out, rows.Err()
554}
555```
556
557- [ ] **Step 5: Run the store tests**
558
559Run: `go test ./internal/store 2>&1 | tail -5`
560Expected: PASS. Callers in `internal/control` do not compile yet; that is Task 3. `go build ./internal/store` must pass here.
561
562- [ ] **Step 6: Commit**
563
564```bash
565git add internal/store/migrations/0050_runner_repos.up.sql internal/store/migrations/0050_runner_repos.down.sql internal/store/runners.go internal/store/runners_test.go
566git commit -m "store: runner keys attach to repositories, heartbeat per key
567
568Migration 0050 adds runner_repos and rekeys runner_seen by ssh key.
569
570Ref #184"
571```
572
573---
574
575### Task 3: Control: the claim rule, per-key heartbeat, and admin runners
576
577**Files:**
578- Modify: `internal/control/build.go` (`requireRunner`, `runRunnerNext`, `runRunnerLog`, `runRunnerDone`, the `runner next` registration)
579- Modify: `internal/control/admin.go:444-475` (`runAdminRunners`)
580- Modify: `internal/control/runnernext_test.go:17-30` (`runnerCtx`)
581- Create: `internal/control/runnerattach_test.go`
582- Modify: `e2e/reap_test.go:112-115` (the admin runners row gains a fingerprint column)
583- Modify: `e2e/mrbuilds_test.go:95`, `e2e/build_cancel_test.go` (claims of a fork head pass `--untrusted`)
584
585**Interfaces:**
586- Consumes: the store functions from Tasks 1 and 2; `Ctx.Source` is the SSH key fingerprint for SSH sessions (`internal/sshd/sshd.go:338`).
587- Produces:
588 - `runnerSession(c *Ctx) (store.SSHKey, int)`: the key behind the session, or an exit code.
589 - `runnerMayBuild(c *Ctx, key store.SSHKey, repoID int64) (bool, error)`: admin, or attached.
590 - `runner next [--untrusted] [<owner/name>...]`.
591 - `admin runners` JSON rows carry `fingerprint`; the text row is `username<TAB>fingerprint<TAB>last_seen<TAB>scope<TAB>held`.
592
593- [ ] **Step 1: Write the failing control tests**
594
595`internal/control/runnerattach_test.go`:
596
597```go
598package control
599
600import (
601 "bytes"
602 "strconv"
603 "strings"
604 "testing"
605
606 "gitbay.org/gitbay/internal/config"
607 "gitbay.org/gitbay/internal/protocol"
608 "gitbay.org/gitbay/internal/store"
609)
610
611// attachFixture: alice (not admin) owns alice/app with a build queued;
612// mallory (not admin) owns mallory/evil with an older build queued. Each
613// has a runner-scoped key. The Ctx polls as the given user with the given
614// key, which is what the SSH listener produces.
615type attachFixture struct {
616 st *store.Store
617 alice, mallory int64
618 aliceKey, malloryKey store.SSHKey
619 app, evil store.Repo
620 appBuild, evilBuild int64
621}
622
623func newAttachFixture(t *testing.T) attachFixture {
624 t.Helper()
625 st, err := store.Open(":memory:")
626 if err != nil {
627 t.Fatal(err)
628 }
629 t.Cleanup(func() { st.Close() })
630 if err := st.MigrateUp(); err != nil {
631 t.Fatal(err)
632 }
633 var f attachFixture
634 f.st = st
635 mk := func(name, fp string) (int64, store.SSHKey, store.Repo, string) {
636 uid, err := st.CreateUser(name, false)
637 if err != nil {
638 t.Fatal(err)
639 }
640 if err := st.AddSSHKey(uid, fp, "ssh-ed25519", []byte(fp), "runner"); err != nil {
641 t.Fatal(err)
642 }
643 k, _ := st.SSHKeyByFingerprint(fp)
644 repoName := map[string]string{"alice": "app", "mallory": "evil"}[name]
645 rid, err := st.CreateRepo("user", uid, repoName, "public")
646 if err != nil {
647 t.Fatal(err)
648 }
649 repo, _ := st.RepoByID(rid)
650 return uid, k, repo, repoName
651 }
652 f.mallory, f.malloryKey, f.evil, _ = mk("mallory", "SHA256:mallory")
653 f.alice, f.aliceKey, f.app, _ = mk("alice", "SHA256:alice")
654 // mallory's build is older, so an unrestricted claim would take it.
655 f.evilBuild, err = st.CreateBuild(f.evil.ID, "unit", "aaa111", "main", "[]", "", "", true)
656 if err != nil {
657 t.Fatal(err)
658 }
659 f.appBuild, err = st.CreateBuild(f.app.ID, "unit", "bbb222", "main", "[]", "", "", true)
660 if err != nil {
661 t.Fatal(err)
662 }
663 return f
664}
665
666func (f attachFixture) ctx(uid int64, key store.SSHKey, admin bool) (*Ctx, *bytes.Buffer) {
667 var out bytes.Buffer
668 name := "alice"
669 if uid == f.mallory {
670 name = "mallory"
671 }
672 return &Ctx{
673 User: store.User{ID: uid, Username: name, IsAdmin: admin},
674 Scope: key.Scope,
675 Source: key.Fingerprint,
676 Store: f.st,
677 Cfg: config.Config{Server: config.Server{Root: "/nonexistent", SiteURL: "https://x.test"}},
678 Stdin: strings.NewReader(""),
679 Stdout: &out,
680 Stderr: &out,
681 }, &out
682}
683
684// A runner key with no attachment claims nothing, whatever is queued.
685func TestRunnerNextUnattachedClaimsNothing(t *testing.T) {
686 f := newAttachFixture(t)
687 c, out := f.ctx(f.alice, f.aliceKey, false)
688 if code := runRunnerNext(c, nil); code != protocol.ExitOK || !strings.Contains(out.String(), "no pending builds") {
689 t.Fatalf("exit %d: %s", code, out.String())
690 }
691 b, _ := f.st.BuildByNumber(f.evil.ID, f.evilBuild)
692 if b.Status != "pending" {
693 t.Fatalf("unattached key claimed a build: %s", b.Status)
694 }
695}
696
697// An attached key claims its repository's build and not the older one
698// queued elsewhere; naming a repository outside the attachments is refused.
699func TestRunnerNextAttachedClaimsOwnRepoOnly(t *testing.T) {
700 f := newAttachFixture(t)
701 if err := f.st.AttachRunner(f.aliceKey.ID, f.app.ID); err != nil {
702 t.Fatal(err)
703 }
704 c, out := f.ctx(f.alice, f.aliceKey, false)
705 if code := runRunnerNext(c, nil); code != protocol.ExitOK || !strings.Contains(out.String(), "alice/app") {
706 t.Fatalf("exit %d: %s", code, out.String())
707 }
708 if b, _ := f.st.BuildByNumber(f.evil.ID, f.evilBuild); b.Status != "pending" {
709 t.Fatalf("mallory's build was touched: %s", b.Status)
710 }
711 c, out = f.ctx(f.alice, f.aliceKey, false)
712 if code := runRunnerNext(c, []string{"mallory/evil"}); code != protocol.ExitDenied {
713 t.Fatalf("naming an unattached repo: exit %d, want %d: %s", code, protocol.ExitDenied, out.String())
714 }
715}
716
717// The heartbeat is recorded against the key, and admin runners shows it
718// with its fingerprint and attachments.
719func TestAdminRunnersShowsKeyAndAttachments(t *testing.T) {
720 f := newAttachFixture(t)
721 if err := f.st.AttachRunner(f.aliceKey.ID, f.app.ID); err != nil {
722 t.Fatal(err)
723 }
724 c, _ := f.ctx(f.alice, f.aliceKey, false)
725 runRunnerNext(c, nil)
726 admin, out := f.ctx(f.alice, f.aliceKey, true)
727 admin.Scope = "full"
728 if code := runAdminRunners(admin, nil); code != protocol.ExitOK {
729 t.Fatalf("admin runners: exit %d: %s", code, out.String())
730 }
731 if !strings.Contains(out.String(), "alice\tSHA256:alice\t") || !strings.Contains(out.String(), "\talice/app\t") {
732 t.Fatalf("row lacks fingerprint or attachments:\n%s", out.String())
733 }
734}
735
736// Untrusted builds are skipped unless the runner asks.
737func TestRunnerNextUntrustedFlag(t *testing.T) {
738 f := newAttachFixture(t)
739 if err := f.st.AttachRunner(f.aliceKey.ID, f.app.ID); err != nil {
740 t.Fatal(err)
741 }
742 c, _ := f.ctx(f.alice, f.aliceKey, false)
743 runRunnerNext(c, nil) // takes the trusted build
744 fork, err := f.st.CreateBuild(f.app.ID, "unit", "ccc333", "refs/merge-requests/1/head", "[]", "", "", false)
745 if err != nil {
746 t.Fatal(err)
747 }
748 c, out := f.ctx(f.alice, f.aliceKey, false)
749 runRunnerNext(c, nil)
750 if !strings.Contains(out.String(), "no pending builds") {
751 t.Fatalf("fork head claimed without --untrusted: %s", out.String())
752 }
753 c, out = f.ctx(f.alice, f.aliceKey, false)
754 if code := runRunnerNext(c, []string{"--untrusted"}); code != protocol.ExitOK || !strings.Contains(out.String(), "alice/app") {
755 t.Fatalf("--untrusted did not claim the fork head: exit %d %s", code, out.String())
756 }
757 if b, _ := f.st.BuildByNumber(f.app.ID, fork); b.Status != "running" {
758 t.Fatalf("fork build is %s, want running", b.Status)
759 }
760}
761
762// runner done and runner log on a build whose repository is not attached
763// to the key are refused.
764func TestRunnerDoneRefusedForUnattachedBuild(t *testing.T) {
765 f := newAttachFixture(t)
766 if err := f.st.AttachRunner(f.malloryKey.ID, f.evil.ID); err != nil {
767 t.Fatal(err)
768 }
769 c, _ := f.ctx(f.mallory, f.malloryKey, false)
770 runRunnerNext(c, nil) // mallory holds her own build
771 evil, _ := f.st.BuildByNumber(f.evil.ID, f.evilBuild)
772 c, out := f.ctx(f.alice, f.aliceKey, false)
773 id := strconv.FormatInt(evil.ID, 10)
774 if code := runRunnerDone(c, []string{id, "success"}); code != protocol.ExitDenied {
775 t.Fatalf("done on an unattached build: exit %d, want %d: %s", code, protocol.ExitDenied, out.String())
776 }
777 c, out = f.ctx(f.alice, f.aliceKey, false)
778 if code := runRunnerLog(c, []string{id}); code != protocol.ExitDenied {
779 t.Fatalf("log on an unattached build: exit %d, want %d: %s", code, protocol.ExitDenied, out.String())
780 }
781 if b, _ := f.st.BuildByNumber(f.evil.ID, f.evilBuild); b.Status != "running" {
782 t.Fatalf("build was finished by a foreign key: %s", b.Status)
783 }
784}
785```
786
787- [ ] **Step 2: Run them to see them fail**
788
789Run: `go test ./internal/control -run 'TestRunnerNext(Unattached|Attached|Untrusted)|TestAdminRunnersShows|TestRunnerDoneRefused' 2>&1 | head`
790Expected: compile errors from `ClaimBuild`, `TouchRunner`, `RunnerDone` signature changes in `build.go`.
791
792- [ ] **Step 3: Rewrite the runner protocol in `internal/control/build.go`**
793
794Change the registration:
795
796```go
797 register(Command{Path: []string{"runner", "next"},
798 Summary: "claim the oldest pending build this key may run (runner protocol)",
799 Usage: "runner next [--untrusted] [<owner/name>...]", SSHOnly: true, Run: runRunnerNext})
800```
801
802Replace `requireRunner` with two helpers:
803
804```go
805// runnerSession resolves the key behind a runner-protocol session. The
806// runner commands are SSHOnly, so Source is the key's fingerprint. An
807// admin key is accepted so an operator can rotate at their own pace; a
808// runner host should hold a key added with --scope runner.
809func runnerSession(c *Ctx) (store.SSHKey, int) {
810 if c.Scope != "runner" && !c.User.IsAdmin {
811 return store.SSHKey{}, c.fail(protocol.ExitDenied, "runner commands need a key added with --scope runner")
812 }
813 key, err := c.Store.SSHKeyByFingerprint(c.Source)
814 if err != nil {
815 return store.SSHKey{}, c.fail(protocol.ExitDenied, "runner commands need an SSH key session")
816 }
817 return key, -1
818}
819
820// runnerMayBuild reports whether a runner session may act on a
821// repository's builds: an admin user may on any, a runner key on the
822// repositories it is attached to (#184).
823func runnerMayBuild(c *Ctx, key store.SSHKey, repoID int64) (bool, error) {
824 if c.User.IsAdmin {
825 return true, nil
826 }
827 return c.Store.RunnerAttached(key.ID, repoID)
828}
829```
830
831Rewrite the head of `runRunnerNext` down to the claim loop:
832
833```go
834func runRunnerNext(c *Ctx, args []string) int {
835 key, code := runnerSession(c)
836 if code >= 0 {
837 return code
838 }
839 f, err := parseFlags(args, flagSpec{Bools: []string{"--untrusted"}, MaxPos: -1,
840 Usage: "runner next [--untrusted] [<owner/name>...]"})
841 if err != nil {
842 return c.fail(protocol.ExitUsage, "%v", err)
843 }
844 // The candidate set. An admin key claims from any repository, narrowed
845 // by the names given. A runner key claims from the repositories it is
846 // attached to; a name outside them is refused, not ignored, so a
847 // misconfigured runner says so instead of idling.
848 var repoIDs []int64
849 for _, arg := range f.Pos {
850 repo, code := resolveRepo(c, arg, policy.CanRead)
851 if code >= 0 {
852 return code
853 }
854 ok, err := runnerMayBuild(c, key, repo.ID)
855 if err != nil {
856 return c.fail(protocol.ExitFailure, "%v", err)
857 }
858 if !ok {
859 return c.fail(protocol.ExitDenied, "this key is not attached to %s", repo.Path())
860 }
861 repoIDs = append(repoIDs, repo.ID)
862 }
863 if !c.User.IsAdmin && len(repoIDs) == 0 {
864 repoIDs, err = c.Store.RunnerRepoIDs(key.ID)
865 if err != nil {
866 return c.fail(protocol.ExitFailure, "%v", err)
867 }
868 if len(repoIDs) == 0 {
869 // Nothing attached: nothing to claim. Still a heartbeat, so
870 // admin runners shows the key polling.
871 c.Store.TouchRunner(key.ID, c.User.ID, "", 0)
872 return c.emit(map[string]any{}, func(w io.Writer) { fmt.Fprintln(w, "no pending builds") })
873 }
874 }
875 untrusted := f.Has("--untrusted")
876 var b store.Build
877 var repo store.Repo
878 var ok bool
879 for attempt := 0; attempt < maxOrphanSkip; attempt++ {
880 b, ok, err = c.Store.ClaimBuild(repoIDs, untrusted)
881```
882
883The rest of the loop is unchanged. Delete the old `var err error` line, since `err` now comes from `parseFlags`. Replace the heartbeat line:
884
885```go
886 c.Store.TouchRunner(key.ID, c.User.ID, strings.Join(f.Pos, ","), b.ID)
887```
888
889In `runRunnerLog`, replace `if code := requireRunner(c); code >= 0 { return code }` with:
890
891```go
892 key, code := runnerSession(c)
893 if code >= 0 {
894 return code
895 }
896```
897
898and directly after the `id, err := strconv.ParseInt(args[0], 10, 64)` block, add:
899
900```go
901 if b, err := c.Store.BuildByID(id); err != nil {
902 return c.fail(protocol.ExitNotFound, "no build %d", id)
903 } else if ok, err := runnerMayBuild(c, key, b.RepoID); err != nil {
904 return c.fail(protocol.ExitFailure, "%v", err)
905 } else if !ok {
906 return c.fail(protocol.ExitDenied, "this key is not attached to the build's repository")
907 }
908```
909
910In `runRunnerDone`, the same `runnerSession` replacement; after `b, err := c.Store.BuildByID(id)` succeeds add:
911
912```go
913 if ok, err := runnerMayBuild(c, key, b.RepoID); err != nil {
914 return c.fail(protocol.ExitFailure, "%v", err)
915 } else if !ok {
916 return c.fail(protocol.ExitDenied, "this key is not attached to the build's repository")
917 }
918```
919
920and both `c.Store.RunnerDone(c.User.ID)` become `c.Store.RunnerDone(key.ID)`.
921
922Delete `requireRunner` if nothing else references it (`grep -n requireRunner internal/`).
923
924- [ ] **Step 4: `admin runners` shows the fingerprint and attachments**
925
926In `internal/control/admin.go`, `runAdminRunners`, after `ListRunners`:
927
928```go
929 for i := range runners {
930 if runners[i].Scope != "" {
931 continue
932 }
933 key, err := c.Store.SSHKeyByID(runners[i].KeyID)
934 if err != nil || key.Scope != "runner" {
935 continue // an admin key with no -repos: any
936 }
937 paths, err := c.Store.RunnerRepoPaths(runners[i].KeyID)
938 if err != nil {
939 return c.fail(protocol.ExitFailure, "%v", err)
940 }
941 runners[i].Scope = strings.Join(paths, ",")
942 }
943```
944
945A runner key that asked for `-repos` shows that; one that did not shows its attachments. Both are what the key may claim. Text row:
946
947```go
948 fmt.Fprintf(w, "%s\t%s\t%s\t%s\t%s\n", r.Username, r.Fingerprint, r.LastSeen, scope, held)
949```
950
951Add `"strings"` to admin.go imports if missing.
952
953- [ ] **Step 5: Fix the existing `runnerCtx` test helper**
954
955`internal/control/runnernext_test.go`, `runnerCtx`: the session needs a real key. Replace the helper body:
956
957```go
958func runnerCtx(st *store.Store, uid int64, root string) (*Ctx, *bytes.Buffer) {
959 var out bytes.Buffer
960 fp := fmt.Sprintf("SHA256:runner-%d", uid)
961 st.AddSSHKey(uid, fp, "ssh-ed25519", []byte(fp), "full") // ErrDuplicateKey on reuse is fine
962 c := &Ctx{
963 User: store.User{ID: uid, Username: "ci", IsAdmin: true},
964 Scope: "full",
965 Source: fp,
966 Store: st,
967 Cfg: config.Config{Server: config.Server{Root: root, SiteURL: "https://x.test"}},
968 Stdin: strings.NewReader(""),
969 Stdout: &out,
970 Stderr: &out,
971 }
972 return c, &out
973}
974```
975
976Any other control test that builds a `Ctx` for `runRunnerNext`, `runRunnerLog` or `runRunnerDone` (`grep -ln "runRunner" internal/control/*_test.go`) needs the same: an `AddSSHKey` and `Source` set to its fingerprint.
977
978- [ ] **Step 6: Run the control tests**
979
980Run: `go build ./... && go vet ./internal/control && go test ./internal/control 2>&1 | tail -5`
981Expected: PASS, including `TestStdinCommandsReadStdin` and `TestReadOnlyCommandsWriteNothing`.
982
983- [ ] **Step 7: Update the e2e tests that this changes**
984
985`e2e/reap_test.go:112-115`: the row is now `ci<TAB>SHA256:...<TAB>last_seen<TAB>alice/app<TAB>idle`. The existing assertions `strings.Contains(out, "\nci\t")` and `strings.Contains(out, "\talice/app\tidle")` still hold. No change unless the test fails; run it in Step 8.
986
987`e2e/mrbuilds_test.go:95` claims a fork head with an admin key. Add the flag:
988
989```go
990 out, errOut, code := inst.ssh(t, runnerKey, "", "runner", "next", "--untrusted", "alice/app", "--json")
991```
992
993`e2e/build_cancel_test.go`: find every `"runner", "next"` that claims a merge request head from a fork (`grep -n 'runner", "next"' e2e/build_cancel_test.go`, and read the test around each). Add `"--untrusted"` right after `"next"` where the queued build is a fork head. Same-repository branches are trusted and need nothing.
994
995- [ ] **Step 8: Run those e2e tests**
996
997Run: `go test ./e2e -run 'TestStaleBuildReapedWithoutRunner|TestForkMRHeadIsBuilt|TestRunnerNextScopedToRepos|TestRunnerScopedKey' -count=1 2>&1 | tail -5`
998Expected: PASS.
999
1000- [ ] **Step 9: Commit**
1001
1002```bash
1003git add internal/control/build.go internal/control/admin.go internal/control/runnernext_test.go internal/control/runnerattach_test.go e2e/reap_test.go e2e/mrbuilds_test.go e2e/build_cancel_test.go
1004git commit -m "control: a runner key claims only the repositories it is attached to
1005
1006runner next takes --untrusted; without it fork heads are skipped. runner
1007log and runner done refuse a build outside the key's attachments. The
1008heartbeat and admin runners are per key.
1009
1010Ref #184"
1011```
1012
1013---
1014
1015### Task 4: Control and CLI: `repo runner add|list|remove`
1016
1017**Files:**
1018- Create: `internal/control/runnerrepo.go`
1019- Create: `internal/control/runnerrepo_test.go`
1020- Modify: `cmd/gitbay/main.go:437-440` (a `group("runner", ...)` beside `deploy-key`)
1021
1022**Interfaces:**
1023- Consumes: the store functions from Task 2; `resolveRepo(c, path, policy.CanAdmin)`; `c.Store.Audit(userID, action string, fields map[string]any)`.
1024- Produces:
1025 - `repo runner add <owner/name>` (stdin: public key) → `{"fingerprint": ..., "repo": ...}`
1026 - `repo runner list <owner/name>` → `[]store.RepoRunner`
1027 - `repo runner remove <owner/name> <fingerprint>` → `{"removed": fingerprint}`
1028
1029- [ ] **Step 1: Write the failing tests**
1030
1031`internal/control/runnerrepo_test.go`:
1032
1033```go
1034package control
1035
1036import (
1037 "bytes"
1038 "strings"
1039 "testing"
1040
1041 "gitbay.org/gitbay/internal/config"
1042 "gitbay.org/gitbay/internal/protocol"
1043 "gitbay.org/gitbay/internal/store"
1044)
1045
1046// Generated once with ssh-keygen -t ed25519; a valid authorized_keys line.
1047const testRunnerPub = "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAILAr2r82jFsCJwsEyrEf2wgKy9Dv45xYYici6Ii7NyCS runner@test\n"
1048
1049func repoRunnerCtx(t *testing.T, st *store.Store, uid int64, admin bool, stdin string) (*Ctx, *bytes.Buffer) {
1050 t.Helper()
1051 var out bytes.Buffer
1052 return &Ctx{
1053 User: store.User{ID: uid, Username: "alice", IsAdmin: admin},
1054 Scope: "full",
1055 Source: "SHA256:session",
1056 Store: st,
1057 Cfg: config.Config{Server: config.Server{SiteURL: "https://x.test"}},
1058 Stdin: strings.NewReader(stdin),
1059 Stdout: &out,
1060 Stderr: &out,
1061 JSON: true,
1062 }, &out
1063}
1064
1065// A fresh key is registered on the caller's account with scope runner and
1066// attached; a second add is a no-op; list shows it; remove detaches and
1067// leaves the key on the account.
1068func TestRepoRunnerAddListRemove(t *testing.T) {
1069 st, repo, uid := newQueueTestRepo(t)
1070 c, out := repoRunnerCtx(t, st, uid, false, testRunnerPub)
1071 if code := runRepoRunnerAdd(c, []string{repo.Path()}); code != protocol.ExitOK {
1072 t.Fatalf("add: exit %d %s", code, out.String())
1073 }
1074 if !strings.Contains(out.String(), `"fingerprint":"SHA256:`) {
1075 t.Fatalf("add output: %s", out.String())
1076 }
1077 keys, _ := st.ListSSHKeys(uid)
1078 if len(keys) != 1 || keys[0].Scope != "runner" {
1079 t.Fatalf("key not registered as runner: %+v", keys)
1080 }
1081 fp := keys[0].Fingerprint
1082 c, out = repoRunnerCtx(t, st, uid, false, testRunnerPub)
1083 if code := runRepoRunnerAdd(c, []string{repo.Path()}); code != protocol.ExitOK {
1084 t.Fatalf("second add: exit %d %s", code, out.String())
1085 }
1086 c, out = repoRunnerCtx(t, st, uid, false, "")
1087 if code := runRepoRunnerList(c, []string{repo.Path()}); code != protocol.ExitOK || strings.Count(out.String(), fp) != 1 {
1088 t.Fatalf("list: exit %d %s", code, out.String())
1089 }
1090 c, out = repoRunnerCtx(t, st, uid, false, "")
1091 if code := runRepoRunnerRemove(c, []string{repo.Path(), fp}); code != protocol.ExitOK {
1092 t.Fatalf("remove: exit %d %s", code, out.String())
1093 }
1094 if ok, _ := st.RunnerAttached(keys[0].ID, repo.ID); ok {
1095 t.Fatal("still attached after remove")
1096 }
1097 if keys, _ = st.ListSSHKeys(uid); len(keys) != 1 {
1098 t.Fatal("remove dropped the key from the account")
1099 }
1100 c, out = repoRunnerCtx(t, st, uid, false, "")
1101 if code := runRepoRunnerRemove(c, []string{repo.Path(), fp}); code != protocol.ExitNotFound {
1102 t.Fatalf("remove twice: exit %d, want %d", code, protocol.ExitNotFound)
1103 }
1104}
1105
1106// A key that already exists with another scope is never promoted, and
1107// another account's runner key is refused unless the caller is an admin.
1108func TestRepoRunnerAddRefusesWrongKeys(t *testing.T) {
1109 st, repo, uid := newQueueTestRepo(t)
1110 c, _ := repoRunnerCtx(t, st, uid, false, testRunnerPub)
1111 // Register the same key as a full key first.
1112 if code := runKeysAdd(c, nil); code != protocol.ExitOK {
1113 t.Fatal("keys add failed")
1114 }
1115 c, out := repoRunnerCtx(t, st, uid, false, testRunnerPub)
1116 if code := runRepoRunnerAdd(c, []string{repo.Path()}); code != protocol.ExitDenied {
1117 t.Fatalf("full key accepted as runner: exit %d %s", code, out.String())
1118 }
1119 keys, _ := st.ListSSHKeys(uid)
1120 if keys[0].Scope != "full" {
1121 t.Fatalf("scope changed to %s", keys[0].Scope)
1122 }
1123 // Someone else's runner key.
1124 bob, _ := st.CreateUser("bob", false)
1125 if err := st.AddSSHKey(bob, "SHA256:bobrunner", "ssh-ed25519", []byte("x"), "runner"); err != nil {
1126 t.Fatal(err)
1127 }
1128 st.RemoveSSHKey(uid, keys[0].Fingerprint)
1129 if err := st.AddSSHKey(bob, keys[0].Fingerprint, "ssh-ed25519", keys[0].Blob, "runner"); err != nil {
1130 t.Fatal(err)
1131 }
1132 c, out = repoRunnerCtx(t, st, uid, false, testRunnerPub)
1133 if code := runRepoRunnerAdd(c, []string{repo.Path()}); code != protocol.ExitDenied {
1134 t.Fatalf("another account's key attached by a non-admin: exit %d %s", code, out.String())
1135 }
1136 c, out = repoRunnerCtx(t, st, uid, true, testRunnerPub)
1137 if code := runRepoRunnerAdd(c, []string{repo.Path()}); code != protocol.ExitOK {
1138 t.Fatalf("admin could not attach another account's runner key: exit %d %s", code, out.String())
1139 }
1140}
1141```
1142
1143- [ ] **Step 2: Run them to see them fail**
1144
1145Run: `go test ./internal/control -run TestRepoRunner 2>&1 | head -5`
1146Expected: `undefined: runRepoRunnerAdd`.
1147
1148- [ ] **Step 3: Write `internal/control/runnerrepo.go`**
1149
1150```go
1151package control
1152
1153import (
1154 "errors"
1155 "fmt"
1156 "io"
1157
1158 "golang.org/x/crypto/ssh"
1159
1160 "gitbay.org/gitbay/internal/policy"
1161 "gitbay.org/gitbay/internal/protocol"
1162 "gitbay.org/gitbay/internal/store"
1163)
1164
1165// Runners attached to a repository (#184). A runner key claims builds only
1166// for the repositories it is attached to; a repository admin attaches it
1167// by pasting the runner's public key. The key lands on the admin's own
1168// account with scope runner, which confines it to the runner protocol and
1169// read-only git.
1170func init() {
1171 register(Command{Path: []string{"repo", "runner", "add"},
1172 Summary: "attach a runner's public key to a repository",
1173 Usage: "repo runner add <owner/name> < key.pub",
1174 ReadsStdin: true, Run: runRepoRunnerAdd})
1175 register(Command{Path: []string{"repo", "runner", "list"},
1176 Summary: "list the runners attached to a repository",
1177 Usage: "repo runner list <owner/name>", ReadOnly: true, Run: runRepoRunnerList})
1178 register(Command{Path: []string{"repo", "runner", "remove"},
1179 Summary: "detach a runner from a repository",
1180 Usage: "repo runner remove <owner/name> <fingerprint>", Run: runRepoRunnerRemove})
1181}
1182
1183func runRepoRunnerAdd(c *Ctx, args []string) int {
1184 f, err := parseFlags(args, flagSpec{MaxPos: 1, Usage: "repo runner add <owner/name> < key.pub"})
1185 if err != nil || len(f.Pos) != 1 {
1186 return c.fail(protocol.ExitUsage, "usage: repo runner add <owner/name> < key.pub")
1187 }
1188 repo, code := resolveRepo(c, f.Pos[0], policy.CanAdmin)
1189 if code >= 0 {
1190 return code
1191 }
1192 raw, err := io.ReadAll(io.LimitReader(c.Stdin, 64<<10))
1193 if err != nil {
1194 return c.fail(protocol.ExitFailure, "reading key: %v", err)
1195 }
1196 pub, _, _, _, err := ssh.ParseAuthorizedKey(raw)
1197 if err != nil {
1198 return c.fail(protocol.ExitUsage, "not a valid public key in authorized_keys format: %v", err)
1199 }
1200 fp := ssh.FingerprintSHA256(pub)
1201 key, err := c.Store.SSHKeyByFingerprint(fp)
1202 switch {
1203 case errors.Is(err, store.ErrNotFound):
1204 if err := c.Store.AddSSHKey(c.User.ID, fp, pub.Type(), pub.Marshal(), "runner"); err != nil {
1205 return c.fail(protocol.ExitFailure, "adding key: %v", err)
1206 }
1207 if key, err = c.Store.SSHKeyByFingerprint(fp); err != nil {
1208 return c.fail(protocol.ExitFailure, "%v", err)
1209 }
1210 case err != nil:
1211 return c.fail(protocol.ExitFailure, "%v", err)
1212 case key.Scope != "runner":
1213 // A full key would let a build step administer the account; a
1214 // deploy key is bound elsewhere. A runner gets a key of its own.
1215 return c.fail(protocol.ExitDenied, "%s is a %s key, not a runner key; give the runner a key of its own", fp, key.Scope)
1216 case key.UserID != c.User.ID && !c.User.IsAdmin:
1217 return c.fail(protocol.ExitDenied, "%s belongs to another account", fp)
1218 }
1219 if err := c.Store.AttachRunner(key.ID, repo.ID); err != nil {
1220 return c.fail(protocol.ExitFailure, "%v", err)
1221 }
1222 c.Store.Audit(c.User.ID, "repo.runner.add", map[string]any{"repo": repo.Path(), "fingerprint": fp})
1223 d := map[string]string{"fingerprint": fp, "repo": repo.Path()}
1224 return c.emit(d, func(w io.Writer) {
1225 fmt.Fprintf(w, "runner %s attached to %s\n", fp, repo.Path())
1226 })
1227}
1228
1229func runRepoRunnerList(c *Ctx, args []string) int {
1230 if len(args) != 1 {
1231 return c.fail(protocol.ExitUsage, "usage: repo runner list <owner/name>")
1232 }
1233 repo, code := resolveRepo(c, args[0], policy.CanAdmin)
1234 if code >= 0 {
1235 return code
1236 }
1237 runners, err := c.Store.ListRepoRunners(repo.ID)
1238 if err != nil {
1239 return c.fail(protocol.ExitFailure, "%v", err)
1240 }
1241 if runners == nil {
1242 runners = []store.RepoRunner{}
1243 }
1244 return c.emit(runners, func(w io.Writer) {
1245 for _, r := range runners {
1246 seen := r.LastSeen
1247 if seen == "" {
1248 seen = "never"
1249 }
1250 held := "idle"
1251 if r.BuildNumber != 0 {
1252 held = fmt.Sprintf("%s #%d %s since %s", r.BuildRepo, r.BuildNumber, r.BuildJob, r.StartedAt)
1253 }
1254 fmt.Fprintf(w, "%s\t%s\t%s\t%s\t%s\n", r.Fingerprint, r.Algo, r.Username, seen, held)
1255 }
1256 })
1257}
1258
1259func runRepoRunnerRemove(c *Ctx, args []string) int {
1260 if len(args) != 2 {
1261 return c.fail(protocol.ExitUsage, "usage: repo runner remove <owner/name> <fingerprint>")
1262 }
1263 repo, code := resolveRepo(c, args[0], policy.CanAdmin)
1264 if code >= 0 {
1265 return code
1266 }
1267 if err := c.Store.DetachRunner(repo.ID, args[1]); err != nil {
1268 if errors.Is(err, store.ErrNotFound) {
1269 return c.fail(protocol.ExitNotFound, "no runner %s on %s", args[1], repo.Path())
1270 }
1271 return c.fail(protocol.ExitFailure, "%v", err)
1272 }
1273 c.Store.Audit(c.User.ID, "repo.runner.remove", map[string]any{"repo": repo.Path(), "fingerprint": args[1]})
1274 return c.emit(map[string]string{"removed": args[1]}, func(w io.Writer) {
1275 fmt.Fprintf(w, "runner %s detached from %s\n", args[1], repo.Path())
1276 })
1277}
1278```
1279
1280`Store.Audit(actorID int64, action string, data map[string]any)` returns nothing.
1281
1282- [ ] **Step 4: Add the CLI table entries**
1283
1284In `cmd/gitbay/main.go`, directly after the `group("deploy-key", ...)` block (line 437-440):
1285
1286```go
1287 group("runner", "runners attached to a repository",
1288 pass("add", "attach a runner's public key: < key.pub", passOpts{server: []string{"repo", "runner", "add"}, needsRepo: true, alwaysStdin: true, stdinWhat: "an SSH public key"}),
1289 pass("list", "list attached runners", passOpts{server: []string{"repo", "runner", "list"}, needsRepo: true}),
1290 pass("remove", "detach a runner: <fingerprint>", passOpts{server: []string{"repo", "runner", "remove"}, needsRepo: true}),
1291 ),
1292```
1293
1294- [ ] **Step 5: Run the tests**
1295
1296Run: `go build ./... && go test ./internal/control ./cmd/gitbay 2>&1 | tail -5`
1297Expected: PASS, including the CLI coverage test.
1298
1299- [ ] **Step 6: Commit**
1300
1301```bash
1302git add internal/control/runnerrepo.go internal/control/runnerrepo_test.go cmd/gitbay/main.go
1303git commit -m "control, cli: repo runner add, list, remove
1304
1305Ref #184"
1306```
1307
1308---
1309
1310### Task 5: Web: Runners on the repository settings page
1311
1312**Files:**
1313- Modify: `internal/httpd/settings.go:18-49` (`settingsPage`, `settingsForm`) and the `switch` in `settingsSubmit`
1314- Modify: `internal/web/templates/settings.html` (a section after Dependencies, before Lifecycle)
1315- Create: `e2e/runnerweb_test.go`
1316
1317**Interfaces:**
1318- Consumes: `repo runner list|add|remove` from Task 4; `s.runControlInto`, `s.runControlStdin(u, argv, stdin) (msg string, ok bool)`, `s.runControl`.
1319- Produces: form fields `field=runner-add` with `key`, and `field=runner-remove` with `fingerprint`.
1320
1321- [ ] **Step 1: Write the failing e2e test**
1322
1323`e2e/runnerweb_test.go`:
1324
1325```go
1326package e2e
1327
1328import (
1329 "net/url"
1330 "os"
1331 "strings"
1332 "testing"
1333)
1334
1335// The settings page attaches and detaches runners through the same
1336// commands the CLI uses, and lists what is attached.
1337func TestRunnerSettingsWeb(t *testing.T) {
1338 inst := startInstanceWith(t, "[web]\nmode = \"accounts\"\n")
1339 aliceKey := inst.newKey(t, "alice")
1340 inst.admin(t, "admin", "user", "create", "alice",
1341 "--key", aliceKey+".pub", "--email", "alice@example.test", "--verified")
1342 if _, errOut, code := inst.ssh(t, aliceKey, "", "repo", "create", "alice/app"); code != 0 {
1343 t.Fatalf("repo create: %s", errOut)
1344 }
1345 runnerKey := inst.newKey(t, "laptop")
1346 pub, _ := os.ReadFile(runnerKey + ".pub")
1347
1348 alice := inst.login(t, aliceKey)
1349 settings := inst.base() + "/alice/app/settings"
1350 _, body := browserGet(t, alice, settings)
1351 if !strings.Contains(body, "No runners attached") {
1352 t.Fatalf("empty state missing:\n%s", body)
1353 }
1354 if status, _ := browserPost(t, alice, settings, url.Values{"field": {"runner-add"}, "key": {string(pub)}}); status != 200 {
1355 t.Fatalf("runner-add post: %d", status)
1356 }
1357 out, _, _ := inst.ssh(t, aliceKey, "", "repo", "runner", "list", "alice/app", "--json")
1358 if !strings.Contains(out, `"fingerprint":"SHA256:`) {
1359 t.Fatalf("not attached after the form: %s", out)
1360 }
1361 fp := out[strings.Index(out, "SHA256:"):]
1362 fp = fp[:strings.Index(fp, `"`)]
1363 _, body = browserGet(t, alice, settings)
1364 if !strings.Contains(body, fp) || !strings.Contains(body, `value="runner-remove"`) {
1365 t.Fatalf("attached runner not listed:\n%s", body)
1366 }
1367 if status, _ := browserPost(t, alice, settings, url.Values{"field": {"runner-remove"}, "fingerprint": {fp}}); status != 200 {
1368 t.Fatalf("runner-remove post: %d", status)
1369 }
1370 if out, _, _ = inst.ssh(t, aliceKey, "", "repo", "runner", "list", "alice/app", "--json"); strings.Contains(out, fp) {
1371 t.Fatalf("still attached after remove: %s", out)
1372 }
1373}
1374```
1375
1376- [ ] **Step 2: Run it to see it fail**
1377
1378Run: `go test ./e2e -run TestRunnerSettingsWeb -count=1 2>&1 | tail -5`
1379Expected: FAIL at "empty state missing".
1380
1381- [ ] **Step 3: Handler changes in `internal/httpd/settings.go`**
1382
1383`settingsPage` gains:
1384
1385```go
1386 Runners []store.RepoRunner
1387```
1388
1389In `settingsForm`, after the deps read:
1390
1391```go
1392 var runners []store.RepoRunner
1393 s.runControlInto(u, []string{"repo", "runner", "list", repo.Path()}, &runners)
1394```
1395
1396and pass `Runners: runners` to the struct literal.
1397
1398In `settingsSubmit`'s switch, before `default:`:
1399
1400```go
1401 case "runner-add":
1402 body := v("key")
1403 if body == "" {
1404 s.settingsRedirect(w, r, "paste the runner's public key")
1405 return
1406 }
1407 msg, ok := s.runControlStdin(u, []string{"repo", "runner", "add", repo}, body+"\n")
1408 if ok {
1409 msg = ""
1410 }
1411 s.settingsRedirect(w, r, msg)
1412 return
1413 case "runner-remove":
1414 argv = []string{"repo", "runner", "remove", repo, v("fingerprint")}
1415```
1416
1417- [ ] **Step 4: Template section**
1418
1419In `internal/web/templates/settings.html`, before `<h2>Lifecycle</h2>`:
1420
1421```html
1422<h2>Runners</h2>
1423{{if .Runners}}
1424<ul class="protlist">
1425{{range .Runners}}<li><code>{{.Fingerprint}}</code> <span class="meta">{{.Username}}{{if .LastSeen}}, last poll {{.LastSeen}}{{else}}, never polled{{end}}{{if .BuildNumber}}, running {{.BuildRepo}} #{{.BuildNumber}} {{.BuildJob}}{{end}}</span>
1426 <form method="post" action="{{$base}}" class="inline">
1427 <input type="hidden" name="field" value="runner-remove">
1428 <input type="hidden" name="fingerprint" value="{{.Fingerprint}}">
1429 <button type="submit" class="linklike">Detach</button>
1430 </form></li>
1431{{end}}
1432</ul>
1433{{else}}<p class="meta">No runners attached. Builds for this repository run on the runners attached here; a repository with none queues builds nothing claims.</p>{{end}}
1434<form method="post" action="{{$base}}" class="setform">
1435 <input type="hidden" name="field" value="runner-add">
1436 <label for="runner-key">Attach a runner</label>
1437 <textarea id="runner-key" name="key" rows="3" placeholder="ssh-ed25519 AAAA… (from gitbay-runner init)"></textarea>
1438 <button type="submit">Attach</button>
1439</form>
1440<p class="meta">Install <code>gitbay-runner</code>, run <code>gitbay-runner init</code>, and paste the key it prints. The runner builds your commits with the repository's secrets; merge requests from forks wait unless it runs with <code>-untrusted</code>.</p>
1441```
1442
1443- [ ] **Step 5: Run the test**
1444
1445Run: `go test ./e2e -run TestRunnerSettingsWeb -count=1 2>&1 | tail -5`
1446Expected: PASS.
1447
1448- [ ] **Step 6: Commit**
1449
1450```bash
1451git add internal/httpd/settings.go internal/web/templates/settings.html e2e/runnerweb_test.go
1452git commit -m "httpd: attach and detach runners on the settings page
1453
1454Ref #184"
1455```
1456
1457---
1458
1459### Task 6: Runner: config file, `-identity`, `-untrusted`
1460
1461**Files:**
1462- Create: `cmd/gitbay-runner/config.go`
1463- Create: `cmd/gitbay-runner/config_test.go`
1464- Modify: `cmd/gitbay-runner/main.go` (flag block, `runner` struct, `step`, ssh option assembly)
1465
1466**Interfaces:**
1467- Produces:
1468 - `configDir() string`: `$XDG_CONFIG_HOME/gitbay-runner` or `$HOME/.config/gitbay-runner`.
1469 - `defaultConfigPath() string`: `configDir()/config.toml`.
1470 - `configPathFromArgs(args []string, def string) string`: honours `-config X`, `--config X`, `-config=X`.
1471 - `loadConfig(path string) (map[string]string, bool, error)`: flag name to value, false when the file is absent.
1472 - `applyConfig(fs *flag.FlagSet, values map[string]string) error`: `fs.Set` each.
1473 - `identityOpts(path string) []string`: `["-i", path, "-o", "IdentitiesOnly=yes"]` or nil.
1474 - Flags `-config`, `-identity`, `-untrusted`.
1475
1476- [ ] **Step 1: Write the failing tests**
1477
1478`cmd/gitbay-runner/config_test.go`:
1479
1480```go
1481package main
1482
1483import (
1484 "flag"
1485 "os"
1486 "path/filepath"
1487 "testing"
1488)
1489
1490// A config file sets the flags' values; a flag on the command line wins.
1491func TestConfigFileFeedsFlagsAndFlagsOverride(t *testing.T) {
1492 dir := t.TempDir()
1493 path := filepath.Join(dir, "config.toml")
1494 os.WriteFile(path, []byte("remote = \"git@example.test\"\npoll = \"9s\"\nuntrusted = true\nidentity = \"/k\"\njobs = 2\n"), 0o600)
1495
1496 values, found, err := loadConfig(path)
1497 if err != nil || !found {
1498 t.Fatalf("loadConfig: found=%v err=%v", found, err)
1499 }
1500 fs := flag.NewFlagSet("t", flag.ContinueOnError)
1501 remote := fs.String("remote", "git@gitbay.org", "")
1502 poll := fs.Duration("poll", 0, "")
1503 untrusted := fs.Bool("untrusted", false, "")
1504 identity := fs.String("identity", "", "")
1505 jobs := fs.Int("jobs", 1, "")
1506 if err := applyConfig(fs, values); err != nil {
1507 t.Fatal(err)
1508 }
1509 if err := fs.Parse([]string{"-poll", "3s"}); err != nil {
1510 t.Fatal(err)
1511 }
1512 if *remote != "git@example.test" || poll.String() != "3s" || !*untrusted || *identity != "/k" || *jobs != 2 {
1513 t.Fatalf("remote=%s poll=%s untrusted=%v identity=%s jobs=%d", *remote, poll, *untrusted, *identity, *jobs)
1514 }
1515 if _, found, err := loadConfig(filepath.Join(dir, "missing.toml")); found || err != nil {
1516 t.Fatalf("missing file: found=%v err=%v", found, err)
1517 }
1518 if _, _, err := loadConfig(path); err != nil {
1519 t.Fatal(err)
1520 }
1521 os.WriteFile(path, []byte("nonsense = \"x\"\n"), 0o600)
1522 if _, _, err := loadConfig(path); err == nil {
1523 t.Fatal("an unknown key was accepted")
1524 }
1525}
1526
1527func TestConfigPathFromArgs(t *testing.T) {
1528 for _, tc := range []struct {
1529 args []string
1530 want string
1531 }{
1532 {nil, "/def"},
1533 {[]string{"-once"}, "/def"},
1534 {[]string{"-config", "/a"}, "/a"},
1535 {[]string{"--config", "/b", "-once"}, "/b"},
1536 {[]string{"-config=/c"}, "/c"},
1537 } {
1538 if got := configPathFromArgs(tc.args, "/def"); got != tc.want {
1539 t.Errorf("%v: got %s want %s", tc.args, got, tc.want)
1540 }
1541 }
1542}
1543
1544func TestConfigDirHonoursXDG(t *testing.T) {
1545 t.Setenv("XDG_CONFIG_HOME", "/x")
1546 if got := configDir(); got != "/x/gitbay-runner" {
1547 t.Fatalf("got %s", got)
1548 }
1549 t.Setenv("XDG_CONFIG_HOME", "")
1550 t.Setenv("HOME", "/h")
1551 if got := configDir(); got != "/h/.config/gitbay-runner" {
1552 t.Fatalf("got %s", got)
1553 }
1554}
1555
1556func TestIdentityOpts(t *testing.T) {
1557 if got := identityOpts(""); got != nil {
1558 t.Fatalf("empty identity produced %v", got)
1559 }
1560 got := identityOpts("/k")
1561 if len(got) != 4 || got[0] != "-i" || got[1] != "/k" || got[3] != "IdentitiesOnly=yes" {
1562 t.Fatalf("got %v", got)
1563 }
1564}
1565```
1566
1567- [ ] **Step 2: Run them to see them fail**
1568
1569Run: `go test ./cmd/gitbay-runner -run 'TestConfig|TestIdentity' 2>&1 | head -5`
1570Expected: `undefined: loadConfig` and friends.
1571
1572- [ ] **Step 3: Write `cmd/gitbay-runner/config.go`**
1573
1574```go
1575package main
1576
1577import (
1578 "errors"
1579 "flag"
1580 "fmt"
1581 "os"
1582 "path/filepath"
1583 "strings"
1584
1585 "github.com/BurntSushi/toml"
1586)
1587
1588// The runner takes everything as flags, which does not work under a
1589// service manager. config.toml in the config directory carries the same
1590// names; a flag on the command line overrides it (#184).
1591
1592func configDir() string {
1593 if x := os.Getenv("XDG_CONFIG_HOME"); x != "" {
1594 return filepath.Join(x, "gitbay-runner")
1595 }
1596 return filepath.Join(os.Getenv("HOME"), ".config", "gitbay-runner")
1597}
1598
1599func defaultConfigPath() string { return filepath.Join(configDir(), "config.toml") }
1600
1601// configPathFromArgs finds -config before the flag set is parsed, since
1602// the file's values must be set before parsing for flags to override them.
1603func configPathFromArgs(args []string, def string) string {
1604 for i, a := range args {
1605 a = strings.TrimPrefix(a, "-")
1606 if a == "-config" || a == "config" {
1607 if i+1 < len(args) {
1608 return args[i+1]
1609 }
1610 }
1611 if v, ok := strings.CutPrefix(a, "config="); ok {
1612 return v
1613 }
1614 if v, ok := strings.CutPrefix(a, "-config="); ok {
1615 return v
1616 }
1617 }
1618 return def
1619}
1620
1621// configKeys is every key the file may carry: the flag names.
1622var configKeys = map[string]bool{"remote": true, "ssh-opts": true, "clone-base": true, "workdir": true,
1623 "poll": true, "timeout": true, "repos": true, "jobs": true, "image": true, "isolation": true,
1624 "memory": true, "cpus": true, "untrusted": true, "identity": true}
1625
1626// loadConfig reads path into flag name → value. Absent file: found is
1627// false and there is no error. An unknown key is an error, not a typo
1628// the runner silently ignores.
1629func loadConfig(path string) (values map[string]string, found bool, err error) {
1630 var raw map[string]any
1631 if _, err := toml.DecodeFile(path, &raw); errors.Is(err, os.ErrNotExist) {
1632 return nil, false, nil
1633 } else if err != nil {
1634 return nil, true, fmt.Errorf("%s: %w", path, err)
1635 }
1636 values = map[string]string{}
1637 for k, v := range raw {
1638 if !configKeys[k] {
1639 return nil, true, fmt.Errorf("%s: unknown key %s", path, k)
1640 }
1641 values[k] = fmt.Sprint(v)
1642 }
1643 return values, true, nil
1644}
1645
1646// applyConfig sets each value on the flag set, which is what parsing the
1647// command line would do; parse afterwards and the command line wins.
1648func applyConfig(fs *flag.FlagSet, values map[string]string) error {
1649 for k, v := range values {
1650 if fs.Lookup(k) == nil {
1651 return fmt.Errorf("config: unknown key %s", k)
1652 }
1653 if err := fs.Set(k, v); err != nil {
1654 return fmt.Errorf("config: %s: %w", k, err)
1655 }
1656 }
1657 return nil
1658}
1659
1660// identityOpts is what makes ssh and git use the runner's own key and no
1661// other: on a laptop the ambient key is the user's full-scope one, which
1662// the runner protocol refuses.
1663func identityOpts(path string) []string {
1664 if path == "" {
1665 return nil
1666 }
1667 return []string{"-i", path, "-o", "IdentitiesOnly=yes"}
1668}
1669```
1670
1671- [ ] **Step 4: Wire it into `main.go`**
1672
1673In `main()`, replace `flag.Parse()` and the flag block with a flag set fed by the config file. Add three flags and keep the others as they are:
1674
1675```go
1676 var (
1677 configPath = flag.String("config", defaultConfigPath(), "config file; keys are these flag names, flags override it")
1678 identity = flag.String("identity", "", "ssh private key to poll and clone with (default: the key gitbay-runner init generated, if present)")
1679 untrusted = flag.Bool("untrusted", false, "also claim untrusted builds: merge request heads from forks (needs -isolation podman to be safe)")
1680 // ... existing flags unchanged ...
1681 )
1682 path := configPathFromArgs(os.Args[1:], *configPath)
1683 if values, found, err := loadConfig(path); err != nil {
1684 log.Fatal(err)
1685 } else if found {
1686 if err := applyConfig(flag.CommandLine, values); err != nil {
1687 log.Fatal(err)
1688 }
1689 log.Printf("config: %s", path)
1690 }
1691 flag.Parse()
1692```
1693
1694After `if *sshOpts != "" { r.sshOpts = strings.Fields(*sshOpts) }`:
1695
1696```go
1697 if *identity == "" {
1698 if p := filepath.Join(configDir(), "id_ed25519"); fileExists(p) {
1699 *identity = p
1700 }
1701 }
1702 r.sshOpts = append(identityOpts(*identity), r.sshOpts...)
1703 r.untrusted = *untrusted
1704```
1705
1706with
1707
1708```go
1709func fileExists(p string) bool { _, err := os.Stat(p); return err == nil }
1710```
1711
1712`runner` struct gains `untrusted bool`. In `step()`:
1713
1714```go
1715 args := []string{"runner", "next"}
1716 if r.untrusted {
1717 args = append(args, "--untrusted")
1718 }
1719 args = append(append(args, r.repos...), "--json")
1720 out, err := r.ssh(nil, args...)
1721```
1722
1723Both `ssh()` and the `gitSSH` line already use `r.sshOpts`, so the identity reaches both.
1724
1725Move the `init` dispatch hook in now so Task 7 has a place to land, at the top of `main()`:
1726
1727```go
1728 if len(os.Args) > 1 && os.Args[1] == "init" {
1729 os.Exit(runInit(os.Args[2:]))
1730 }
1731```
1732
1733and a stub in `config.go` until Task 7 replaces it:
1734
1735```go
1736func runInit(args []string) int { fmt.Fprintln(os.Stderr, "init: not implemented"); return 2 }
1737```
1738
1739- [ ] **Step 5: Run the tests**
1740
1741Run: `go build ./... && go vet ./cmd/gitbay-runner && go test ./cmd/gitbay-runner 2>&1 | tail -3`
1742Expected: PASS.
1743
1744- [ ] **Step 6: Commit**
1745
1746```bash
1747git add cmd/gitbay-runner/config.go cmd/gitbay-runner/config_test.go cmd/gitbay-runner/main.go
1748git commit -m "runner: config.toml, -identity, -untrusted
1749
1750Ref #184"
1751```
1752
1753---
1754
1755### Task 7: Runner: `gitbay-runner init`
1756
1757**Files:**
1758- Create: `cmd/gitbay-runner/init.go` (replaces the stub `runInit` in `config.go`; delete the stub)
1759- Create: `cmd/gitbay-runner/init_test.go`
1760
1761**Interfaces:**
1762- Consumes: `configDir()`, `defaultWorkdir()`, `toolpath.Look("ssh-keygen")`.
1763- Produces: `runInit(args []string) int`; files `<configDir>/id_ed25519`, `id_ed25519.pub`, `config.toml`.
1764
1765- [ ] **Step 1: Write the failing test**
1766
1767`cmd/gitbay-runner/init_test.go`:
1768
1769```go
1770package main
1771
1772import (
1773 "bytes"
1774 "os"
1775 "os/exec"
1776 "path/filepath"
1777 "strings"
1778 "testing"
1779)
1780
1781// init creates the key and config once, prints the key and the attach
1782// command, and running it again changes nothing.
1783func TestInitWritesKeyAndConfigOnce(t *testing.T) {
1784 if _, err := exec.LookPath("ssh-keygen"); err != nil {
1785 t.Skip("ssh-keygen not on PATH")
1786 }
1787 dir := t.TempDir()
1788 t.Setenv("XDG_CONFIG_HOME", dir)
1789 var out bytes.Buffer
1790 initOut = &out
1791 defer func() { initOut = os.Stdout }()
1792
1793 if code := runInit([]string{"-remote", "git@example.test"}); code != 0 {
1794 t.Fatalf("init: exit %d\n%s", code, out.String())
1795 }
1796 cdir := filepath.Join(dir, "gitbay-runner")
1797 key := filepath.Join(cdir, "id_ed25519")
1798 pub, err := os.ReadFile(key + ".pub")
1799 if err != nil || !strings.HasPrefix(string(pub), "ssh-ed25519 ") {
1800 t.Fatalf("public key: %v %q", err, pub)
1801 }
1802 if fi, _ := os.Stat(key); fi.Mode().Perm() != 0o600 {
1803 t.Fatalf("private key mode %o", fi.Mode().Perm())
1804 }
1805 if fi, _ := os.Stat(cdir); fi.Mode().Perm() != 0o700 {
1806 t.Fatalf("config dir mode %o", fi.Mode().Perm())
1807 }
1808 cfg, _ := os.ReadFile(filepath.Join(cdir, "config.toml"))
1809 for _, want := range []string{"remote = \"git@example.test\"", "isolation = \"none\"", "untrusted = false", "identity = \"" + key + "\""} {
1810 if !strings.Contains(string(cfg), want) {
1811 t.Fatalf("config lacks %q:\n%s", want, cfg)
1812 }
1813 }
1814 for _, want := range []string{strings.TrimSpace(string(pub)), "gitbay repo runner add owner/name < " + key + ".pub", "https://example.test/owner/name/settings"} {
1815 if !strings.Contains(out.String(), want) {
1816 t.Fatalf("output lacks %q:\n%s", want, out.String())
1817 }
1818 }
1819
1820 out.Reset()
1821 if code := runInit([]string{"-remote", "git@other.test"}); code != 0 {
1822 t.Fatalf("second init: exit %d\n%s", code, out.String())
1823 }
1824 if pub2, _ := os.ReadFile(key + ".pub"); string(pub2) != string(pub) {
1825 t.Fatal("second init replaced the key")
1826 }
1827 if cfg2, _ := os.ReadFile(filepath.Join(cdir, "config.toml")); string(cfg2) != string(cfg) {
1828 t.Fatal("second init rewrote the config")
1829 }
1830}
1831
1832// podman needs an image; init refuses to write a config the runner would
1833// refuse to start with.
1834func TestInitPodmanNeedsImage(t *testing.T) {
1835 t.Setenv("XDG_CONFIG_HOME", t.TempDir())
1836 var out bytes.Buffer
1837 initOut = &out
1838 defer func() { initOut = os.Stdout }()
1839 if code := runInit([]string{"-isolation", "podman"}); code != 2 {
1840 t.Fatalf("exit %d, want 2:\n%s", code, out.String())
1841 }
1842}
1843```
1844
1845- [ ] **Step 2: Run it to see it fail**
1846
1847Run: `go test ./cmd/gitbay-runner -run TestInit 2>&1 | head -5`
1848Expected: `undefined: initOut`.
1849
1850- [ ] **Step 3: Write `cmd/gitbay-runner/init.go`**
1851
1852```go
1853package main
1854
1855import (
1856 "flag"
1857 "fmt"
1858 "io"
1859 "os"
1860 "os/exec"
1861 "path/filepath"
1862 "strings"
1863
1864 "gitbay.org/gitbay/internal/toolpath"
1865)
1866
1867// initOut is where init prints; tests capture it.
1868var initOut io.Writer = os.Stdout
1869
1870// runInit makes a fresh install ready to attach: a key of its own, a
1871// config file the service reads, and the one command to run next. It never
1872// overwrites a key or a config that exists, so running it twice is safe.
1873func runInit(args []string) int {
1874 fs := flag.NewFlagSet("init", flag.ContinueOnError)
1875 fs.SetOutput(initOut)
1876 remote := fs.String("remote", "git@gitbay.org", "ssh destination of the gitbay server")
1877 workdir := fs.String("workdir", defaultWorkdir(), "build workspace root")
1878 isolation := fs.String("isolation", isolationNone, "how steps run: none, or podman with -image")
1879 image := fs.String("image", "", "container image for -isolation podman")
1880 if err := fs.Parse(args); err != nil {
1881 return 2
1882 }
1883 if *isolation == isolationPodman && *image == "" {
1884 fmt.Fprintln(initOut, "-isolation podman needs -image <ref>: the runner refuses to start without one, and there is no image to guess")
1885 return 2
1886 }
1887 if *isolation != isolationPodman && *isolation != isolationNone {
1888 fmt.Fprintf(initOut, "unknown isolation %q\n", *isolation)
1889 return 2
1890 }
1891
1892 dir := configDir()
1893 if err := os.MkdirAll(dir, 0o700); err != nil {
1894 fmt.Fprintln(initOut, err)
1895 return 1
1896 }
1897 os.Chmod(dir, 0o700)
1898 key := filepath.Join(dir, "id_ed25519")
1899 if !fileExists(key) {
1900 cmd := exec.Command(toolpath.Look("ssh-keygen"), "-q", "-t", "ed25519", "-N", "", "-C", "gitbay-runner", "-f", key)
1901 if out, err := cmd.CombinedOutput(); err != nil {
1902 fmt.Fprintf(initOut, "ssh-keygen: %v\n%s", err, out)
1903 return 1
1904 }
1905 }
1906 os.Chmod(key, 0o600)
1907
1908 cfgPath := filepath.Join(dir, "config.toml")
1909 if !fileExists(cfgPath) {
1910 var b strings.Builder
1911 fmt.Fprintf(&b, "remote = %q\n", *remote)
1912 fmt.Fprintf(&b, "workdir = %q\n", *workdir)
1913 fmt.Fprintf(&b, "isolation = %q\n", *isolation)
1914 if *image != "" {
1915 fmt.Fprintf(&b, "image = %q\n", *image)
1916 }
1917 fmt.Fprintf(&b, "untrusted = false\n")
1918 fmt.Fprintf(&b, "identity = %q\n", key)
1919 if err := os.WriteFile(cfgPath, []byte(b.String()), 0o600); err != nil {
1920 fmt.Fprintln(initOut, err)
1921 return 1
1922 }
1923 }
1924
1925 pub, err := os.ReadFile(key + ".pub")
1926 if err != nil {
1927 fmt.Fprintln(initOut, err)
1928 return 1
1929 }
1930 host := *remote
1931 if i := strings.LastIndex(host, "@"); i >= 0 {
1932 host = host[i+1:]
1933 }
1934 fmt.Fprintf(initOut, "config: %s\nkey: %s\n\n", cfgPath, key)
1935 if *isolation == isolationNone {
1936 fmt.Fprintln(initOut, "Steps run on this machine as your user, with no container. Untrusted builds\n(merge requests from forks) are excluded unless the runner is started with\n-untrusted, so that means your own commits.\n")
1937 }
1938 fmt.Fprintf(initOut, "This runner's public key:\n\n %s\nAttach it to each repository it should build, as a repository admin:\n\n gitbay repo runner add owner/name < %s.pub\n\nor paste it under Runners at https://%s/owner/name/settings\n\nThen start it:\n\n brew services start krz/tap/gitbay-runner\n\nor run gitbay-runner with no arguments.\n",
1939 strings.TrimSpace(string(pub)), key, host)
1940 return 0
1941}
1942```
1943
1944`isolationNone` and `isolationPodman` are the constants in `isolate.go:20-21`. Delete the stub `runInit` from `config.go`.
1945
1946- [ ] **Step 4: Run the tests**
1947
1948Run: `go build ./... && go vet ./cmd/gitbay-runner && go test ./cmd/gitbay-runner 2>&1 | tail -3`
1949Expected: PASS.
1950
1951- [ ] **Step 5: Commit**
1952
1953```bash
1954git add cmd/gitbay-runner/init.go cmd/gitbay-runner/init_test.go cmd/gitbay-runner/config.go
1955git commit -m "runner: init generates the key and config and prints the attach step
1956
1957Ref #184"
1958```
1959
1960---
1961
1962### Task 8: e2e: init, attach, build; fork head waits
1963
1964**Files:**
1965- Create: `e2e/runnerattach_test.go`
1966
1967**Interfaces:**
1968- Consumes: `buildRunner(t)`, `inst.newKey`, `inst.admin`, `inst.ssh(t, key, stdin, argv...)`, `inst.gitEnv`, `inst.sshURL`, `mustGit`, `inst.port`, `inst.sshDir` (all in `e2e/ci_test.go` and the instance helpers).
1969
1970- [ ] **Step 1: Write the test**
1971
1972`e2e/runnerattach_test.go`:
1973
1974```go
1975package e2e
1976
1977import (
1978 "fmt"
1979 "os"
1980 "os/exec"
1981 "path/filepath"
1982 "strings"
1983 "testing"
1984)
1985
1986// The whole flow a user goes through: init on their machine, attach the
1987// printed key to their repository, start the runner from the config init
1988// wrote. The runner builds their push and leaves a fork's merge request
1989// head alone until started with -untrusted.
1990func TestAttachedRunnerBuildsOwnRepo(t *testing.T) {
1991 inst := startInstance(t)
1992 inst.runner = buildRunner(t)
1993 aliceKey := inst.newKey(t, "alice")
1994 inst.admin(t, "admin", "user", "create", "alice", "--key", aliceKey+".pub")
1995 bobKey := inst.newKey(t, "bob")
1996 inst.admin(t, "admin", "user", "create", "bob", "--key", bobKey+".pub")
1997 if _, errOut, code := inst.ssh(t, aliceKey, "", "repo", "create", "alice/app"); code != 0 {
1998 t.Fatalf("repo create: %s", errOut)
1999 }
2000
2001 // init on "alice's laptop".
2002 xdg := t.TempDir()
2003 initCmd := exec.Command(inst.runner, "init", "-remote", "git@127.0.0.1")
2004 initCmd.Env = append(os.Environ(), "XDG_CONFIG_HOME="+xdg)
2005 initOut, err := initCmd.CombinedOutput()
2006 if err != nil {
2007 t.Fatalf("init: %v\n%s", err, initOut)
2008 }
2009 cdir := filepath.Join(xdg, "gitbay-runner")
2010 pub, err := os.ReadFile(filepath.Join(cdir, "id_ed25519.pub"))
2011 if err != nil {
2012 t.Fatal(err)
2013 }
2014 if !strings.Contains(string(initOut), strings.TrimSpace(string(pub))) {
2015 t.Fatalf("init did not print the key:\n%s", initOut)
2016 }
2017
2018 // The unattached key claims nothing, even with a build queued.
2019 work := t.TempDir()
2020 env := inst.gitEnv(aliceKey)
2021 mustGit(t, work, env, "clone", inst.sshURL("alice/app"), "w")
2022 dir := filepath.Join(work, "w")
2023 os.MkdirAll(filepath.Join(dir, ".gitbay"), 0o755)
2024 os.WriteFile(filepath.Join(dir, ".gitbay", "ci.yml"), []byte("jobs:\n unit:\n steps:\n - echo built\n"), 0o644)
2025 mustGit(t, dir, env, "checkout", "-q", "-b", "main")
2026 mustGit(t, dir, env, "add", ".")
2027 mustGit(t, dir, env, "commit", "-q", "-m", "ci")
2028 mustGit(t, dir, env, "push", "-q", "origin", "main")
2029
2030 run := func(extra ...string) string {
2031 t.Helper()
2032 // No -i in ssh-opts: the identity from the config is what
2033 // authenticates, which is the point.
2034 opts := fmt.Sprintf("-p %d -o StrictHostKeyChecking=no -o UserKnownHostsFile=%s -o BatchMode=yes",
2035 inst.port, filepath.Join(inst.sshDir, "known_hosts"))
2036 args := append([]string{"-config", filepath.Join(cdir, "config.toml"), "-once",
2037 "-ssh-opts", opts,
2038 "-clone-base", fmt.Sprintf("ssh://git@127.0.0.1:%d", inst.port),
2039 "-workdir", t.TempDir()}, extra...)
2040 cmd := exec.Command(inst.runner, args...)
2041 cmd.Env = append(os.Environ(), "XDG_CONFIG_HOME="+xdg, "GIT_CONFIG_NOSYSTEM=1", "GIT_CONFIG_GLOBAL=/dev/null")
2042 out, err := cmd.CombinedOutput()
2043 if err != nil {
2044 t.Fatalf("runner: %v\n%s", err, out)
2045 }
2046 return string(out)
2047 }
2048 if out, _, _ := inst.ssh(t, aliceKey, "", "build", "list", "alice/app"); !strings.Contains(out, "unit\tpending") {
2049 t.Fatalf("build not pending before attach: %s", out)
2050 }
2051
2052 // Attach with the printed key.
2053 if _, errOut, code := inst.ssh(t, aliceKey, string(pub), "repo", "runner", "add", "alice/app"); code != 0 {
2054 t.Fatalf("repo runner add: %s", errOut)
2055 }
2056 out, _, _ := inst.ssh(t, aliceKey, "", "repo", "runner", "list", "alice/app", "--json")
2057 if !strings.Contains(out, `"username":"alice"`) || strings.Contains(out, `"last_seen":"20`) {
2058 t.Fatalf("list after attach: %s", out)
2059 }
2060
2061 // The runner builds it.
2062 run()
2063 if out, _, _ = inst.ssh(t, aliceKey, "", "build", "list", "alice/app"); !strings.Contains(out, "unit\tsuccess") {
2064 t.Fatalf("build not built by the attached runner: %s", out)
2065 }
2066 if out, _, _ = inst.ssh(t, aliceKey, "", "repo", "runner", "list", "alice/app", "--json"); !strings.Contains(out, `"last_seen":"20`) {
2067 t.Fatalf("no heartbeat after a poll: %s", out)
2068 }
2069
2070 // bob forks and opens a merge request: an untrusted build in the target.
2071 if _, errOut, code := inst.ssh(t, bobKey, "", "repo", "fork", "alice/app"); code != 0 {
2072 t.Fatalf("fork: %s", errOut)
2073 }
2074 bwork := t.TempDir()
2075 benv := inst.gitEnv(bobKey)
2076 mustGit(t, bwork, benv, "clone", inst.sshURL("bob/app"), "w")
2077 bdir := filepath.Join(bwork, "w")
2078 mustGit(t, bdir, benv, "checkout", "-q", "-b", "feat")
2079 os.WriteFile(filepath.Join(bdir, "f.txt"), []byte("y\n"), 0o644)
2080 mustGit(t, bdir, benv, "add", ".")
2081 mustGit(t, bdir, benv, "commit", "-q", "-m", "change")
2082 mustGit(t, bdir, benv, "push", "-q", "origin", "feat")
2083 if _, _, code := inst.ssh(t, bobKey, "", "build", "cancel", "bob/app", "1"); code != 0 {
2084 t.Fatal("cancel bob's own build")
2085 }
2086 if _, errOut, code := inst.ssh(t, bobKey, "", "mr", "create", "alice/app",
2087 "--source", "bob/app:feat", "--target", "main", "--title", "change"); code != 0 {
2088 t.Fatalf("mr create: %s", errOut)
2089 }
2090 run()
2091 if out, _, _ = inst.ssh(t, aliceKey, "", "build", "list", "alice/app"); strings.Count(out, "pending") != 1 {
2092 t.Fatalf("fork head was claimed without -untrusted:\n%s", out)
2093 }
2094 run("-untrusted")
2095 if out, _, _ = inst.ssh(t, aliceKey, "", "build", "list", "alice/app"); strings.Contains(out, "pending") {
2096 t.Fatalf("fork head not built with -untrusted:\n%s", out)
2097 }
2098}
2099```
2100
2101- [ ] **Step 2: Run it**
2102
2103Run: `go test ./e2e -run TestAttachedRunnerBuildsOwnRepo -count=1 -v 2>&1 | tail -20`
2104Expected: PASS. If the fork's build numbering differs (two jobs, or the fork's push queues more than one build), adjust the `build cancel bob/app` loop to cancel every pending build listed by `build list bob/app --json`.
2105
2106- [ ] **Step 3: Commit**
2107
2108```bash
2109git add e2e/runnerattach_test.go
2110git commit -m "e2e: init, attach, and an attached runner building its repository
2111
2112Ref #184"
2113```
2114
2115---
2116
2117### Task 9: Docs: wiki pages
2118
2119**Files:**
2120- Modify: `.gitbay/wiki/Users.org` (after the "CI builds" section's last paragraph, before "* Large files (LFS)")
2121- Modify: `.gitbay/wiki/Admin.org:329-340` and `:395-402`
2122- Modify: `.gitbay/wiki/Threat-Model.org:120-126`
2123- Modify: `.gitbay/wiki/Parity.org` (repo table, after the `webhooks` row; and the "SSH only, by design" paragraph is unchanged)
2124- Modify: `.gitbay/wiki/FAQ.org:20-25`
2125- Modify: `.gitbay/wiki/CI.org` (one sentence after the three mechanisms list)
2126
2127- [ ] **Step 1: Users: "Your own runner"**
2128
2129Insert before `* Large files (LFS)`:
2130
2131```org
2132** Your own runner
2133
2134Builds run on runners attached to the repository. An instance need not
2135offer any: install =gitbay-runner= on a machine of yours and attach it.
2136
2137#+begin_src sh
2138brew install krz/tap/gitbay-runner # or a binary from the release
2139gitbay-runner init -remote git@gitbay.org
2140#+end_src
2141
2142=init= generates a key under =~/.config/gitbay-runner/=, writes
2143=config.toml= beside it, and prints the public key with the command to
2144attach it:
2145
2146#+begin_src sh
2147gitbay repo runner add owner/name < ~/.config/gitbay-runner/id_ed25519.pub
2148#+end_src
2149
2150or paste the key under Runners on the repository's settings page. Then
2151=brew services start krz/tap/gitbay-runner=, or run =gitbay-runner= with
2152no arguments; it reads the config file, and any flag overrides it.
2153
2154What it builds: every build for the repositories it is attached to,
2155with the repository's secrets, and nothing else. Merge requests from
2156forks are untrusted and wait unless the runner runs with =-untrusted=,
2157which is only sensible with =-isolation podman -image <ref>= (see
2158[[Admin][Admin]]). Attach one runner to several repositories by repeating
2159=repo runner add=; run several runners on one account by running =init=
2160on each machine. =repo runner list= shows each attached key, when it
2161last polled and the build it holds; =repo runner remove <fingerprint>=
2162detaches one (the key stays on your account; =keys remove= drops it).
2163A runner key reaches only the runner protocol and read-only git, so a
2164build step that reads it off disk cannot administer your account.
2165```
2166
2167- [ ] **Step 2: Admin**
2168
2169Replace lines 329-340's opening paragraph ("=gitbay-runner= executes builds ... then removed:") with:
2170
2171```org
2172=gitbay-runner= executes builds queued by pushes and merge requests. It
2173polls over SSH with a key of scope =runner=, which reaches only the
2174runner protocol and read-only git (a runner executes arbitrary
2175repository code, so the key it holds must not do more). A runner key
2176claims builds only for the repositories it is attached to, by =repo
2177runner add= from a repository admin or an instance admin; an admin key
2178claims any. Users attach their own runners: see the Users page. For an
2179instance runner, run it as a dedicated unprivileged user on a non-admin
2180account. =admin user create --key= registers a full-scope key, so the
2181runner key is added afterwards through a bootstrap key that is then
2182removed, and attached to each repository it should build:
2183```
2184
2185After the existing bootstrap code block, add:
2186
2187```org
2188#+begin_src sh
2189gitbay repo runner add krz/site < /var/lib/gitbay-runner/.ssh/id_ed25519.pub
2190#+end_src
2191```
2192
2193Replace lines 395-402 (from "and the isolation canary, nothing else" to "the boundary is you choosing how to start it.") with:
2194
2195```org
2196and the isolation canary, nothing else, because it shares the host with
2197the forge; any other repository builds on a runner its owner attaches.
2198
2199=-repos= narrows an admin runner; for a runner key the attachments are
2200the boundary, held by the server, and =-repos= may only name
2201repositories among them. =-untrusted= makes a runner claim merge
2202request heads from forks; the bay1 unit sets it because it isolates in
2203podman. A runner without it builds trusted commits only.
2204```
2205
2206Add `-untrusted` to the `gitbay-runner -remote git@gitbay.org -repos krz/site,krz/docs` example only if that runner isolates; leave the example as is and note under it: "Add =-untrusted= only with =-isolation podman=."
2207
2208- [ ] **Step 3: Threat-Model**
2209
2210Replace the "What the runner holds" bullet (lines 120-126) with:
2211
2212```org
2213- *What the runner holds.* A key of scope =runner=, which the dispatcher
2214 confines to =runner next=, =runner log= and =runner done= and to
2215 read-only git, and which claims, logs and finishes builds only for
2216 the repositories it is attached to (=repo runner add=). A step that
2217 reads the key off the disk gets exactly that: it cannot administer
2218 the instance, push, read a repository the runner's account cannot, or
2219 touch another repository's builds. An admin key still works for the
2220 runner protocol so an operator can rotate at their own pace; a runner
2221 host should not hold one. Untrusted builds are skipped unless the
2222 runner asks with =-untrusted=, so a runner on a user's machine never
2223 executes a stranger's branch by default.
2224```
2225
2226- [ ] **Step 4: Parity, FAQ, CI**
2227
2228Parity, repo table, after the `webhooks` row:
2229
2230```org
2231| runners attach, list, detach | yes | yes | no |
2232```
2233
2234The columns are cli, web, ios. iOS is `no`: outstanding, not intended.
2235
2236FAQ, replace the "Does CI run for my repository on gitbay.org?" answer:
2237
2238```org
2239- Does CI run for my repository on gitbay.org? :: On a runner you
2240 attach. The instance's own runner builds the forge's repositories and
2241 its isolation canary, since it shares the host with the forge.
2242 Install =gitbay-runner= on a machine of yours, run =gitbay-runner
2243 init=, and attach the key it prints with =repo runner add= or on the
2244 repository's settings page; see the Users page. A self-hosted
2245 instance can do the same, or run one runner for whichever
2246 repositories its operator attaches it to.
2247```
2248
2249CI.org, after the three-mechanism list:
2250
2251```org
2252Which runner takes a build is the fourth: a build is claimed only by a
2253runner attached to its repository (or an instance admin's runner), and
2254an untrusted build only by one started with =-untrusted=. A repository
2255with no runner attached queues builds nothing claims. See the Users
2256page.
2257```
2258
2259- [ ] **Step 5: Check the wiki tests**
2260
2261Run: `go test ./internal/hookd ./internal/ci -run 'Wiki|Parity' 2>&1 | tail -3`
2262Expected: PASS (nothing here changes the push-shape table).
2263
2264- [ ] **Step 6: Commit**
2265
2266```bash
2267git add .gitbay/wiki/Users.org .gitbay/wiki/Admin.org .gitbay/wiki/Threat-Model.org .gitbay/wiki/Parity.org .gitbay/wiki/FAQ.org .gitbay/wiki/CI.org
2268git commit -m "wiki: runners attached to repositories
2269
2270Ref #184"
2271```
2272
2273---
2274
2275### Task 10: Deploy, release, and the tap
2276
2277**Files:**
2278- Modify: `deploy/gitbay-runner.override.conf` (the `ExecStart` line)
2279- Modify: `deploy/release.sh:22` (the binary list)
2280- Create in `krz/homebrew-tap` (separate clone, after the release is tagged): `Formula/gitbay-runner.rb`; modify `Formula/gitbay.rb`
2281
2282- [ ] **Step 1: The bay1 unit claims fork heads**
2283
2284In `deploy/gitbay-runner.override.conf`, the `ExecStart=` line gains ` -untrusted` at the end, and the comment above `ExecStart` gains:
2285
2286```
2287# -untrusted: this runner isolates in podman, so it takes merge request
2288# heads from forks; a runner without a container must not.
2289```
2290
2291- [ ] **Step 2: Release binaries include the runner**
2292
2293`deploy/release.sh`: `for bin in gitbay gitbayd; do` becomes `for bin in gitbay gitbayd gitbay-runner; do`, and the header comment's "gitbay and gitbayd" becomes "gitbay, gitbayd and gitbay-runner".
2294
2295- [ ] **Step 3: Build, vet, and the touched unit tests**
2296
2297Run: `go build ./... && go vet ./... && go test ./internal/store ./internal/control ./cmd/gitbay ./cmd/gitbay-runner ./internal/httpd 2>&1 | tail -8`
2298Expected: all PASS.
2299
2300- [ ] **Step 4: Commit and open the MR**
2301
2302```bash
2303git add deploy/gitbay-runner.override.conf deploy/release.sh
2304git commit -m "deploy: the bay1 runner claims untrusted builds; release ships gitbay-runner
2305
2306Ref #184"
2307git push -u origin user-runners
2308gitbay mr create --source user-runners --target main --title "Runners attached to repositories" --file - <<'MR'
2309A runner key claims builds only for the repositories it is attached to
2310(`repo runner add|list|remove`, also on the settings page). `runner next`
2311skips untrusted builds unless `--untrusted`. Migration 0050.
2312`gitbay-runner init`, `config.toml`, `-identity`, `-untrusted`.
2313
2314After deploy, attach the bay1 runner to krz/gitbay and cmc/ci-smoke as
2315the admin; until then it claims nothing.
2316
2317Ref #184
2318MR
2319```
2320
2321Wait for CI on bay1 (`gitbay build list` on the MR head) before merging: `gitbay mr merge <n> --strategy ff`.
2322
2323- [ ] **Step 5: Deploy and attach (operator, after merge)**
2324
2325```bash
2326make deploy && make deploy-runner
2327ssh -p 2222 root@46.232.248.67 cat /var/lib/gitbay-runner/.ssh/id_ed25519.pub | gitbay repo runner add krz/gitbay
2328ssh -p 2222 root@46.232.248.67 cat /var/lib/gitbay-runner/.ssh/id_ed25519.pub | gitbay repo runner add cmc/ci-smoke
2329gitbay admin runners
2330```
2331
2332The last line must show the `ci` row with its fingerprint and `krz/gitbay,cmc/ci-smoke`.
2333
2334- [ ] **Step 6: The tap, after the release is tagged**
2335
2336In a clone of `https://gitbay.org/krz/homebrew-tap.git`, `Formula/gitbay-runner.rb`:
2337
2338```ruby
2339class GitbayRunner < Formula
2340 desc "CI runner for gitbay: builds the repositories you attach it to"
2341 homepage "https://gitbay.org/krz/gitbay"
2342 url "https://gitbay.org/krz/gitbay.git",
2343 tag: "v1.17.0",
2344 revision: "<commit of the tag>"
2345 license "0BSD"
2346 head "https://gitbay.org/krz/gitbay.git", branch: "main"
2347
2348 depends_on "go" => :build
2349
2350 def install
2351 system "go", "build", *std_go_args(ldflags: "-s -w"), "./cmd/gitbay-runner"
2352 end
2353
2354 service do
2355 run [opt_bin/"gitbay-runner"]
2356 keep_alive true
2357 log_path var/"log/gitbay-runner.log"
2358 error_log_path var/"log/gitbay-runner.err.log"
2359 end
2360
2361 def caveats
2362 <<~EOS
2363 Generate this machine's key and config, and print the key to attach:
2364 gitbay-runner init -remote git@gitbay.org
2365 Attach it to each repository it should build (as a repository admin):
2366 gitbay repo runner add owner/name < ~/.config/gitbay-runner/id_ed25519.pub
2367 Then:
2368 brew services start krz/tap/gitbay-runner
2369 EOS
2370 end
2371
2372 test do
2373 assert_match "gitbay-runner", shell_output("#{bin}/gitbay-runner -version")
2374 end
2375end
2376```
2377
2378In `Formula/gitbay.rb`, set `tag:` and `revision:` to the same release. Check with `brew install --build-from-source krz/tap/gitbay-runner && brew test krz/tap/gitbay-runner`, then commit on a branch of the tap and merge with an MR there.