", passOpts{server: []string{"repo", "runner", "remove"}, needsRepo: true}),
),
```
- [ ] **Step 5: Run the tests**
Run: `go build ./... && go test ./internal/control ./cmd/gitbay 2>&1 | tail -5`
Expected: PASS, including the CLI coverage test.
- [ ] **Step 6: Commit**
```bash
git add internal/control/runnerrepo.go internal/control/runnerrepo_test.go cmd/gitbay/main.go
git commit -m "control, cli: repo runner add, list, remove
Ref #184"
```
---
### Task 5: Web: Runners on the repository settings page
**Files:**
- Modify: `internal/httpd/settings.go:18-49` (`settingsPage`, `settingsForm`) and the `switch` in `settingsSubmit`
- Modify: `internal/web/templates/settings.html` (a section after Dependencies, before Lifecycle)
- Create: `e2e/runnerweb_test.go`
**Interfaces:**
- Consumes: `repo runner list|add|remove` from Task 4; `s.runControlInto`, `s.runControlStdin(u, argv, stdin) (msg string, ok bool)`, `s.runControl`.
- Produces: form fields `field=runner-add` with `key`, and `field=runner-remove` with `fingerprint`.
- [ ] **Step 1: Write the failing e2e test**
`e2e/runnerweb_test.go`:
```go
package e2e
import (
"net/url"
"os"
"strings"
"testing"
)
// The settings page attaches and detaches runners through the same
// commands the CLI uses, and lists what is attached.
func TestRunnerSettingsWeb(t *testing.T) {
inst := startInstanceWith(t, "[web]\nmode = \"accounts\"\n")
aliceKey := inst.newKey(t, "alice")
inst.admin(t, "admin", "user", "create", "alice",
"--key", aliceKey+".pub", "--email", "alice@example.test", "--verified")
if _, errOut, code := inst.ssh(t, aliceKey, "", "repo", "create", "alice/app"); code != 0 {
t.Fatalf("repo create: %s", errOut)
}
runnerKey := inst.newKey(t, "laptop")
pub, _ := os.ReadFile(runnerKey + ".pub")
alice := inst.login(t, aliceKey)
settings := inst.base() + "/alice/app/settings"
_, body := browserGet(t, alice, settings)
if !strings.Contains(body, "No runners attached") {
t.Fatalf("empty state missing:\n%s", body)
}
if status, _ := browserPost(t, alice, settings, url.Values{"field": {"runner-add"}, "key": {string(pub)}}); status != 200 {
t.Fatalf("runner-add post: %d", status)
}
out, _, _ := inst.ssh(t, aliceKey, "", "repo", "runner", "list", "alice/app", "--json")
if !strings.Contains(out, `"fingerprint":"SHA256:`) {
t.Fatalf("not attached after the form: %s", out)
}
fp := out[strings.Index(out, "SHA256:"):]
fp = fp[:strings.Index(fp, `"`)]
_, body = browserGet(t, alice, settings)
if !strings.Contains(body, fp) || !strings.Contains(body, `value="runner-remove"`) {
t.Fatalf("attached runner not listed:\n%s", body)
}
if status, _ := browserPost(t, alice, settings, url.Values{"field": {"runner-remove"}, "fingerprint": {fp}}); status != 200 {
t.Fatalf("runner-remove post: %d", status)
}
if out, _, _ = inst.ssh(t, aliceKey, "", "repo", "runner", "list", "alice/app", "--json"); strings.Contains(out, fp) {
t.Fatalf("still attached after remove: %s", out)
}
}
```
- [ ] **Step 2: Run it to see it fail**
Run: `go test ./e2e -run TestRunnerSettingsWeb -count=1 2>&1 | tail -5`
Expected: FAIL at "empty state missing".
- [ ] **Step 3: Handler changes in `internal/httpd/settings.go`**
`settingsPage` gains:
```go
Runners []store.RepoRunner
```
In `settingsForm`, after the deps read:
```go
var runners []store.RepoRunner
s.runControlInto(u, []string{"repo", "runner", "list", repo.Path()}, &runners)
```
and pass `Runners: runners` to the struct literal.
In `settingsSubmit`'s switch, before `default:`:
```go
case "runner-add":
body := v("key")
if body == "" {
s.settingsRedirect(w, r, "paste the runner's public key")
return
}
msg, ok := s.runControlStdin(u, []string{"repo", "runner", "add", repo}, body+"\n")
if ok {
msg = ""
}
s.settingsRedirect(w, r, msg)
return
case "runner-remove":
argv = []string{"repo", "runner", "remove", repo, v("fingerprint")}
```
- [ ] **Step 4: Template section**
In `internal/web/templates/settings.html`, before `Lifecycle
`:
```html
Runners
{{if .Runners}}
{{range .Runners}}{{.Fingerprint}} {{.Username}}{{if .LastSeen}}, last poll {{.LastSeen}}{{else}}, never polled{{end}}{{if .BuildNumber}}, running {{.BuildRepo}} #{{.BuildNumber}} {{.BuildJob}}{{end}}
{{end}}
{{else}}No runners attached. Builds for this repository run on the runners attached here; a repository with none queues builds nothing claims.
{{end}}
Install gitbay-runner, run gitbay-runner init, and paste the key it prints. The runner builds your commits with the repository's secrets; merge requests from forks wait unless it runs with -untrusted.
```
- [ ] **Step 5: Run the test**
Run: `go test ./e2e -run TestRunnerSettingsWeb -count=1 2>&1 | tail -5`
Expected: PASS.
- [ ] **Step 6: Commit**
```bash
git add internal/httpd/settings.go internal/web/templates/settings.html e2e/runnerweb_test.go
git commit -m "httpd: attach and detach runners on the settings page
Ref #184"
```
---
### Task 6: Runner: config file, `-identity`, `-untrusted`
**Files:**
- Create: `cmd/gitbay-runner/config.go`
- Create: `cmd/gitbay-runner/config_test.go`
- Modify: `cmd/gitbay-runner/main.go` (flag block, `runner` struct, `step`, ssh option assembly)
**Interfaces:**
- Produces:
- `configDir() string`: `$XDG_CONFIG_HOME/gitbay-runner` or `$HOME/.config/gitbay-runner`.
- `defaultConfigPath() string`: `configDir()/config.toml`.
- `configPathFromArgs(args []string, def string) string`: honours `-config X`, `--config X`, `-config=X`.
- `loadConfig(path string) (map[string]string, bool, error)`: flag name to value, false when the file is absent.
- `applyConfig(fs *flag.FlagSet, values map[string]string) error`: `fs.Set` each.
- `identityOpts(path string) []string`: `["-i", path, "-o", "IdentitiesOnly=yes"]` or nil.
- Flags `-config`, `-identity`, `-untrusted`.
- [ ] **Step 1: Write the failing tests**
`cmd/gitbay-runner/config_test.go`:
```go
package main
import (
"flag"
"os"
"path/filepath"
"testing"
)
// A config file sets the flags' values; a flag on the command line wins.
func TestConfigFileFeedsFlagsAndFlagsOverride(t *testing.T) {
dir := t.TempDir()
path := filepath.Join(dir, "config.toml")
os.WriteFile(path, []byte("remote = \"git@example.test\"\npoll = \"9s\"\nuntrusted = true\nidentity = \"/k\"\njobs = 2\n"), 0o600)
values, found, err := loadConfig(path)
if err != nil || !found {
t.Fatalf("loadConfig: found=%v err=%v", found, err)
}
fs := flag.NewFlagSet("t", flag.ContinueOnError)
remote := fs.String("remote", "git@gitbay.org", "")
poll := fs.Duration("poll", 0, "")
untrusted := fs.Bool("untrusted", false, "")
identity := fs.String("identity", "", "")
jobs := fs.Int("jobs", 1, "")
if err := applyConfig(fs, values); err != nil {
t.Fatal(err)
}
if err := fs.Parse([]string{"-poll", "3s"}); err != nil {
t.Fatal(err)
}
if *remote != "git@example.test" || poll.String() != "3s" || !*untrusted || *identity != "/k" || *jobs != 2 {
t.Fatalf("remote=%s poll=%s untrusted=%v identity=%s jobs=%d", *remote, poll, *untrusted, *identity, *jobs)
}
if _, found, err := loadConfig(filepath.Join(dir, "missing.toml")); found || err != nil {
t.Fatalf("missing file: found=%v err=%v", found, err)
}
if _, _, err := loadConfig(path); err != nil {
t.Fatal(err)
}
os.WriteFile(path, []byte("nonsense = \"x\"\n"), 0o600)
if _, _, err := loadConfig(path); err == nil {
t.Fatal("an unknown key was accepted")
}
}
func TestConfigPathFromArgs(t *testing.T) {
for _, tc := range []struct {
args []string
want string
}{
{nil, "/def"},
{[]string{"-once"}, "/def"},
{[]string{"-config", "/a"}, "/a"},
{[]string{"--config", "/b", "-once"}, "/b"},
{[]string{"-config=/c"}, "/c"},
} {
if got := configPathFromArgs(tc.args, "/def"); got != tc.want {
t.Errorf("%v: got %s want %s", tc.args, got, tc.want)
}
}
}
func TestConfigDirHonoursXDG(t *testing.T) {
t.Setenv("XDG_CONFIG_HOME", "/x")
if got := configDir(); got != "/x/gitbay-runner" {
t.Fatalf("got %s", got)
}
t.Setenv("XDG_CONFIG_HOME", "")
t.Setenv("HOME", "/h")
if got := configDir(); got != "/h/.config/gitbay-runner" {
t.Fatalf("got %s", got)
}
}
func TestIdentityOpts(t *testing.T) {
if got := identityOpts(""); got != nil {
t.Fatalf("empty identity produced %v", got)
}
got := identityOpts("/k")
if len(got) != 4 || got[0] != "-i" || got[1] != "/k" || got[3] != "IdentitiesOnly=yes" {
t.Fatalf("got %v", got)
}
}
```
- [ ] **Step 2: Run them to see them fail**
Run: `go test ./cmd/gitbay-runner -run 'TestConfig|TestIdentity' 2>&1 | head -5`
Expected: `undefined: loadConfig` and friends.
- [ ] **Step 3: Write `cmd/gitbay-runner/config.go`**
```go
package main
import (
"errors"
"flag"
"fmt"
"os"
"path/filepath"
"strings"
"github.com/BurntSushi/toml"
)
// The runner takes everything as flags, which does not work under a
// service manager. config.toml in the config directory carries the same
// names; a flag on the command line overrides it (#184).
func configDir() string {
if x := os.Getenv("XDG_CONFIG_HOME"); x != "" {
return filepath.Join(x, "gitbay-runner")
}
return filepath.Join(os.Getenv("HOME"), ".config", "gitbay-runner")
}
func defaultConfigPath() string { return filepath.Join(configDir(), "config.toml") }
// configPathFromArgs finds -config before the flag set is parsed, since
// the file's values must be set before parsing for flags to override them.
func configPathFromArgs(args []string, def string) string {
for i, a := range args {
a = strings.TrimPrefix(a, "-")
if a == "-config" || a == "config" {
if i+1 < len(args) {
return args[i+1]
}
}
if v, ok := strings.CutPrefix(a, "config="); ok {
return v
}
if v, ok := strings.CutPrefix(a, "-config="); ok {
return v
}
}
return def
}
// configKeys is every key the file may carry: the flag names.
var configKeys = map[string]bool{"remote": true, "ssh-opts": true, "clone-base": true, "workdir": true,
"poll": true, "timeout": true, "repos": true, "jobs": true, "image": true, "isolation": true,
"memory": true, "cpus": true, "untrusted": true, "identity": true}
// loadConfig reads path into flag name → value. Absent file: found is
// false and there is no error. An unknown key is an error, not a typo
// the runner silently ignores.
func loadConfig(path string) (values map[string]string, found bool, err error) {
var raw map[string]any
if _, err := toml.DecodeFile(path, &raw); errors.Is(err, os.ErrNotExist) {
return nil, false, nil
} else if err != nil {
return nil, true, fmt.Errorf("%s: %w", path, err)
}
values = map[string]string{}
for k, v := range raw {
if !configKeys[k] {
return nil, true, fmt.Errorf("%s: unknown key %s", path, k)
}
values[k] = fmt.Sprint(v)
}
return values, true, nil
}
// applyConfig sets each value on the flag set, which is what parsing the
// command line would do; parse afterwards and the command line wins.
func applyConfig(fs *flag.FlagSet, values map[string]string) error {
for k, v := range values {
if fs.Lookup(k) == nil {
return fmt.Errorf("config: unknown key %s", k)
}
if err := fs.Set(k, v); err != nil {
return fmt.Errorf("config: %s: %w", k, err)
}
}
return nil
}
// identityOpts is what makes ssh and git use the runner's own key and no
// other: on a laptop the ambient key is the user's full-scope one, which
// the runner protocol refuses.
func identityOpts(path string) []string {
if path == "" {
return nil
}
return []string{"-i", path, "-o", "IdentitiesOnly=yes"}
}
```
- [ ] **Step 4: Wire it into `main.go`**
In `main()`, replace `flag.Parse()` and the flag block with a flag set fed by the config file. Add three flags and keep the others as they are:
```go
var (
configPath = flag.String("config", defaultConfigPath(), "config file; keys are these flag names, flags override it")
identity = flag.String("identity", "", "ssh private key to poll and clone with (default: the key gitbay-runner init generated, if present)")
untrusted = flag.Bool("untrusted", false, "also claim untrusted builds: merge request heads from forks (needs -isolation podman to be safe)")
// ... existing flags unchanged ...
)
path := configPathFromArgs(os.Args[1:], *configPath)
if values, found, err := loadConfig(path); err != nil {
log.Fatal(err)
} else if found {
if err := applyConfig(flag.CommandLine, values); err != nil {
log.Fatal(err)
}
log.Printf("config: %s", path)
}
flag.Parse()
```
After `if *sshOpts != "" { r.sshOpts = strings.Fields(*sshOpts) }`:
```go
if *identity == "" {
if p := filepath.Join(configDir(), "id_ed25519"); fileExists(p) {
*identity = p
}
}
r.sshOpts = append(identityOpts(*identity), r.sshOpts...)
r.untrusted = *untrusted
```
with
```go
func fileExists(p string) bool { _, err := os.Stat(p); return err == nil }
```
`runner` struct gains `untrusted bool`. In `step()`:
```go
args := []string{"runner", "next"}
if r.untrusted {
args = append(args, "--untrusted")
}
args = append(append(args, r.repos...), "--json")
out, err := r.ssh(nil, args...)
```
Both `ssh()` and the `gitSSH` line already use `r.sshOpts`, so the identity reaches both.
Move the `init` dispatch hook in now so Task 7 has a place to land, at the top of `main()`:
```go
if len(os.Args) > 1 && os.Args[1] == "init" {
os.Exit(runInit(os.Args[2:]))
}
```
and a stub in `config.go` until Task 7 replaces it:
```go
func runInit(args []string) int { fmt.Fprintln(os.Stderr, "init: not implemented"); return 2 }
```
- [ ] **Step 5: Run the tests**
Run: `go build ./... && go vet ./cmd/gitbay-runner && go test ./cmd/gitbay-runner 2>&1 | tail -3`
Expected: PASS.
- [ ] **Step 6: Commit**
```bash
git add cmd/gitbay-runner/config.go cmd/gitbay-runner/config_test.go cmd/gitbay-runner/main.go
git commit -m "runner: config.toml, -identity, -untrusted
Ref #184"
```
---
### Task 7: Runner: `gitbay-runner init`
**Files:**
- Create: `cmd/gitbay-runner/init.go` (replaces the stub `runInit` in `config.go`; delete the stub)
- Create: `cmd/gitbay-runner/init_test.go`
**Interfaces:**
- Consumes: `configDir()`, `defaultWorkdir()`, `toolpath.Look("ssh-keygen")`.
- Produces: `runInit(args []string) int`; files `/id_ed25519`, `id_ed25519.pub`, `config.toml`.
- [ ] **Step 1: Write the failing test**
`cmd/gitbay-runner/init_test.go`:
```go
package main
import (
"bytes"
"os"
"os/exec"
"path/filepath"
"strings"
"testing"
)
// init creates the key and config once, prints the key and the attach
// command, and running it again changes nothing.
func TestInitWritesKeyAndConfigOnce(t *testing.T) {
if _, err := exec.LookPath("ssh-keygen"); err != nil {
t.Skip("ssh-keygen not on PATH")
}
dir := t.TempDir()
t.Setenv("XDG_CONFIG_HOME", dir)
var out bytes.Buffer
initOut = &out
defer func() { initOut = os.Stdout }()
if code := runInit([]string{"-remote", "git@example.test"}); code != 0 {
t.Fatalf("init: exit %d\n%s", code, out.String())
}
cdir := filepath.Join(dir, "gitbay-runner")
key := filepath.Join(cdir, "id_ed25519")
pub, err := os.ReadFile(key + ".pub")
if err != nil || !strings.HasPrefix(string(pub), "ssh-ed25519 ") {
t.Fatalf("public key: %v %q", err, pub)
}
if fi, _ := os.Stat(key); fi.Mode().Perm() != 0o600 {
t.Fatalf("private key mode %o", fi.Mode().Perm())
}
if fi, _ := os.Stat(cdir); fi.Mode().Perm() != 0o700 {
t.Fatalf("config dir mode %o", fi.Mode().Perm())
}
cfg, _ := os.ReadFile(filepath.Join(cdir, "config.toml"))
for _, want := range []string{"remote = \"git@example.test\"", "isolation = \"none\"", "untrusted = false", "identity = \"" + key + "\""} {
if !strings.Contains(string(cfg), want) {
t.Fatalf("config lacks %q:\n%s", want, cfg)
}
}
for _, want := range []string{strings.TrimSpace(string(pub)), "gitbay repo runner add owner/name < " + key + ".pub", "https://example.test/owner/name/settings"} {
if !strings.Contains(out.String(), want) {
t.Fatalf("output lacks %q:\n%s", want, out.String())
}
}
out.Reset()
if code := runInit([]string{"-remote", "git@other.test"}); code != 0 {
t.Fatalf("second init: exit %d\n%s", code, out.String())
}
if pub2, _ := os.ReadFile(key + ".pub"); string(pub2) != string(pub) {
t.Fatal("second init replaced the key")
}
if cfg2, _ := os.ReadFile(filepath.Join(cdir, "config.toml")); string(cfg2) != string(cfg) {
t.Fatal("second init rewrote the config")
}
}
// podman needs an image; init refuses to write a config the runner would
// refuse to start with.
func TestInitPodmanNeedsImage(t *testing.T) {
t.Setenv("XDG_CONFIG_HOME", t.TempDir())
var out bytes.Buffer
initOut = &out
defer func() { initOut = os.Stdout }()
if code := runInit([]string{"-isolation", "podman"}); code != 2 {
t.Fatalf("exit %d, want 2:\n%s", code, out.String())
}
}
```
- [ ] **Step 2: Run it to see it fail**
Run: `go test ./cmd/gitbay-runner -run TestInit 2>&1 | head -5`
Expected: `undefined: initOut`.
- [ ] **Step 3: Write `cmd/gitbay-runner/init.go`**
```go
package main
import (
"flag"
"fmt"
"io"
"os"
"os/exec"
"path/filepath"
"strings"
"gitbay.org/gitbay/internal/toolpath"
)
// initOut is where init prints; tests capture it.
var initOut io.Writer = os.Stdout
// runInit makes a fresh install ready to attach: a key of its own, a
// config file the service reads, and the one command to run next. It never
// overwrites a key or a config that exists, so running it twice is safe.
func runInit(args []string) int {
fs := flag.NewFlagSet("init", flag.ContinueOnError)
fs.SetOutput(initOut)
remote := fs.String("remote", "git@gitbay.org", "ssh destination of the gitbay server")
workdir := fs.String("workdir", defaultWorkdir(), "build workspace root")
isolation := fs.String("isolation", isolationNone, "how steps run: none, or podman with -image")
image := fs.String("image", "", "container image for -isolation podman")
if err := fs.Parse(args); err != nil {
return 2
}
if *isolation == isolationPodman && *image == "" {
fmt.Fprintln(initOut, "-isolation podman needs -image [: the runner refuses to start without one, and there is no image to guess")
return 2
}
if *isolation != isolationPodman && *isolation != isolationNone {
fmt.Fprintf(initOut, "unknown isolation %q\n", *isolation)
return 2
}
dir := configDir()
if err := os.MkdirAll(dir, 0o700); err != nil {
fmt.Fprintln(initOut, err)
return 1
}
os.Chmod(dir, 0o700)
key := filepath.Join(dir, "id_ed25519")
if !fileExists(key) {
cmd := exec.Command(toolpath.Look("ssh-keygen"), "-q", "-t", "ed25519", "-N", "", "-C", "gitbay-runner", "-f", key)
if out, err := cmd.CombinedOutput(); err != nil {
fmt.Fprintf(initOut, "ssh-keygen: %v\n%s", err, out)
return 1
}
}
os.Chmod(key, 0o600)
cfgPath := filepath.Join(dir, "config.toml")
if !fileExists(cfgPath) {
var b strings.Builder
fmt.Fprintf(&b, "remote = %q\n", *remote)
fmt.Fprintf(&b, "workdir = %q\n", *workdir)
fmt.Fprintf(&b, "isolation = %q\n", *isolation)
if *image != "" {
fmt.Fprintf(&b, "image = %q\n", *image)
}
fmt.Fprintf(&b, "untrusted = false\n")
fmt.Fprintf(&b, "identity = %q\n", key)
if err := os.WriteFile(cfgPath, []byte(b.String()), 0o600); err != nil {
fmt.Fprintln(initOut, err)
return 1
}
}
pub, err := os.ReadFile(key + ".pub")
if err != nil {
fmt.Fprintln(initOut, err)
return 1
}
host := *remote
if i := strings.LastIndex(host, "@"); i >= 0 {
host = host[i+1:]
}
fmt.Fprintf(initOut, "config: %s\nkey: %s\n\n", cfgPath, key)
if *isolation == isolationNone {
fmt.Fprintln(initOut, "Steps run on this machine as your user, with no container. Untrusted builds\n(merge requests from forks) are excluded unless the runner is started with\n-untrusted, so that means your own commits.\n")
}
fmt.Fprintf(initOut, "This runner's public key:\n\n %s\nAttach it to each repository it should build, as a repository admin:\n\n gitbay repo runner add owner/name < %s.pub\n\nor paste it under Runners at https://%s/owner/name/settings\n\nThen start it:\n\n brew services start krz/tap/gitbay-runner\n\nor run gitbay-runner with no arguments.\n",
strings.TrimSpace(string(pub)), key, host)
return 0
}
```
`isolationNone` and `isolationPodman` are the constants in `isolate.go:20-21`. Delete the stub `runInit` from `config.go`.
- [ ] **Step 4: Run the tests**
Run: `go build ./... && go vet ./cmd/gitbay-runner && go test ./cmd/gitbay-runner 2>&1 | tail -3`
Expected: PASS.
- [ ] **Step 5: Commit**
```bash
git add cmd/gitbay-runner/init.go cmd/gitbay-runner/init_test.go cmd/gitbay-runner/config.go
git commit -m "runner: init generates the key and config and prints the attach step
Ref #184"
```
---
### Task 8: e2e: init, attach, build; fork head waits
**Files:**
- Create: `e2e/runnerattach_test.go`
**Interfaces:**
- Consumes: `buildRunner(t)`, `inst.newKey`, `inst.admin`, `inst.ssh(t, key, stdin, argv...)`, `inst.gitEnv`, `inst.sshURL`, `mustGit`, `inst.port`, `inst.sshDir` (all in `e2e/ci_test.go` and the instance helpers).
- [ ] **Step 1: Write the test**
`e2e/runnerattach_test.go`:
```go
package e2e
import (
"fmt"
"os"
"os/exec"
"path/filepath"
"strings"
"testing"
)
// The whole flow a user goes through: init on their machine, attach the
// printed key to their repository, start the runner from the config init
// wrote. The runner builds their push and leaves a fork's merge request
// head alone until started with -untrusted.
func TestAttachedRunnerBuildsOwnRepo(t *testing.T) {
inst := startInstance(t)
inst.runner = buildRunner(t)
aliceKey := inst.newKey(t, "alice")
inst.admin(t, "admin", "user", "create", "alice", "--key", aliceKey+".pub")
bobKey := inst.newKey(t, "bob")
inst.admin(t, "admin", "user", "create", "bob", "--key", bobKey+".pub")
if _, errOut, code := inst.ssh(t, aliceKey, "", "repo", "create", "alice/app"); code != 0 {
t.Fatalf("repo create: %s", errOut)
}
// init on "alice's laptop".
xdg := t.TempDir()
initCmd := exec.Command(inst.runner, "init", "-remote", "git@127.0.0.1")
initCmd.Env = append(os.Environ(), "XDG_CONFIG_HOME="+xdg)
initOut, err := initCmd.CombinedOutput()
if err != nil {
t.Fatalf("init: %v\n%s", err, initOut)
}
cdir := filepath.Join(xdg, "gitbay-runner")
pub, err := os.ReadFile(filepath.Join(cdir, "id_ed25519.pub"))
if err != nil {
t.Fatal(err)
}
if !strings.Contains(string(initOut), strings.TrimSpace(string(pub))) {
t.Fatalf("init did not print the key:\n%s", initOut)
}
// The unattached key claims nothing, even with a build queued.
work := t.TempDir()
env := inst.gitEnv(aliceKey)
mustGit(t, work, env, "clone", inst.sshURL("alice/app"), "w")
dir := filepath.Join(work, "w")
os.MkdirAll(filepath.Join(dir, ".gitbay"), 0o755)
os.WriteFile(filepath.Join(dir, ".gitbay", "ci.yml"), []byte("jobs:\n unit:\n steps:\n - echo built\n"), 0o644)
mustGit(t, dir, env, "checkout", "-q", "-b", "main")
mustGit(t, dir, env, "add", ".")
mustGit(t, dir, env, "commit", "-q", "-m", "ci")
mustGit(t, dir, env, "push", "-q", "origin", "main")
run := func(extra ...string) string {
t.Helper()
// No -i in ssh-opts: the identity from the config is what
// authenticates, which is the point.
opts := fmt.Sprintf("-p %d -o StrictHostKeyChecking=no -o UserKnownHostsFile=%s -o BatchMode=yes",
inst.port, filepath.Join(inst.sshDir, "known_hosts"))
args := append([]string{"-config", filepath.Join(cdir, "config.toml"), "-once",
"-ssh-opts", opts,
"-clone-base", fmt.Sprintf("ssh://git@127.0.0.1:%d", inst.port),
"-workdir", t.TempDir()}, extra...)
cmd := exec.Command(inst.runner, args...)
cmd.Env = append(os.Environ(), "XDG_CONFIG_HOME="+xdg, "GIT_CONFIG_NOSYSTEM=1", "GIT_CONFIG_GLOBAL=/dev/null")
out, err := cmd.CombinedOutput()
if err != nil {
t.Fatalf("runner: %v\n%s", err, out)
}
return string(out)
}
if out, _, _ := inst.ssh(t, aliceKey, "", "build", "list", "alice/app"); !strings.Contains(out, "unit\tpending") {
t.Fatalf("build not pending before attach: %s", out)
}
// Attach with the printed key.
if _, errOut, code := inst.ssh(t, aliceKey, string(pub), "repo", "runner", "add", "alice/app"); code != 0 {
t.Fatalf("repo runner add: %s", errOut)
}
out, _, _ := inst.ssh(t, aliceKey, "", "repo", "runner", "list", "alice/app", "--json")
if !strings.Contains(out, `"username":"alice"`) || strings.Contains(out, `"last_seen":"20`) {
t.Fatalf("list after attach: %s", out)
}
// The runner builds it.
run()
if out, _, _ = inst.ssh(t, aliceKey, "", "build", "list", "alice/app"); !strings.Contains(out, "unit\tsuccess") {
t.Fatalf("build not built by the attached runner: %s", out)
}
if out, _, _ = inst.ssh(t, aliceKey, "", "repo", "runner", "list", "alice/app", "--json"); !strings.Contains(out, `"last_seen":"20`) {
t.Fatalf("no heartbeat after a poll: %s", out)
}
// bob forks and opens a merge request: an untrusted build in the target.
if _, errOut, code := inst.ssh(t, bobKey, "", "repo", "fork", "alice/app"); code != 0 {
t.Fatalf("fork: %s", errOut)
}
bwork := t.TempDir()
benv := inst.gitEnv(bobKey)
mustGit(t, bwork, benv, "clone", inst.sshURL("bob/app"), "w")
bdir := filepath.Join(bwork, "w")
mustGit(t, bdir, benv, "checkout", "-q", "-b", "feat")
os.WriteFile(filepath.Join(bdir, "f.txt"), []byte("y\n"), 0o644)
mustGit(t, bdir, benv, "add", ".")
mustGit(t, bdir, benv, "commit", "-q", "-m", "change")
mustGit(t, bdir, benv, "push", "-q", "origin", "feat")
if _, _, code := inst.ssh(t, bobKey, "", "build", "cancel", "bob/app", "1"); code != 0 {
t.Fatal("cancel bob's own build")
}
if _, errOut, code := inst.ssh(t, bobKey, "", "mr", "create", "alice/app",
"--source", "bob/app:feat", "--target", "main", "--title", "change"); code != 0 {
t.Fatalf("mr create: %s", errOut)
}
run()
if out, _, _ = inst.ssh(t, aliceKey, "", "build", "list", "alice/app"); strings.Count(out, "pending") != 1 {
t.Fatalf("fork head was claimed without -untrusted:\n%s", out)
}
run("-untrusted")
if out, _, _ = inst.ssh(t, aliceKey, "", "build", "list", "alice/app"); strings.Contains(out, "pending") {
t.Fatalf("fork head not built with -untrusted:\n%s", out)
}
}
```
- [ ] **Step 2: Run it**
Run: `go test ./e2e -run TestAttachedRunnerBuildsOwnRepo -count=1 -v 2>&1 | tail -20`
Expected: PASS. If the fork's build numbering differs (two jobs, or the fork's push queues more than one build), adjust the `build cancel bob/app` loop to cancel every pending build listed by `build list bob/app --json`.
- [ ] **Step 3: Commit**
```bash
git add e2e/runnerattach_test.go
git commit -m "e2e: init, attach, and an attached runner building its repository
Ref #184"
```
---
### Task 9: Docs: wiki pages
**Files:**
- Modify: `.gitbay/wiki/Users.org` (after the "CI builds" section's last paragraph, before "* Large files (LFS)")
- Modify: `.gitbay/wiki/Admin.org:329-340` and `:395-402`
- Modify: `.gitbay/wiki/Threat-Model.org:120-126`
- Modify: `.gitbay/wiki/Parity.org` (repo table, after the `webhooks` row; and the "SSH only, by design" paragraph is unchanged)
- Modify: `.gitbay/wiki/FAQ.org:20-25`
- Modify: `.gitbay/wiki/CI.org` (one sentence after the three mechanisms list)
- [ ] **Step 1: Users: "Your own runner"**
Insert before `* Large files (LFS)`:
```org
** Your own runner
Builds run on runners attached to the repository. An instance need not
offer any: install =gitbay-runner= on a machine of yours and attach it.
#+begin_src sh
brew install krz/tap/gitbay-runner # or a binary from the release
gitbay-runner init -remote git@gitbay.org
#+end_src
=init= generates a key under =~/.config/gitbay-runner/=, writes
=config.toml= beside it, and prints the public key with the command to
attach it:
#+begin_src sh
gitbay repo runner add owner/name < ~/.config/gitbay-runner/id_ed25519.pub
#+end_src
or paste the key under Runners on the repository's settings page. Then
=brew services start krz/tap/gitbay-runner=, or run =gitbay-runner= with
no arguments; it reads the config file, and any flag overrides it.
What it builds: every build for the repositories it is attached to,
with the repository's secrets, and nothing else. Merge requests from
forks are untrusted and wait unless the runner runs with =-untrusted=,
which is only sensible with =-isolation podman -image ][= (see
[[Admin][Admin]]). Attach one runner to several repositories by repeating
=repo runner add=; run several runners on one account by running =init=
on each machine. =repo runner list= shows each attached key, when it
last polled and the build it holds; =repo runner remove =
detaches one (the key stays on your account; =keys remove= drops it).
A runner key reaches only the runner protocol and read-only git, so a
build step that reads it off disk cannot administer your account.
```
- [ ] **Step 2: Admin**
Replace lines 329-340's opening paragraph ("=gitbay-runner= executes builds ... then removed:") with:
```org
=gitbay-runner= executes builds queued by pushes and merge requests. It
polls over SSH with a key of scope =runner=, which reaches only the
runner protocol and read-only git (a runner executes arbitrary
repository code, so the key it holds must not do more). A runner key
claims builds only for the repositories it is attached to, by =repo
runner add= from a repository admin or an instance admin; an admin key
claims any. Users attach their own runners: see the Users page. For an
instance runner, run it as a dedicated unprivileged user on a non-admin
account. =admin user create --key= registers a full-scope key, so the
runner key is added afterwards through a bootstrap key that is then
removed, and attached to each repository it should build:
```
After the existing bootstrap code block, add:
```org
#+begin_src sh
gitbay repo runner add krz/site < /var/lib/gitbay-runner/.ssh/id_ed25519.pub
#+end_src
```
Replace lines 395-402 (from "and the isolation canary, nothing else" to "the boundary is you choosing how to start it.") with:
```org
and the isolation canary, nothing else, because it shares the host with
the forge; any other repository builds on a runner its owner attaches.
=-repos= narrows an admin runner; for a runner key the attachments are
the boundary, held by the server, and =-repos= may only name
repositories among them. =-untrusted= makes a runner claim merge
request heads from forks; the bay1 unit sets it because it isolates in
podman. A runner without it builds trusted commits only.
```
Add `-untrusted` to the `gitbay-runner -remote git@gitbay.org -repos krz/site,krz/docs` example only if that runner isolates; leave the example as is and note under it: "Add =-untrusted= only with =-isolation podman=."
- [ ] **Step 3: Threat-Model**
Replace the "What the runner holds" bullet (lines 120-126) with:
```org
- *What the runner holds.* A key of scope =runner=, which the dispatcher
confines to =runner next=, =runner log= and =runner done= and to
read-only git, and which claims, logs and finishes builds only for
the repositories it is attached to (=repo runner add=). A step that
reads the key off the disk gets exactly that: it cannot administer
the instance, push, read a repository the runner's account cannot, or
touch another repository's builds. An admin key still works for the
runner protocol so an operator can rotate at their own pace; a runner
host should not hold one. Untrusted builds are skipped unless the
runner asks with =-untrusted=, so a runner on a user's machine never
executes a stranger's branch by default.
```
- [ ] **Step 4: Parity, FAQ, CI**
Parity, repo table, after the `webhooks` row:
```org
| runners attach, list, detach | yes | yes | no |
```
The columns are cli, web, ios. iOS is `no`: outstanding, not intended.
FAQ, replace the "Does CI run for my repository on gitbay.org?" answer:
```org
- Does CI run for my repository on gitbay.org? :: On a runner you
attach. The instance's own runner builds the forge's repositories and
its isolation canary, since it shares the host with the forge.
Install =gitbay-runner= on a machine of yours, run =gitbay-runner
init=, and attach the key it prints with =repo runner add= or on the
repository's settings page; see the Users page. A self-hosted
instance can do the same, or run one runner for whichever
repositories its operator attaches it to.
```
CI.org, after the three-mechanism list:
```org
Which runner takes a build is the fourth: a build is claimed only by a
runner attached to its repository (or an instance admin's runner), and
an untrusted build only by one started with =-untrusted=. A repository
with no runner attached queues builds nothing claims. See the Users
page.
```
- [ ] **Step 5: Check the wiki tests**
Run: `go test ./internal/hookd ./internal/ci -run 'Wiki|Parity' 2>&1 | tail -3`
Expected: PASS (nothing here changes the push-shape table).
- [ ] **Step 6: Commit**
```bash
git add .gitbay/wiki/Users.org .gitbay/wiki/Admin.org .gitbay/wiki/Threat-Model.org .gitbay/wiki/Parity.org .gitbay/wiki/FAQ.org .gitbay/wiki/CI.org
git commit -m "wiki: runners attached to repositories
Ref #184"
```
---
### Task 10: Deploy, release, and the tap
**Files:**
- Modify: `deploy/gitbay-runner.override.conf` (the `ExecStart` line)
- Modify: `deploy/release.sh:22` (the binary list)
- Create in `krz/homebrew-tap` (separate clone, after the release is tagged): `Formula/gitbay-runner.rb`; modify `Formula/gitbay.rb`
- [ ] **Step 1: The bay1 unit claims fork heads**
In `deploy/gitbay-runner.override.conf`, the `ExecStart=` line gains ` -untrusted` at the end, and the comment above `ExecStart` gains:
```
# -untrusted: this runner isolates in podman, so it takes merge request
# heads from forks; a runner without a container must not.
```
- [ ] **Step 2: Release binaries include the runner**
`deploy/release.sh`: `for bin in gitbay gitbayd; do` becomes `for bin in gitbay gitbayd gitbay-runner; do`, and the header comment's "gitbay and gitbayd" becomes "gitbay, gitbayd and gitbay-runner".
- [ ] **Step 3: Build, vet, and the touched unit tests**
Run: `go build ./... && go vet ./... && go test ./internal/store ./internal/control ./cmd/gitbay ./cmd/gitbay-runner ./internal/httpd 2>&1 | tail -8`
Expected: all PASS.
- [ ] **Step 4: Commit and open the MR**
```bash
git add deploy/gitbay-runner.override.conf deploy/release.sh
git commit -m "deploy: the bay1 runner claims untrusted builds; release ships gitbay-runner
Ref #184"
git push -u origin user-runners
gitbay mr create --source user-runners --target main --title "Runners attached to repositories" --file - <<'MR'
A runner key claims builds only for the repositories it is attached to
(`repo runner add|list|remove`, also on the settings page). `runner next`
skips untrusted builds unless `--untrusted`. Migration 0050.
`gitbay-runner init`, `config.toml`, `-identity`, `-untrusted`.
After deploy, attach the bay1 runner to krz/gitbay and cmc/ci-smoke as
the admin; until then it claims nothing.
Ref #184
MR
```
Wait for CI on bay1 (`gitbay build list` on the MR head) before merging: `gitbay mr merge --strategy ff`.
- [ ] **Step 5: Deploy and attach (operator, after merge)**
```bash
make deploy && make deploy-runner
ssh -p 2222 root@46.232.248.67 cat /var/lib/gitbay-runner/.ssh/id_ed25519.pub | gitbay repo runner add krz/gitbay
ssh -p 2222 root@46.232.248.67 cat /var/lib/gitbay-runner/.ssh/id_ed25519.pub | gitbay repo runner add cmc/ci-smoke
gitbay admin runners
```
The last line must show the `ci` row with its fingerprint and `krz/gitbay,cmc/ci-smoke`.
- [ ] **Step 6: The tap, after the release is tagged**
In a clone of `https://gitbay.org/krz/homebrew-tap.git`, `Formula/gitbay-runner.rb`:
```ruby
class GitbayRunner < Formula
desc "CI runner for gitbay: builds the repositories you attach it to"
homepage "https://gitbay.org/krz/gitbay"
url "https://gitbay.org/krz/gitbay.git",
tag: "v1.17.0",
revision: ""
license "0BSD"
head "https://gitbay.org/krz/gitbay.git", branch: "main"
depends_on "go" => :build
def install
system "go", "build", *std_go_args(ldflags: "-s -w"), "./cmd/gitbay-runner"
end
service do
run [opt_bin/"gitbay-runner"]
keep_alive true
log_path var/"log/gitbay-runner.log"
error_log_path var/"log/gitbay-runner.err.log"
end
def caveats
<<~EOS
Generate this machine's key and config, and print the key to attach:
gitbay-runner init -remote git@gitbay.org
Attach it to each repository it should build (as a repository admin):
gitbay repo runner add owner/name < ~/.config/gitbay-runner/id_ed25519.pub
Then:
brew services start krz/tap/gitbay-runner
EOS
end
test do
assert_match "gitbay-runner", shell_output("#{bin}/gitbay-runner -version")
end
end
```
In `Formula/gitbay.rb`, set `tag:` and `revision:` to the same release. Check with `brew install --build-from-source krz/tap/gitbay-runner && brew test krz/tap/gitbay-runner`, then commit on a branch of the tap and merge with an MR there.
]