docs/plans/2026-09-27-server-hardening.md

   1# Server hardening implementation plan
   2
   3> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
   4
   5**Goal:** Close #281, #280, #279, #282, #275 and #262 from the
   62026-09-27 architecture review: an explicit TLS floor, mail that
   7refuses plaintext to a remote relay, mirror syncs that connect only to
   8an address checked at sync time, an authenticated hook socket, audited
   9refusals with a tamper-evident audit log, and a shared limit on git
  10pack generation.
  11
  12**Architecture:** Six MRs, each small enough to review alone. The TLS
  13and mail changes are local to `cmd/gitbayd/main.go` and
  14`internal/mail`. Mirror sync resolves and checks the host itself, then
  15pins git's connection to those addresses with `http.curloptResolve`.
  16The hook socket gets mode 0600, a Linux peer-uid check behind build
  17tags, and a per-push token that sshd stores (hashed) in a new
  18`push_tokens` table and hookd requires. Audit rows gain a SHA-256
  19chain over an immutable actor column, refusals of mutating commands
  20are recorded through a per-actor limiter, and the daemon writes each
  21row to its log. A new `internal/packlimit` package holds one limiter
  22shared by the SSH, smart HTTP and git:// transports.
  23
  24**Tech stack:** Go 1.27, SQLite (modernc, `_txlock=immediate`),
  25`golang.org/x/crypto/ssh`, `net/smtp`, `crypto/tls`, `syscall`
  26(Linux `SO_PEERCRED`), git ≥ 2.37 (`http.curloptResolve`), cobra.
  27
  28**Spec:** the issue texts of #262, #275, #279, #280, #281, #282 on
  29krz/gitbay, and the decisions recorded in the brief for this plan set
  30(copied under "Decisions" below).
  31
  32## Global constraints
  33
  34- Each MR on its own branch off `main`. Commits are signed (the repo
  35  refuses unsigned), messages reference issues (`Ref #N`, and
  36  `Closes #N` on the commit that finishes one). No attribution to any
  37  assistant, model or AI anywhere: commits, MR bodies, comments.
  38- MR: `gitbay mr create --source <branch> --target main --title "..."`;
  39  merge with `gitbay mr merge <n> --strategy ff` once CI is green, then
  40  delete the branch locally and on the remote. Behind main → rebase,
  41  force-push, merge again.
  42- Locally: `go build ./...`, `go vet ./...`, unit tests of touched
  43  packages, and at most the one e2e test being written
  44  (`go test ./e2e -run TestName -count=1`). CI on bay1 runs the full suite.
  45- Registries that fail CI when a new thing lacks its row: top-level route
  46  word in `internal/policy/names.go`; new page template in the width map
  47  of `TestMainWidthClass` (`internal/web/web_test.go`); new `ReadOnly`
  48  command in `readArgs` in `e2e/readonly_test.go`; new control command
  49  needs a `pass()` entry in `cmd/gitbay/main.go` (coverage test);
  50  a command reading stdin needs `ReadsStdin: true`.
  51  This plan adds no route, no template and no control command:
  52  `gitbayd admin audit verify` is a host-local cobra command, not a
  53  registry entry.
  54- New migrations: the highest today is 0059. Six plans are written in
  55  parallel, so numbers are pre-assigned: plan 1 uses 0060–0064, plan 2
  56  0065–0068, plan 3 0069–0071, plan 4 0072–0074, plan 5 0075–0077,
  57  plan 6 0078–0079. Whoever lands second renumbers to the next free
  58  number at execution time. Migrations come in `.up.sql`/`.down.sql`
  59  pairs. Hand-written SQL, no ORM. This plan uses 0069
  60  (`push_tokens`, MR 4) and 0070 (`audit_chain`, MR 5); 0071 is unused.
  61- Secrets travel on stdin, never argv; never logged or echoed. The
  62  push token is minted per receive-pack and passed only in the hook's
  63  environment; only its SHA-256 is stored.
  64- Wiki pages live in `.gitbay/wiki/` (Parity, API, Admin, Threat-Model,
  65  CI, Users, Performance, and the `Architecture/` folder with its
  66  Known-Gaps table and controls matrix). Update the page in the same MR
  67  that changes the behaviour it describes, and close the matching
  68  Known-Gaps row (`Architecture/10-Known-Gaps.org`) and controls row
  69  (`Architecture/09-Controls.org`).
  70- Writing style: plain, direct, no hype; code comments match the
  71  surrounding density. Comments and docs state facts, never
  72  before/after narration.
  73- The worktree may carry another session's edits; work in a fresh
  74  worktree per MR (`git worktree add ../gitbay-<branch> -b <branch> main`).
  75- `CHANGELOG.org` is written at release time (`CHANGELOG: vX.Y.Z`
  76  commits), not in these MRs. The release notes each MR needs are
  77  listed at the end of this plan.
  78
  79## Decisions (from the brief)
  80
  81- #275: audit refused mutating commands (rate-limited per actor); each
  82  audit row carries a hash of the previous row, `gitbayd admin audit
  83  verify` checks the chain, and rows are also written to the journal
  84  (a slog line on the daemon's stderr, which the unit's journal
  85  collects).
  86- #282: chmod 0600, SO_PEERCRED uid check, and a per-push token in the
  87  hook environment that hookd requires.
  88- #279: resolve and check before each sync and pin the address for git.
  89
  90## Findings from the code that shape this plan
  91
  92- Mirror URLs are http/https only. `runMirrorAdd`
  93  (`internal/control/mirrorcmd.go:58`) calls `webhook.ValidateURL`,
  94  which refuses any other scheme (`internal/webhook/webhook.go:31-33`).
  95  There is no ssh mirror URL to pin. Sync (`internal/mirror/mirror.go:76-116`)
  96  runs `git fetch|push <url>` with a replaced environment (no proxy
  97  variables), so the pin is `-c http.curloptResolve=<host>:<port>:<addrs>`
  98  plus `-c http.followRedirects=false` (git's default follows a
  99  redirect on the first request, which would reach an unchecked
 100  host). Sync also refuses a non-http(s) scheme, for rows that
 101  predate the save-time check.
 102- Hook environment is set in exactly one place,
 103  `runGit` (`internal/sshd/sshd.go:441-447`). `runGit` runs in the
 104  daemon (embedded SSH) and in `gitbayd shell` (system SSH mode,
 105  `cmd/gitbayd/system.go:97`), a separate process. The push token must
 106  therefore be visible across processes: it goes in SQLite, not in
 107  daemon memory.
 108- In both SSH modes the hook runs as the uid git runs as, which is the
 109  daemon's (`User=gitbay`, `deploy/cloud-init.yaml:210`; system mode
 110  logs in as the account that owns `<root>`). So the peer-uid rule is
 111  "equal to `os.Getuid()`".
 112- `audit_log.actor_id` is `REFERENCES users(id) ON DELETE SET NULL`
 113  (`0001_init.up.sql:186`). Hashing it would break the chain when an
 114  account is deleted, so migration 0070 adds `actor_ref`, the id as
 115  written, and the hash covers that.
 116- Retention deletes the oldest audit rows (`internal/store/retention.go:75`).
 117  Verification therefore takes the first remaining chained row's
 118  `prev_hash` as given.
 119- Audit rows are written from three kinds of process: the daemon,
 120  `gitbayd shell`, and `gitbayd admin …`. The chain is computed inside
 121  one `BEGIN IMMEDIATE` transaction (the store's DSN sets
 122  `_txlock=immediate`, `internal/store/store.go:47`), which serialises
 123  writers across processes. The journal line is written only where
 124  stderr is the journal: `gitbayd serve`.
 125- Pack generation runs in four places: SSH `gitutil.Transport`
 126  (`internal/gitutil/gitutil.go:39`), smart-HTTP `uploadPack`
 127  (`internal/httpd/smart.go:122`), git:// (`internal/gitd/gitd.go:71-77`),
 128  and SSH `git-upload-archive`. The info/refs advertisement
 129  (`smart.go:89`) and protocol-v2 `ls-refs` POSTs are ref listings and
 130  stay outside the limit. Web archives are already bounded by
 131  `archiveTimeout` and `MaxArchiveBytes` (`internal/gitutil/read.go:124-130`)
 132  and stay outside. receive-pack stays outside: killing or queueing
 133  it risks losing post-receive, which runs after the client has its
 134  report.
 135- `done` in `handleSession` (`internal/sshd/sshd.go:263-270`) closes
 136  when the client closes the channel *or* the server stops. A queued
 137  clone should give up on either; a running clone should be killed
 138  only when the client left, never on a restart drain.
 139- Dispatcher tests call `Dispatch` with a nil `Store` and expect
 140  refusals (`internal/control/control_test.go:192-231`). Auditing a
 141  refusal must tolerate a nil store.
 142
 143## Order and dependencies
 144
 145| # | Branch | Closes | Migration | Depends on |
 146|---|---|---|---|---|
 147| 1 | `https-tls-minimum` | #281 | — | — |
 148| 2 | `mail-require-tls` | #280 | — | — |
 149| 3 | `mirror-pin-address` | #279 | — | — |
 150| 4 | `hook-socket-auth` | #282 | 0069 | — |
 151| 5 | `audit-refusals-chain` | #275 | 0070 | — |
 152| 6 | `pack-limit` | #262 | — | MR 5 (both edit `sshd.Exec`) |
 153
 154Cross-plan overlaps, all textual (rebase, no design dependency):
 155
 156- Plan 1 (credentials-and-sessions, #256) closes connections when a key
 157  is removed; it edits `internal/sshd/sshd.go`, as do MRs 4, 5 and 6.
 158- Plan 4 (data-at-rest-and-backup, #273) decrypts `m.Token` in
 159  `internal/mirror/mirror.go`'s `sync`; MR 3 edits the same function.
 160  Whichever lands second keeps both: decryption of the token and the
 161  resolve-check-pin block.
 162- Plan 5 (web-ux, #261 doc drift) may edit the `[limits]` section of
 163  `Admin.org`, which MR 6 also edits (its "reserved, not yet enforced"
 164  line for `max_pack_bytes`/`ssh_auth_rate` is stale; leave that line
 165  to #261 unless it has not landed when MR 6 merges).
 166
 167## File map
 168
 169| File | MR | Responsibility |
 170|---|---|---|
 171| `cmd/gitbayd/main.go` | 1, 5, 6 | `serverTLS`; `st.AuditJournal`; `admin audit verify` wiring; one `packlimit.Limiter` |
 172| `cmd/gitbayd/tls_test.go` | 1 | TLS floor |
 173| `internal/config/config.go` | 2, 6 | `Mail.RequireTLS`, `Mail.TLS`, `Mail.TLSRequired`; `Limits.Pack*`, `PackLimits` |
 174| `internal/config/config_test.go` | 2, 6 | validation and defaults |
 175| `internal/mail/mail.go`, `mail_test.go` | 2 | require TLS, implicit TLS |
 176| `internal/webhook/webhook.go` | 3 | `CheckAddrs` |
 177| `internal/mirror/mirror.go`, `mirror_test.go` | 3 | resolve, check, pin |
 178| `internal/store/migrations/0069_push_tokens.*.sql` | 4 | table |
 179| `internal/store/pushtokens.go`, `pushtokens_test.go` | 4 | create, look up, delete |
 180| `internal/store/retention.go` | 4 | sweep expired push tokens |
 181| `internal/hookd/hookd.go`, `peercred_linux.go`, `peercred_other.go`, `socket_test.go` | 4 | 0600, peer uid, token check |
 182| `internal/sshd/sshd.go` | 4, 5, 6 | mint token; audit refused push; acquire pack slot |
 183| `cmd/gitbayd/hook.go` | 4 | send the token |
 184| `internal/store/migrations/0070_audit_chain.*.sql` | 5 | chain columns |
 185| `internal/store/audit.go`, `auditchain_test.go` | 5 | chained append, journal, verify |
 186| `internal/control/control.go`, `auditrefusal.go`, `auditrefusal_test.go` | 5 | refusal auditing |
 187| `cmd/gitbayd/auditverify.go` | 5 | `gitbayd admin audit verify` |
 188| `internal/sshd/refusal_test.go` | 5, 6 | refused push audited; busy clone |
 189| `e2e/audit_test.go` | 5 | `TestAuditChainVerify` |
 190| `internal/packlimit/packlimit.go`, `packlimit_test.go` | 6 | the limiter |
 191| `internal/gitutil/gitutil.go` | 6 | `Transport` takes a context |
 192| `internal/httpd/smart.go` (`Server`, `New`, `uploadPack`), `packlimit_test.go` | 6 | limit on POST upload-pack, `ls-refs` outside |
 193| `internal/gitd/gitd.go`, `gitd_test.go` | 6 | limit on git:// |
 194| `cmd/gitbayd/system.go` | 5, 6 | `Exec` signature |
 195| `deploy/clonebench.sh` | 6 | concurrent-clone benchmark |
 196| `.gitbay/wiki/Admin.org`, `Performance.org`, `Threat-Model.org`, `Architecture/03-Deployment.org`, `04-Trust-Boundaries.org`, `06-Data-and-Cryptography.org`, `09-Controls.org`, `10-Known-Gaps.org` | 1–6 | docs per MR |
 197
 198---
 199
 200# MR 1: explicit TLS minimum (branch `https-tls-minimum`, closes #281)
 201
 202### Task 1.1: `serverTLS` sets TLS 1.2 on both TLS modes
 203
 204**Files:**
 205- Create: `cmd/gitbayd/tls.go`
 206- Create: `cmd/gitbayd/tls_test.go`
 207- Modify: `cmd/gitbayd/main.go:241-242` (`case "files"`), `:301-302` (`case "acme"`)
 208- Modify: `.gitbay/wiki/Admin.org` (`** [http]`, lines 72-80),
 209  `.gitbay/wiki/Architecture/06-Data-and-Cryptography.org` (HTTPS row, line 59),
 210  `.gitbay/wiki/Architecture/10-Known-Gaps.org` (#281 row)
 211
 212**Interfaces:**
 213- Produces: `func serverTLS(c *tls.Config) *tls.Config` in package `main` — sets `MinVersion = tls.VersionTLS12` on `c` (a new config when nil) and returns it.
 214
 215- [ ] **Step 1: Write the failing test**
 216
 217`cmd/gitbayd/tls_test.go`:
 218
 219```go
 220package main
 221
 222import (
 223	"crypto/tls"
 224	"testing"
 225)
 226
 227// The floor is stated in code rather than inherited from the Go
 228// release the binary was built with (#281).
 229func TestServerTLSMinimum(t *testing.T) {
 230	if got := serverTLS(nil).MinVersion; got != tls.VersionTLS12 {
 231		t.Fatalf("files mode: MinVersion %#x, want %#x", got, tls.VersionTLS12)
 232	}
 233	// autocert's config carries the ALPN protocols TLS-ALPN-01 needs;
 234	// setting the floor must keep them.
 235	base := &tls.Config{NextProtos: []string{"h2", "http/1.1", "acme-tls/1"}}
 236	got := serverTLS(base)
 237	if got.MinVersion != tls.VersionTLS12 || len(got.NextProtos) != 3 {
 238		t.Fatalf("acme mode: %+v", got)
 239	}
 240}
 241```
 242
 243- [ ] **Step 2: Run it and see it fail**
 244
 245Run: `go test ./cmd/gitbayd -run TestServerTLSMinimum -count=1`
 246Expected: build failure, `undefined: serverTLS`.
 247
 248- [ ] **Step 3: Implement**
 249
 250`cmd/gitbayd/tls.go`:
 251
 252```go
 253package main
 254
 255import "crypto/tls"
 256
 257// serverTLS sets the HTTPS listener's protocol floor: TLS 1.2 and 1.3,
 258// with Go's default cipher suites.
 259func serverTLS(c *tls.Config) *tls.Config {
 260	if c == nil {
 261		c = &tls.Config{}
 262	}
 263	c.MinVersion = tls.VersionTLS12
 264	return c
 265}
 266```
 267
 268In `cmd/gitbayd/main.go`, `case "files":` becomes:
 269
 270```go
 271				case "files":
 272					hs.TLSConfig = serverTLS(nil)
 273					errCh <- hs.ListenAndServeTLS(cfg.HTTP.CertFile, cfg.HTTP.KeyFile)
 274```
 275
 276and in `case "acme":` replace `hs.TLSConfig = m.TLSConfig()` with:
 277
 278```go
 279					hs.TLSConfig = serverTLS(m.TLSConfig())
 280```
 281
 282`ListenAndServeTLS` clones `TLSConfig` and loads the certificate files
 283into the clone, so setting it in files mode changes nothing else.
 284
 285- [ ] **Step 4: Run the test and the package**
 286
 287Run: `go test ./cmd/gitbayd -count=1 && go vet ./cmd/gitbayd`
 288Expected: PASS.
 289
 290- [ ] **Step 5: Docs**
 291
 292`Admin.org`, append to the `** [http]` bullet list, after the `=off=` bullet:
 293
 294```org
 295- The HTTPS listener accepts TLS 1.2 and 1.3 only (=serverTLS= in
 296  =cmd/gitbayd/tls.go=), with the default cipher suites of the Go
 297  release the binary was built with. =openssl s_client -connect
 298  <host>:443 -tls1_1= fails the handshake.
 299```
 300
 301`Architecture/06-Data-and-Cryptography.org`, HTTPS row:
 302
 303```org
 304| HTTPS                    | TLS 1.2 minimum (=cmd/gitbayd/tls.go=), ACME or operator certificates; HSTS one year |
 305```
 306
 307`Architecture/10-Known-Gaps.org`: delete the `#281` row.
 308
 309- [ ] **Step 6: Commit**
 310
 311```bash
 312git add cmd/gitbayd/tls.go cmd/gitbayd/tls_test.go cmd/gitbayd/main.go \
 313  .gitbay/wiki/Admin.org .gitbay/wiki/Architecture/06-Data-and-Cryptography.org \
 314  .gitbay/wiki/Architecture/10-Known-Gaps.org
 315git commit -S -m "https: TLS 1.2 minimum, set explicitly
 316
 317Closes #281"
 318```
 319
 320- [ ] **Step 7: MR and merge**
 321
 322```bash
 323git push -u origin https-tls-minimum
 324gitbay mr create --source https-tls-minimum --target main --title "https: TLS 1.2 minimum, set explicitly"
 325```
 326
 327After CI is green: `gitbay mr merge <n> --strategy ff`, then
 328`git branch -d https-tls-minimum && git push origin --delete https-tls-minimum`.
 329
 330---
 331
 332# MR 2: mail requires TLS to a remote relay (branch `mail-require-tls`, closes #280)
 333
 334### Task 2.1: config `mail.require_tls` and `mail.tls`
 335
 336**Files:**
 337- Modify: `internal/config/config.go:211-216` (`Mail`), `Validate` (after the `[mail] from` check, line 406-408)
 338- Test: `internal/config/config_test.go`
 339
 340**Interfaces:**
 341- Produces: `Mail.RequireTLS *bool` (`toml:"require_tls,omitempty"`), `Mail.TLS string` (`toml:"tls,omitempty"`, `""`/`"starttls"`/`"implicit"`), `func (m Mail) TLSRequired() bool`.
 342
 343- [ ] **Step 1: Write the failing tests**
 344
 345Append to `internal/config/config_test.go`:
 346
 347```go
 348func TestMailTLSRequired(t *testing.T) {
 349	off, on := false, true
 350	for _, tc := range []struct {
 351		m    Mail
 352		want bool
 353	}{
 354		{Mail{SMTPHost: "smtp.example.com:587"}, true},
 355		{Mail{SMTPHost: "smtp.example.com"}, true},
 356		{Mail{SMTPHost: "localhost:25"}, false},
 357		{Mail{SMTPHost: "localhost"}, false},
 358		{Mail{SMTPHost: "127.0.0.1:25"}, false},
 359		{Mail{SMTPHost: "[::1]:25"}, false},
 360		{Mail{SMTPHost: "smtp.example.com:587", RequireTLS: &off}, false},
 361		{Mail{SMTPHost: "127.0.0.1:25", RequireTLS: &on}, true},
 362	} {
 363		if got := tc.m.TLSRequired(); got != tc.want {
 364			t.Errorf("%+v: TLSRequired = %v, want %v", tc.m, got, tc.want)
 365		}
 366	}
 367}
 368```
 369
 370Add one case to the `cases` table in `TestContradictions`:
 371
 372```go
 373		{
 374			"unknown mail.tls",
 375			minimal + "\n[mail]\nsmtp_host = \"mx.example\"\nfrom = \"gitbay@example\"\ntls = \"ssl\"\n",
 376			"mail.tls must be starttls or implicit",
 377		},
 378```
 379
 380- [ ] **Step 2: Run them and see them fail**
 381
 382Run: `go test ./internal/config -run 'TestMailTLSRequired|TestContradictions' -count=1`
 383Expected: build failure, `unknown field RequireTLS` / `TLSRequired undefined`.
 384
 385- [ ] **Step 3: Implement**
 386
 387`Mail` in `internal/config/config.go`:
 388
 389```go
 390type Mail struct {
 391	SMTPHost string `toml:"smtp_host"` // host:port (port defaults to 587, 465 with tls = "implicit")
 392	From     string `toml:"from"`
 393	SMTPUser string `toml:"smtp_user,omitempty"`
 394	SMTPPass string `toml:"smtp_pass,omitempty"`
 395	// RequireTLS fails delivery when the relay does not offer STARTTLS,
 396	// instead of sending in clear. Unset, it is on for any relay but
 397	// localhost or a loopback address (TLSRequired).
 398	RequireTLS *bool `toml:"require_tls,omitempty"`
 399	// TLS is "starttls" (the default, also when empty) or "implicit":
 400	// TLS from the first byte, as relays on port 465 expect.
 401	TLS string `toml:"tls,omitempty"`
 402}
 403
 404// TLSRequired reports whether mail must not go to the relay in clear.
 405func (m Mail) TLSRequired() bool {
 406	if m.RequireTLS != nil {
 407		return *m.RequireTLS
 408	}
 409	host := m.SMTPHost
 410	if h, _, err := net.SplitHostPort(host); err == nil {
 411		host = h
 412	}
 413	host = strings.Trim(host, "[]")
 414	if host == "localhost" {
 415		return false
 416	}
 417	ip := net.ParseIP(host)
 418	return ip == nil || !ip.IsLoopback()
 419}
 420```
 421
 422In `Validate`, after the `[mail] from is required` check:
 423
 424```go
 425	if t := c.Mail.TLS; t != "" && t != "starttls" && t != "implicit" {
 426		errs = append(errs, fmt.Errorf("mail.tls must be starttls or implicit, got %q", t))
 427	}
 428```
 429
 430- [ ] **Step 4: Run the package**
 431
 432Run: `go test ./internal/config -count=1`
 433Expected: PASS.
 434
 435- [ ] **Step 5: Commit**
 436
 437```bash
 438git add internal/config/config.go internal/config/config_test.go
 439git commit -S -m "config: mail.require_tls and mail.tls
 440
 441Ref #280"
 442```
 443
 444### Task 2.2: `mail.Send` refuses plaintext when TLS is required; implicit TLS
 445
 446**Files:**
 447- Modify: `internal/mail/mail.go` (whole `Send`, package comment lines 1-3)
 448- Create: `internal/mail/mail_test.go`
 449
 450**Interfaces:**
 451- Consumes: `config.Mail.TLSRequired()`, `config.Mail.TLS` (Task 2.1).
 452- Produces: unexported `var rootCAs *x509.CertPool` (tests set it); `Send` signature unchanged.
 453
 454- [ ] **Step 1: Write the failing tests**
 455
 456`internal/mail/mail_test.go`:
 457
 458```go
 459package mail
 460
 461import (
 462	"bufio"
 463	"crypto/tls"
 464	"crypto/x509"
 465	"fmt"
 466	"net"
 467	"net/http"
 468	"net/http/httptest"
 469	"strings"
 470	"sync"
 471	"testing"
 472
 473	"gitbay.org/gitbay/internal/config"
 474)
 475
 476// fakeRelay is an SMTP server that never offers STARTTLS. Given a TLS
 477// config it speaks TLS from the first byte, as a port-465 relay does.
 478type fakeRelay struct {
 479	addr string
 480	mu   sync.Mutex
 481	data []string
 482}
 483
 484func startRelay(t *testing.T, tlsCfg *tls.Config) *fakeRelay {
 485	t.Helper()
 486	ln, err := net.Listen("tcp", "127.0.0.1:0")
 487	if err != nil {
 488		t.Fatal(err)
 489	}
 490	if tlsCfg != nil {
 491		ln = tls.NewListener(ln, tlsCfg)
 492	}
 493	t.Cleanup(func() { ln.Close() })
 494	f := &fakeRelay{addr: ln.Addr().String()}
 495	go func() {
 496		for {
 497			conn, err := ln.Accept()
 498			if err != nil {
 499				return
 500			}
 501			go f.serve(conn)
 502		}
 503	}()
 504	return f
 505}
 506
 507func (f *fakeRelay) serve(conn net.Conn) {
 508	defer conn.Close()
 509	r := bufio.NewReader(conn)
 510	fmt.Fprint(conn, "220 fake\r\n")
 511	var body strings.Builder
 512	inData := false
 513	for {
 514		line, err := r.ReadString('\n')
 515		if err != nil {
 516			return
 517		}
 518		line = strings.TrimRight(line, "\r\n")
 519		switch {
 520		case inData && line == ".":
 521			f.mu.Lock()
 522			f.data = append(f.data, body.String())
 523			f.mu.Unlock()
 524			inData = false
 525			fmt.Fprint(conn, "250 ok\r\n")
 526		case inData:
 527			body.WriteString(line + "\n")
 528		case strings.HasPrefix(line, "EHLO"), strings.HasPrefix(line, "HELO"):
 529			fmt.Fprint(conn, "250-fake\r\n250 SIZE 1000000\r\n")
 530		case line == "DATA":
 531			inData = true
 532			fmt.Fprint(conn, "354 go\r\n")
 533		case line == "QUIT":
 534			fmt.Fprint(conn, "221 bye\r\n")
 535			return
 536		default:
 537			fmt.Fprint(conn, "250 ok\r\n")
 538		}
 539	}
 540}
 541
 542func (f *fakeRelay) delivered() int {
 543	f.mu.Lock()
 544	defer f.mu.Unlock()
 545	return len(f.data)
 546}
 547
 548func mailCfg(host string) config.Config {
 549	var cfg config.Config
 550	cfg.Mail.SMTPHost, cfg.Mail.From = host, "gitbay@example.test"
 551	return cfg
 552}
 553
 554func TestRequireTLSRefusesPlaintextRelay(t *testing.T) {
 555	relay := startRelay(t, nil)
 556	cfg := mailCfg(relay.addr)
 557	on := true
 558	cfg.Mail.RequireTLS = &on
 559	err := Send(cfg, "a@example.test", "subject", "body")
 560	if err == nil || !strings.Contains(err.Error(), "STARTTLS") {
 561		t.Fatalf("Send = %v, want a refusal naming STARTTLS", err)
 562	}
 563	if n := relay.delivered(); n != 0 {
 564		t.Fatalf("%d message(s) sent in clear", n)
 565	}
 566}
 567
 568// A loopback relay has no network to cross; the default leaves it in
 569// clear, which is what the e2e suite's fake relay relies on.
 570func TestLoopbackRelayDefaultsToPlaintext(t *testing.T) {
 571	relay := startRelay(t, nil)
 572	if err := Send(mailCfg(relay.addr), "a@example.test", "subject", "body"); err != nil {
 573		t.Fatal(err)
 574	}
 575	if n := relay.delivered(); n != 1 {
 576		t.Fatalf("delivered %d, want 1", n)
 577	}
 578}
 579
 580func TestImplicitTLS(t *testing.T) {
 581	ts := httptest.NewTLSServer(http.NotFoundHandler())
 582	defer ts.Close()
 583	pool := x509.NewCertPool()
 584	pool.AddCert(ts.Certificate())
 585	prev := rootCAs
 586	rootCAs = pool
 587	defer func() { rootCAs = prev }()
 588
 589	relay := startRelay(t, &tls.Config{Certificates: ts.TLS.Certificates})
 590	cfg := mailCfg(relay.addr)
 591	cfg.Mail.TLS = "implicit"
 592	on := true
 593	cfg.Mail.RequireTLS = &on
 594	if err := Send(cfg, "a@example.test", "subject", "body"); err != nil {
 595		t.Fatal(err)
 596	}
 597	if n := relay.delivered(); n != 1 {
 598		t.Fatalf("delivered %d, want 1", n)
 599	}
 600}
 601```
 602
 603`httptest`'s certificate carries `127.0.0.1` as an IP SAN, so
 604verification against `ServerName: "127.0.0.1"` passes.
 605
 606- [ ] **Step 2: Run them and see them fail**
 607
 608Run: `go test ./internal/mail -count=1`
 609Expected: build failure, `undefined: rootCAs`.
 610
 611- [ ] **Step 3: Implement**
 612
 613`internal/mail/mail.go`:
 614
 615```go
 616// Package mail sends transactional email over SMTP: verification codes and
 617// invites. The connection is encrypted with STARTTLS, or with TLS from the
 618// first byte when mail.tls = "implicit"; a relay that offers neither gets
 619// nothing unless mail.require_tls is off. PLAIN auth when credentials are
 620// configured.
 621package mail
 622
 623import (
 624	"crypto/tls"
 625	"crypto/x509"
 626	"fmt"
 627	"net"
 628	"net/smtp"
 629	"strings"
 630	"time"
 631
 632	"gitbay.org/gitbay/internal/config"
 633)
 634
 635// rootCAs verifies the relay's certificate; nil is the system pool.
 636var rootCAs *x509.CertPool
 637
 638// Send delivers one plain-text message. cfg.Mail.SMTPHost is host:port.
 639func Send(cfg config.Config, to, subject, body string) error {
 640	m := cfg.Mail
 641	if m.SMTPHost == "" || m.From == "" {
 642		return fmt.Errorf("[mail] smtp_host and from must be configured")
 643	}
 644	implicit := m.TLS == "implicit"
 645	host := m.SMTPHost
 646	if !strings.Contains(host, ":") {
 647		if implicit {
 648			host += ":465"
 649		} else {
 650			host += ":587"
 651		}
 652	}
 653	hostname, _, _ := net.SplitHostPort(host)
 654	tlsCfg := &tls.Config{ServerName: hostname, RootCAs: rootCAs}
 655
 656	msg := strings.NewReplacer("\n", "\r\n").Replace(fmt.Sprintf(
 657		"From: %s\nTo: %s\nSubject: %s\nDate: %s\nMIME-Version: 1.0\nContent-Type: text/plain; charset=utf-8\n\n%s\n",
 658		m.From, to, subject, time.Now().Format(time.RFC1123Z), body))
 659
 660	c, err := dial(host, hostname, implicit, tlsCfg)
 661	if err != nil {
 662		return fmt.Errorf("smtp dial %s: %w", host, err)
 663	}
 664	defer c.Close()
 665	if !implicit {
 666		if ok, _ := c.Extension("STARTTLS"); ok {
 667			if err := c.StartTLS(tlsCfg); err != nil {
 668				return fmt.Errorf("starttls: %w", err)
 669			}
 670		} else if m.TLSRequired() {
 671			return fmt.Errorf("%s does not offer STARTTLS and mail.require_tls is on; not sending in clear", host)
 672		}
 673	}
 674	if m.SMTPUser != "" {
 675		if err := c.Auth(smtp.PlainAuth("", m.SMTPUser, m.SMTPPass, hostname)); err != nil {
 676			return fmt.Errorf("smtp auth: %w", err)
 677		}
 678	}
 679	if err := c.Mail(m.From); err != nil {
 680		return err
 681	}
 682	if err := c.Rcpt(to); err != nil {
 683		return err
 684	}
 685	w, err := c.Data()
 686	if err != nil {
 687		return err
 688	}
 689	if _, err := w.Write([]byte(msg)); err != nil {
 690		return err
 691	}
 692	if err := w.Close(); err != nil {
 693		return err
 694	}
 695	return c.Quit()
 696}
 697
 698// dial opens the SMTP session: plain TCP for STARTTLS, or TLS from the
 699// first byte.
 700func dial(addr, hostname string, implicit bool, tlsCfg *tls.Config) (*smtp.Client, error) {
 701	if !implicit {
 702		return smtp.Dial(addr)
 703	}
 704	conn, err := tls.Dial("tcp", addr, tlsCfg)
 705	if err != nil {
 706		return nil, err
 707	}
 708	c, err := smtp.NewClient(conn, hostname)
 709	if err != nil {
 710		conn.Close()
 711		return nil, err
 712	}
 713	return c, nil
 714}
 715```
 716
 717- [ ] **Step 4: Run the package**
 718
 719Run: `go test ./internal/mail ./internal/config -count=1 && go vet ./internal/mail`
 720Expected: PASS.
 721
 722- [ ] **Step 5: Docs**
 723
 724`Admin.org`, `** [mail]` becomes:
 725
 726```org
 727** [mail]
 728- =smtp_host= (host:port; 587 assumed, 465 with =tls = "implicit"=),
 729  =from=, optional =smtp_user= / =smtp_pass=. Required for invite/open
 730  registration and self-service =email add=; in closed mode you may omit
 731  it entirely and assert addresses by hand (below).
 732- =tls= — =starttls= (default) or =implicit= (TLS from the first byte,
 733  for relays on 465).
 734- =require_tls= — with =starttls=, a relay that does not offer STARTTLS
 735  gets no mail: delivery fails and retries, and the admin page's Mail
 736  table shows the error. Unset, it is on for every relay except
 737  =localhost= and loopback addresses; set =false= to allow plaintext
 738  to a remote relay.
 739```
 740
 741`Architecture/03-Deployment.org`, SMTP relay row:
 742
 743```org
 744| SMTP relay             | queued mail                   | STARTTLS required for a non-local relay, or implicit TLS | =mail.require_tls=; Go's =PlainAuth= will not send credentials over plaintext to a non-local host (=internal/mail/mail.go=) |
 745```
 746
 747`Architecture/06-Data-and-Cryptography.org`, SMTP row:
 748
 749```org
 750| SMTP                     | STARTTLS required unless the relay is local (=mail.require_tls=), or implicit TLS (=mail.tls=) |
 751```
 752
 753`Architecture/09-Controls.org`, "SMTP credentials protected in transit" row:
 754
 755```org
 756| SMTP credentials protected in transit       | in place | STARTTLS required for non-local relays, implicit TLS optional (=internal/mail/mail.go=) |
 757```
 758
 759`Architecture/10-Known-Gaps.org`: delete the `#280` row.
 760
 761- [ ] **Step 6: Commit, MR, merge**
 762
 763```bash
 764git add internal/mail .gitbay/wiki/Admin.org .gitbay/wiki/Architecture/03-Deployment.org \
 765  .gitbay/wiki/Architecture/06-Data-and-Cryptography.org \
 766  .gitbay/wiki/Architecture/09-Controls.org .gitbay/wiki/Architecture/10-Known-Gaps.org
 767git commit -S -m "mail: require TLS to a non-local relay; implicit TLS option
 768
 769Closes #280"
 770git push -u origin mail-require-tls
 771gitbay mr create --source mail-require-tls --target main --title "mail: require TLS to a non-local relay"
 772```
 773
 774Before merging, run the runbook's #280 check (bay1's relay must offer
 775STARTTLS or be local). Merge `--strategy ff` after CI, delete the branch
 776both places.
 777
 778---
 779
 780# MR 3: mirrors connect only to an address checked at sync time (branch `mirror-pin-address`, closes #279)
 781
 782### Task 3.1: `webhook.CheckAddrs`
 783
 784**Files:**
 785- Modify: `internal/webhook/webhook.go` (add after `isForbidden`, line 55)
 786- Create: `internal/webhook/webhook_test.go`
 787
 788**Interfaces:**
 789- Produces: `func CheckAddrs(host string, ips []net.IP, allowLocal bool) error`.
 790
 791- [ ] **Step 1: Write the failing test**
 792
 793```go
 794package webhook
 795
 796import (
 797	"net"
 798	"strings"
 799	"testing"
 800)
 801
 802func TestCheckAddrs(t *testing.T) {
 803	public := []net.IP{net.ParseIP("203.0.113.5")}
 804	mixed := []net.IP{net.ParseIP("203.0.113.5"), net.ParseIP("10.1.2.3")}
 805	if err := CheckAddrs("git.example", public, false); err != nil {
 806		t.Fatalf("public: %v", err)
 807	}
 808	if err := CheckAddrs("git.example", mixed, false); err == nil || !strings.Contains(err.Error(), "10.1.2.3") {
 809		t.Fatalf("mixed: %v", err)
 810	}
 811	if err := CheckAddrs("git.example", mixed, true); err != nil {
 812		t.Fatalf("allow_local: %v", err)
 813	}
 814}
 815```
 816
 817- [ ] **Step 2: Run it and see it fail**
 818
 819Run: `go test ./internal/webhook -run TestCheckAddrs -count=1`
 820Expected: `undefined: CheckAddrs`.
 821
 822- [ ] **Step 3: Implement**
 823
 824```go
 825// CheckAddrs refuses host when any of its resolved addresses is
 826// loopback, private or link-local, unless allowLocal. A caller resolves
 827// immediately before connecting and connects only to the addresses it
 828// checked.
 829func CheckAddrs(host string, ips []net.IP, allowLocal bool) error {
 830	if allowLocal {
 831		return nil
 832	}
 833	for _, ip := range ips {
 834		if isForbidden(ip) {
 835			return fmt.Errorf("%s resolves to private or local address %s; refusing (SSRF)", host, ip)
 836		}
 837	}
 838	return nil
 839}
 840```
 841
 842- [ ] **Step 4: Run it**
 843
 844Run: `go test ./internal/webhook -count=1`
 845Expected: PASS.
 846
 847- [ ] **Step 5: Commit**
 848
 849```bash
 850git add internal/webhook
 851git commit -S -m "webhook: CheckAddrs for callers that resolve before connecting
 852
 853Ref #279"
 854```
 855
 856### Task 3.2: mirror sync resolves, checks and pins
 857
 858**Files:**
 859- Modify: `internal/mirror/mirror.go:30-44` (`Worker`, `New`), `:76-116` (`sync`)
 860- Create: `internal/mirror/mirror_test.go`
 861
 862**Interfaces:**
 863- Consumes: `webhook.CheckAddrs` (Task 3.1).
 864- Produces: `Worker.Lookup func(ctx context.Context, host string) ([]net.IP, error)` (set by `New`); unexported `pinArgs(u *url.URL, ips []net.IP) []string`.
 865
 866- [ ] **Step 1: Write the failing tests**
 867
 868`internal/mirror/mirror_test.go`:
 869
 870```go
 871package mirror
 872
 873import (
 874	"context"
 875	"net"
 876	"net/http/cgi"
 877	"net/http/httptest"
 878	"net/url"
 879	"os"
 880	"os/exec"
 881	"path/filepath"
 882	"slices"
 883	"strings"
 884	"testing"
 885
 886	"gitbay.org/gitbay/internal/config"
 887	"gitbay.org/gitbay/internal/control"
 888	"gitbay.org/gitbay/internal/store"
 889)
 890
 891func git(t *testing.T, dir string, args ...string) string {
 892	t.Helper()
 893	cmd := exec.Command("git", append([]string{"-C", dir}, args...)...)
 894	cmd.Env = append(os.Environ(), "GIT_CONFIG_NOSYSTEM=1", "HOME="+t.TempDir(),
 895		"GIT_AUTHOR_NAME=t", "GIT_AUTHOR_EMAIL=t@example.test",
 896		"GIT_COMMITTER_NAME=t", "GIT_COMMITTER_EMAIL=t@example.test")
 897	out, err := cmd.CombinedOutput()
 898	if err != nil {
 899		t.Fatalf("git %v: %v\n%s", args, err, out)
 900	}
 901	return strings.TrimSpace(string(out))
 902}
 903
 904// upstream serves a bare repository with one commit on main over smart
 905// HTTP and returns its URL and that commit.
 906func upstream(t *testing.T) (string, string) {
 907	t.Helper()
 908	parent := t.TempDir()
 909	bare := filepath.Join(parent, "remote.git")
 910	work := filepath.Join(parent, "work")
 911	git(t, parent, "init", "-q", "--bare", "--initial-branch=main", bare)
 912	git(t, parent, "init", "-q", "--initial-branch=main", work)
 913	git(t, work, "commit", "-q", "--allow-empty", "-m", "one")
 914	git(t, work, "push", "-q", bare, "main")
 915	sha := git(t, work, "rev-parse", "HEAD")
 916	execPath := git(t, parent, "--exec-path")
 917	srv := httptest.NewServer(&cgi.Handler{
 918		Path: filepath.Join(execPath, "git-http-backend"),
 919		Env:  []string{"GIT_PROJECT_ROOT=" + parent, "GIT_HTTP_EXPORT_ALL=1"},
 920	})
 921	t.Cleanup(srv.Close)
 922	return srv.URL + "/remote.git", sha
 923}
 924
 925// local returns a store with alice/app, its bare repository under root,
 926// and the pull mirror row for url.
 927func local(t *testing.T, root, mirrorURL string) (*store.Store, store.Mirror, string) {
 928	t.Helper()
 929	st, err := store.Open(filepath.Join(t.TempDir(), "gitbay.db"))
 930	if err != nil {
 931		t.Fatal(err)
 932	}
 933	t.Cleanup(func() { st.Close() })
 934	if err := st.MigrateUp(); err != nil {
 935		t.Fatal(err)
 936	}
 937	uid, err := st.CreateUser("alice", false)
 938	if err != nil {
 939		t.Fatal(err)
 940	}
 941	repoID, err := st.CreateRepo("user", uid, "app", "public")
 942	if err != nil {
 943		t.Fatal(err)
 944	}
 945	dir := control.RepoDir(root, "alice", "app")
 946	os.MkdirAll(filepath.Dir(dir), 0o755)
 947	git(t, root, "init", "-q", "--bare", dir)
 948	if _, err := st.AddMirror(repoID, "pull", mirrorURL, "", ""); err != nil {
 949		t.Fatal(err)
 950	}
 951	due, err := st.DueMirrors(900)
 952	if err != nil || len(due) != 1 {
 953		t.Fatalf("due mirrors: %v %v", due, err)
 954	}
 955	return st, due[0], dir
 956}
 957
 958// mirror.test does not resolve; the fetch works only because git was
 959// pinned to the address the worker looked up and checked.
 960func TestSyncConnectsToTheCheckedAddress(t *testing.T) {
 961	remote, sha := upstream(t)
 962	u, _ := url.Parse(remote)
 963	root := t.TempDir()
 964	st, m, dir := local(t, root, "http://mirror.test:"+u.Port()+"/remote.git")
 965	var cfg config.Config
 966	cfg.Server.Root = root
 967	cfg.Webhooks.AllowLocal = true
 968	var asked []string
 969	w := &Worker{St: st, Cfg: cfg, Lookup: func(ctx context.Context, host string) ([]net.IP, error) {
 970		asked = append(asked, host)
 971		return []net.IP{net.ParseIP("127.0.0.1")}, nil
 972	}}
 973	if err := w.sync(m); err != nil {
 974		t.Fatal(err)
 975	}
 976	if got := git(t, dir, "rev-parse", "refs/heads/main"); got != sha {
 977		t.Fatalf("main = %s, want %s", got, sha)
 978	}
 979	if !slices.Equal(asked, []string{"mirror.test"}) {
 980		t.Fatalf("looked up %v", asked)
 981	}
 982}
 983
 984// The URL passed the check when it was saved; the answer at sync time
 985// is what counts.
 986func TestSyncRefusesAPrivateAddressAtSyncTime(t *testing.T) {
 987	root := t.TempDir()
 988	st, m, _ := local(t, root, "https://mirror.test/x.git")
 989	var cfg config.Config
 990	cfg.Server.Root = root
 991	w := &Worker{St: st, Cfg: cfg, Lookup: func(context.Context, string) ([]net.IP, error) {
 992		return []net.IP{net.ParseIP("10.0.0.7")}, nil
 993	}}
 994	err := w.sync(m)
 995	if err == nil || !strings.Contains(err.Error(), "10.0.0.7") {
 996		t.Fatalf("sync = %v, want a refusal naming 10.0.0.7", err)
 997	}
 998}
 999
1000func TestPinArgs(t *testing.T) {
1001	u, _ := url.Parse("https://git.example/x.git")
1002	got := pinArgs(u, []net.IP{net.ParseIP("203.0.113.5"), net.ParseIP("2001:db8::1")})
1003	want := []string{"-c", "http.followRedirects=false",
1004		"-c", "http.curloptResolve=git.example:443:203.0.113.5,[2001:db8::1]"}
1005	if !slices.Equal(got, want) {
1006		t.Fatalf("https: %q", got)
1007	}
1008	u, _ = url.Parse("http://git.example:8080/x.git")
1009	if got := pinArgs(u, []net.IP{net.ParseIP("203.0.113.5")}); got[3] != "http.curloptResolve=git.example:8080:203.0.113.5" {
1010		t.Fatalf("http with port: %q", got)
1011	}
1012	// An address literal is its own resolution; there is nothing to pin.
1013	u, _ = url.Parse("https://203.0.113.5/x.git")
1014	if got := pinArgs(u, []net.IP{net.ParseIP("203.0.113.5")}); !slices.Equal(got, []string{"-c", "http.followRedirects=false"}) {
1015		t.Fatalf("literal: %q", got)
1016	}
1017}
1018```
1019
1020- [ ] **Step 2: Run them and see them fail**
1021
1022Run: `go test ./internal/mirror -count=1`
1023Expected: build failure, `unknown field Lookup` / `undefined: pinArgs`.
1024
1025- [ ] **Step 3: Implement**
1026
1027`Worker` and `New`:
1028
1029```go
1030type Worker struct {
1031	St   *store.Store
1032	Cfg  config.Config
1033	Tick time.Duration
1034	// Lookup resolves a mirror's host immediately before each sync.
1035	Lookup func(ctx context.Context, host string) ([]net.IP, error)
1036}
1037
1038func New(st *store.Store, cfg config.Config) *Worker {
1039	tick := 10 * time.Second
1040	if v := os.Getenv("GITBAY_MIRROR_TICK"); v != "" {
1041		if d, err := time.ParseDuration(v); err == nil {
1042			tick = d
1043		}
1044	}
1045	return &Worker{St: st, Cfg: cfg, Tick: tick,
1046		Lookup: func(ctx context.Context, host string) ([]net.IP, error) {
1047			return net.DefaultResolver.LookupIP(ctx, "ip", host)
1048		}}
1049}
1050```
1051
1052`sync` from the top through the argv; the askpass block is unchanged
1053and the timeout context moves above the lookup so the lookup shares it:
1054
1055```go
1056func (w *Worker) sync(m store.Mirror) error {
1057	repo, err := w.St.RepoByID(m.RepoID)
1058	if err != nil {
1059		return err
1060	}
1061	dir := control.RepoDir(w.Cfg.Server.Root, repo.OwnerName, repo.Name)
1062	u, err := url.Parse(m.URL)
1063	if err != nil {
1064		return err
1065	}
1066	if u.Scheme != "https" && u.Scheme != "http" {
1067		return fmt.Errorf("mirror URL scheme %q is not http or https", u.Scheme)
1068	}
1069
1070	ctx, cancel := context.WithTimeout(context.Background(), 10*time.Minute)
1071	defer cancel()
1072	// The URL was checked when saved, but DNS can answer differently
1073	// now. Check what it resolves to at sync time, then let git connect
1074	// to exactly those addresses.
1075	ips, err := w.Lookup(ctx, u.Hostname())
1076	if err != nil {
1077		return fmt.Errorf("resolving %s: %w", u.Hostname(), err)
1078	}
1079	if err := webhook.CheckAddrs(u.Hostname(), ips, w.Cfg.Webhooks.AllowLocal); err != nil {
1080		return err
1081	}
1082
1083	env := []string{"GIT_TERMINAL_PROMPT=0", "HOME=" + w.Cfg.Server.Root}
1084	if m.Token != "" {
1085		// (askpass block unchanged)
1086	}
1087
1088	args := append(pinArgs(u, ips), "-C", dir)
1089	if m.Direction == "push" {
1090		// Branches and tags only: internal refs (merge-requests) stay home.
1091		args = append(args, "push", "--prune", m.URL,
1092			"+refs/heads/*:refs/heads/*", "+refs/tags/*:refs/tags/*")
1093	} else {
1094		args = append(args, "fetch", "--prune", m.URL,
1095			"+refs/heads/*:refs/heads/*", "+refs/tags/*:refs/tags/*")
1096	}
1097	cmd := exec.CommandContext(ctx, toolpath.Look("git"), args...)
1098	cmd.Env = env
1099	if out, err := cmd.CombinedOutput(); err != nil {
1100		return fmt.Errorf("git %s: %v: %.300s", m.Direction, err, out)
1101	}
1102	return nil
1103}
1104
1105// pinArgs keeps git on the addresses just checked: curl's resolve list
1106// pins the host, and with redirects off a server cannot send git on to
1107// a host nobody checked. An address literal needs no pin.
1108func pinArgs(u *url.URL, ips []net.IP) []string {
1109	args := []string{"-c", "http.followRedirects=false"}
1110	host := u.Hostname()
1111	if net.ParseIP(host) != nil {
1112		return args
1113	}
1114	port := u.Port()
1115	if port == "" {
1116		port = "443"
1117		if u.Scheme == "http" {
1118			port = "80"
1119		}
1120	}
1121	addrs := make([]string, len(ips))
1122	for i, ip := range ips {
1123		if ip.To4() == nil {
1124			addrs[i] = "[" + ip.String() + "]"
1125		} else {
1126			addrs[i] = ip.String()
1127		}
1128	}
1129	return append(args, "-c", "http.curloptResolve="+host+":"+port+":"+strings.Join(addrs, ","))
1130}
1131```
1132
1133Imports gain `net`, `net/url`, `strings`, and
1134`gitbay.org/gitbay/internal/webhook`. `mirror` does not import
1135`webhook` today; `webhook` imports only `store`, so there is no cycle.
1136The `// (askpass block unchanged)` line stands for lines 84-97 kept as
1137they are; do not type it literally.
1138
1139- [ ] **Step 4: Run the package and the existing mirror e2e**
1140
1141Run: `go test ./internal/mirror ./internal/webhook -count=1 && go vet ./internal/mirror`
1142Expected: PASS.
1143
1144Run: `go test ./e2e -run TestMirrors -count=1`
1145Expected: PASS (its URLs are `http://127.0.0.1:<port>/…`, address
1146literals, so they take the no-pin branch; redirects are not used).
1147
1148- [ ] **Step 5: Docs**
1149
1150`Admin.org`, `** [mirrors]` last line becomes:
1151
1152```org
1153  Mirror URLs pass the same SSRF rules as webhook targets, when saved
1154  and again before every sync; git then connects only to the addresses
1155  that were checked (=http.curloptResolve=) and does not follow
1156  redirects, so a mirror of a renamed repository fails until its URL
1157  is updated. Needs git 2.37 or later on the server.
1158```
1159
1160`Threat-Model.org`, the paragraph under `* Network-facing request forgery`:
1161
1162```org
1163Anything that makes the *server* open an outbound connection to a
1164user-supplied address — webhook delivery, GitHub-history import
1165=--api-base=, mirror remotes — passes the same SSRF guard: the scheme
1166must be http/https and, unless =webhooks.allow_local= is set, the
1167resolved address must not be loopback, private, or link-local. The
1168webhook dialer re-checks at connect time, and the mirror worker
1169resolves and checks before each sync and pins git to the checked
1170addresses, so a DNS answer that changes after validation still cannot
1171reach private space. Redirects are never followed.
1172```
1173
1174`Architecture/03-Deployment.org`, Mirror URLs row:
1175
1176```org
1177| Mirror URLs            | mirror schedule               | per URL                                      | address check at save and before each sync; git pinned to the checked addresses, no redirects (=internal/mirror/mirror.go=) |
1178```
1179
1180`Architecture/09-Controls.org`, SSRF row:
1181
1182```org
1183| SSRF protection on user-supplied URLs       | in place | webhooks at save and connect; mirrors at save and sync, git pinned to the checked address (=internal/mirror/mirror.go=) |
1184```
1185
1186`Architecture/10-Known-Gaps.org`: delete the `#279` row.
1187
1188- [ ] **Step 6: Commit, MR, merge**
1189
1190```bash
1191git add internal/mirror .gitbay/wiki/Admin.org .gitbay/wiki/Threat-Model.org \
1192  .gitbay/wiki/Architecture/03-Deployment.org .gitbay/wiki/Architecture/09-Controls.org \
1193  .gitbay/wiki/Architecture/10-Known-Gaps.org
1194git commit -S -m "mirror: check the address before each sync and pin git to it
1195
1196Closes #279"
1197git push -u origin mirror-pin-address
1198gitbay mr create --source mirror-pin-address --target main --title "mirror: check the address before each sync and pin git to it"
1199```
1200
1201Before merging, the runbook's #279 check (git ≥ 2.37 on bay1). Merge
1202`--strategy ff` after CI, delete the branch both places.
1203
1204---
1205
1206# MR 4: authenticated hook socket (branch `hook-socket-auth`, closes #282)
1207
1208### Task 4.1: `push_tokens` table and store methods
1209
1210**Files:**
1211- Create: `internal/store/migrations/0069_push_tokens.up.sql`, `0069_push_tokens.down.sql`
1212- Create: `internal/store/pushtokens.go`, `internal/store/pushtokens_test.go`
1213- Modify: `internal/store/retention.go:47-54` (`expired` list)
1214
1215**Interfaces:**
1216- Produces:
1217  - `type PushToken struct { RepoID, UserID int64; Scope string }`
1218  - `func (s *Store) CreatePushToken(repoID, userID int64, scope string) (string, error)` — returns the raw token; stores `HashToken(token)`; expires in 24h.
1219  - `func (s *Store) PushTokenByHash(hash string) (PushToken, error)` — `ErrNotFound` when absent or expired.
1220  - `func (s *Store) DeletePushToken(token string) error` — takes the raw token.
1221
1222- [ ] **Step 1: Write the failing test**
1223
1224`internal/store/pushtokens_test.go`:
1225
1226```go
1227package store
1228
1229import (
1230	"errors"
1231	"testing"
1232	"time"
1233)
1234
1235func TestPushTokens(t *testing.T) {
1236	s := open(t)
1237	if err := s.MigrateUp(); err != nil {
1238		t.Fatal(err)
1239	}
1240	uid, err := s.CreateUser("alice", false)
1241	if err != nil {
1242		t.Fatal(err)
1243	}
1244	repoID, err := s.CreateRepo("user", uid, "app", "public")
1245	if err != nil {
1246		t.Fatal(err)
1247	}
1248	token, err := s.CreatePushToken(repoID, uid, "full")
1249	if err != nil {
1250		t.Fatal(err)
1251	}
1252	got, err := s.PushTokenByHash(HashToken(token))
1253	if err != nil || got != (PushToken{RepoID: repoID, UserID: uid, Scope: "full"}) {
1254		t.Fatalf("lookup = %+v, %v", got, err)
1255	}
1256	if err := s.DeletePushToken(token); err != nil {
1257		t.Fatal(err)
1258	}
1259	if _, err := s.PushTokenByHash(HashToken(token)); !errors.Is(err, ErrNotFound) {
1260		t.Fatalf("after delete: %v", err)
1261	}
1262
1263	// A token whose receive-pack never cleaned up is swept after a day.
1264	stale, err := s.CreatePushToken(repoID, uid, "full")
1265	if err != nil {
1266		t.Fatal(err)
1267	}
1268	swept, err := s.Sweep(Retention{}, time.Now().Add(25*time.Hour))
1269	if err != nil || swept["push_tokens"] != 1 {
1270		t.Fatalf("sweep = %v, %v", swept, err)
1271	}
1272	if _, err := s.PushTokenByHash(HashToken(stale)); !errors.Is(err, ErrNotFound) {
1273		t.Fatalf("after sweep: %v", err)
1274	}
1275}
1276```
1277
1278- [ ] **Step 2: Run it and see it fail**
1279
1280Run: `go test ./internal/store -run TestPushTokens -count=1`
1281Expected: build failure, `s.CreatePushToken undefined`.
1282
1283- [ ] **Step 3: Implement**
1284
1285`0069_push_tokens.up.sql`:
1286
1287```sql
1288-- One row per receive-pack in flight. The hook names its push by the
1289-- token; hookd answers only a live one. Only the SHA-256 is stored.
1290CREATE TABLE push_tokens (
1291    token_hash TEXT PRIMARY KEY,
1292    repo_id    INTEGER NOT NULL REFERENCES repos(id) ON DELETE CASCADE,
1293    user_id    INTEGER NOT NULL REFERENCES users(id) ON DELETE CASCADE,
1294    scope      TEXT NOT NULL,
1295    created_at TEXT NOT NULL DEFAULT (strftime('%Y-%m-%dT%H:%M:%fZ','now')),
1296    expires_at TEXT NOT NULL
1297);
1298```
1299
1300`0069_push_tokens.down.sql`:
1301
1302```sql
1303DROP TABLE push_tokens;
1304```
1305
1306`internal/store/pushtokens.go`:
1307
1308```go
1309package store
1310
1311import (
1312	"database/sql"
1313	"errors"
1314	"time"
1315)
1316
1317// PushToken is the receive-pack a hook request speaks for.
1318type PushToken struct {
1319	RepoID int64
1320	UserID int64
1321	Scope  string
1322}
1323
1324// pushTokenTTL bounds a row whose receive-pack died before deleting it.
1325const pushTokenTTL = 24 * time.Hour
1326
1327// CreatePushToken records a token for one receive-pack and returns it.
1328func (s *Store) CreatePushToken(repoID, userID int64, scope string) (string, error) {
1329	token, hash, err := NewToken()
1330	if err != nil {
1331		return "", err
1332	}
1333	_, err = s.DB.Exec(
1334		"INSERT INTO push_tokens (token_hash, repo_id, user_id, scope, expires_at) VALUES (?, ?, ?, ?, ?)",
1335		hash, repoID, userID, scope, fmtTime(time.Now().Add(pushTokenTTL)))
1336	if err != nil {
1337		return "", err
1338	}
1339	return token, nil
1340}
1341
1342func (s *Store) PushTokenByHash(hash string) (PushToken, error) {
1343	var t PushToken
1344	err := s.DB.QueryRow(
1345		"SELECT repo_id, user_id, scope FROM push_tokens WHERE token_hash = ? AND expires_at > ?",
1346		hash, fmtTime(time.Now())).Scan(&t.RepoID, &t.UserID, &t.Scope)
1347	if errors.Is(err, sql.ErrNoRows) {
1348		return PushToken{}, ErrNotFound
1349	}
1350	return t, err
1351}
1352
1353func (s *Store) DeletePushToken(token string) error {
1354	_, err := s.DB.Exec("DELETE FROM push_tokens WHERE token_hash = ?", HashToken(token))
1355	return err
1356}
1357```
1358
1359`internal/store/retention.go`, the `expired` list gains a row:
1360
1361```go
1362		{"web_sessions", "expires_at <= ?"},
1363		{"login_tokens", "expires_at <= ?"},
1364		{"email_tokens", "expires_at <= ?"},
1365		{"push_tokens", "expires_at <= ?"},
1366```
1367
1368- [ ] **Step 4: Run the package**
1369
1370Run: `go test ./internal/store -count=1`
1371Expected: PASS, including `TestMigrateUpDown`.
1372
1373- [ ] **Step 5: Commit**
1374
1375```bash
1376git add internal/store
1377git commit -S -m "store: push tokens for receive-pack
1378
1379Ref #282"
1380```
1381
1382### Task 4.2: hookd requires 0600, the daemon's uid, and a live push token
1383
1384**Files:**
1385- Modify: `internal/hookd/hookd.go:34-50` (constants, `Request`), `:90-128` (`Serve`, `handle`)
1386- Create: `internal/hookd/peercred_linux.go`, `internal/hookd/peercred_other.go`
1387- Create: `internal/hookd/socket_test.go`
1388
1389**Interfaces:**
1390- Consumes: `store.CreatePushToken`, `store.PushTokenByHash`, `store.HashToken` (Task 4.1).
1391- Produces: `hookd.EnvToken = "GITBAY_PUSH_TOKEN"`; `Request.Token string` (`json:"token"`); unexported `checkPeer(net.Conn) error`.
1392
1393- [ ] **Step 1: Write the failing tests**
1394
1395`internal/hookd/socket_test.go`:
1396
1397```go
1398package hookd
1399
1400import (
1401	"os"
1402	"path/filepath"
1403	"strings"
1404	"testing"
1405
1406	"gitbay.org/gitbay/internal/config"
1407	"gitbay.org/gitbay/internal/store"
1408)
1409
1410func serveSocket(t *testing.T) (sock string, st *store.Store, repoID, uid int64) {
1411	t.Helper()
1412	st, err := store.Open(filepath.Join(t.TempDir(), "gitbay.db"))
1413	if err != nil {
1414		t.Fatal(err)
1415	}
1416	t.Cleanup(func() { st.Close() })
1417	if err := st.MigrateUp(); err != nil {
1418		t.Fatal(err)
1419	}
1420	if uid, err = st.CreateUser("alice", false); err != nil {
1421		t.Fatal(err)
1422	}
1423	if repoID, err = st.CreateRepo("user", uid, "app", "public"); err != nil {
1424		t.Fatal(err)
1425	}
1426	var cfg config.Config
1427	cfg.Server.Root = t.TempDir()
1428	stop, err := Serve(cfg, st)
1429	if err != nil {
1430		t.Fatal(err)
1431	}
1432	t.Cleanup(func() { stop() })
1433	return SocketPath(cfg.Server.Root), st, repoID, uid
1434}
1435
1436func TestSocketIsOwnerOnly(t *testing.T) {
1437	sock, _, _, _ := serveSocket(t)
1438	fi, err := os.Stat(sock)
1439	if err != nil {
1440		t.Fatal(err)
1441	}
1442	if fi.Mode().Perm() != 0o600 {
1443		t.Fatalf("mode %v, want 0600", fi.Mode().Perm())
1444	}
1445}
1446
1447// A request speaks for a receive-pack sshd started, and only for the
1448// repository, account and scope that push was started with (#282).
1449func TestHookRequestNeedsItsPushToken(t *testing.T) {
1450	sock, st, repoID, uid := serveSocket(t)
1451	req := Request{Hook: "pre-receive", RepoID: repoID, UserID: uid, Scope: "full"}
1452
1453	resp, err := Ask(sock, req, nil)
1454	if err != nil {
1455		t.Fatal(err)
1456	}
1457	if resp.Allow || !strings.Contains(resp.Message, "not started by this server") {
1458		t.Fatalf("no token: %+v", resp)
1459	}
1460
1461	token, err := st.CreatePushToken(repoID, uid, "full")
1462	if err != nil {
1463		t.Fatal(err)
1464	}
1465	req.Token = token
1466	if resp, err = Ask(sock, req, nil); err != nil || !resp.Allow {
1467		t.Fatalf("with token: %+v, %v", resp, err)
1468	}
1469
1470	other, err := st.CreateUser("mallory", false)
1471	if err != nil {
1472		t.Fatal(err)
1473	}
1474	forged := req
1475	forged.UserID = other
1476	if resp, err = Ask(sock, forged, nil); err != nil || resp.Allow {
1477		t.Fatalf("token for another account: %+v, %v", resp, err)
1478	}
1479
1480	if err := st.DeletePushToken(token); err != nil {
1481		t.Fatal(err)
1482	}
1483	if resp, err = Ask(sock, req, nil); err != nil || resp.Allow {
1484		t.Fatalf("finished push: %+v, %v", resp, err)
1485	}
1486}
1487```
1488
1489The peer-uid check is exercised by the same test on Linux (CI on
1490bay1): the test process is the daemon's uid, so a refusal there fails
1491the "with token" case.
1492
1493- [ ] **Step 2: Run them and see them fail**
1494
1495Run: `go test ./internal/hookd -run 'TestSocketIsOwnerOnly|TestHookRequestNeedsItsPushToken' -count=1`
1496Expected: build failure, `unknown field Token in struct literal`.
1497
1498- [ ] **Step 3: Implement**
1499
1500`internal/hookd/hookd.go`, constants and `Request`:
1501
1502```go
1503const (
1504	EnvSocket = "GITBAY_HOOK_SOCKET"
1505	EnvRepoID = "GITBAY_REPO_ID"
1506	EnvUserID = "GITBAY_USER_ID"
1507	EnvScope  = "GITBAY_KEY_SCOPE"
1508	// EnvToken names the receive-pack this hook runs under. sshd mints
1509	// it per push; hookd answers only a request carrying a live one
1510	// whose repository, account and scope match the request's.
1511	EnvToken = "GITBAY_PUSH_TOKEN"
1512)
1513
1514type Request struct {
1515	Hook   string `json:"hook"` // pre-receive | post-receive
1516	RepoID int64  `json:"repo_id"`
1517	UserID int64  `json:"user_id"`
1518	// Scope is the pushing key's scope. The user id alone is the account
1519	// the key belongs to, and a deploy key grants nothing outside its
1520	// binding, so anything acting on another repository needs this too.
1521	Scope   string             `json:"scope"`
1522	Token   string             `json:"token"`
1523	Updates []policy.RefUpdate `json:"updates"`
1524}
1525```
1526
1527`Serve`, after `net.Listen`:
1528
1529```go
1530	ln, err := net.Listen("unix", path)
1531	if err != nil {
1532		return nil, err
1533	}
1534	// Listen creates the socket under the process umask. Hooks run as
1535	// the daemon's own user; nobody else has a reason to connect.
1536	if err := os.Chmod(path, 0o600); err != nil {
1537		ln.Close()
1538		return nil, err
1539	}
1540```
1541
1542`handle`:
1543
1544```go
1545func (s *Server) handle(conn net.Conn) {
1546	defer conn.Close()
1547	dec := json.NewDecoder(conn)
1548	enc := json.NewEncoder(conn)
1549	if err := checkPeer(conn); err != nil {
1550		slog.Warn("hook socket: refused connection", "err", err)
1551		enc.Encode(Response{Allow: false, Message: "hook socket: " + err.Error()})
1552		return
1553	}
1554	var req Request
1555	if err := dec.Decode(&req); err != nil {
1556		enc.Encode(Response{Allow: false, Message: "bad hook request"})
1557		return
1558	}
1559	if msg := s.authorize(req); msg != "" {
1560		enc.Encode(Response{Allow: false, Message: msg})
1561		return
1562	}
1563	switch req.Hook {
1564	case "pre-receive":
1565		s.preReceive(req, dec, enc)
1566	case "post-receive":
1567		s.postReceive(req)
1568		enc.Encode(Response{Allow: true})
1569	default:
1570		enc.Encode(Response{Allow: false, Message: fmt.Sprintf("unknown hook %q", req.Hook)})
1571	}
1572}
1573
1574// authorize ties a request to a receive-pack sshd started: its token
1575// must be live and name the same repository, account and key scope.
1576func (s *Server) authorize(req Request) string {
1577	if req.Token == "" {
1578		return "push not started by this server"
1579	}
1580	tok, err := s.st.PushTokenByHash(store.HashToken(req.Token))
1581	if err != nil {
1582		return "push not started by this server"
1583	}
1584	if tok.RepoID != req.RepoID || tok.UserID != req.UserID || tok.Scope != req.Scope {
1585		return "push token does not match this request"
1586	}
1587	return ""
1588}
1589```
1590
1591`internal/hookd/peercred_linux.go`:
1592
1593```go
1594//go:build linux
1595
1596package hookd
1597
1598import (
1599	"fmt"
1600	"net"
1601	"os"
1602	"syscall"
1603)
1604
1605// checkPeer refuses a connection from any uid but the daemon's: git,
1606// and so every hook, runs as the daemon's user.
1607func checkPeer(conn net.Conn) error {
1608	uc, ok := conn.(*net.UnixConn)
1609	if !ok {
1610		return fmt.Errorf("not a unix socket connection")
1611	}
1612	raw, err := uc.SyscallConn()
1613	if err != nil {
1614		return err
1615	}
1616	var cred *syscall.Ucred
1617	var credErr error
1618	if err := raw.Control(func(fd uintptr) {
1619		cred, credErr = syscall.GetsockoptUcred(int(fd), syscall.SOL_SOCKET, syscall.SO_PEERCRED)
1620	}); err != nil {
1621		return err
1622	}
1623	if credErr != nil {
1624		return credErr
1625	}
1626	if int(cred.Uid) != os.Getuid() {
1627		return fmt.Errorf("peer uid %d is not the daemon's (%d)", cred.Uid, os.Getuid())
1628	}
1629	return nil
1630}
1631```
1632
1633`internal/hookd/peercred_other.go`:
1634
1635```go
1636//go:build !linux
1637
1638package hookd
1639
1640import "net"
1641
1642// checkPeer reads peer credentials on Linux only; elsewhere the
1643// socket's 0600 mode is the boundary.
1644func checkPeer(net.Conn) error { return nil }
1645```
1646
1647- [ ] **Step 4: Run the package, and vet for Linux**
1648
1649Run: `go test ./internal/hookd -count=1 && GOOS=linux go vet ./internal/hookd`
1650Expected: PASS; vet clean for both build-tag files.
1651
1652- [ ] **Step 5: Commit**
1653
1654```bash
1655git add internal/hookd
1656git commit -S -m "hookd: 0600 socket, peer uid check, push token required
1657
1658Ref #282"
1659```
1660
1661### Task 4.3: sshd mints the token; the hook sends it
1662
1663**Files:**
1664- Modify: `internal/sshd/sshd.go:441-464` (`runGit`, env and transport)
1665- Modify: `cmd/gitbayd/hook.go:170-178` (`hookd.Ask` request)
1666
1667**Interfaces:**
1668- Consumes: `store.CreatePushToken`, `store.DeletePushToken`, `hookd.EnvToken`, `hookd.Request.Token`.
1669
1670- [ ] **Step 1: Implement in sshd**
1671
1672In `runGit`, after the quota block (line 463) and before
1673`gitutil.Transport`:
1674
1675```go
1676	if write {
1677		// hookd answers only a hook that names this receive-pack.
1678		token, err := st.CreatePushToken(repo.ID, user.ID, scope)
1679		if err != nil {
1680			fmt.Fprintln(stderr, "internal error")
1681			return protocol.ExitFailure
1682		}
1683		defer st.DeletePushToken(token)
1684		env = append(env, hookd.EnvToken+"="+token)
1685	}
1686```
1687
1688The token is deleted when `Transport` returns, which is after
1689post-receive: receive-pack runs post-receive before it exits.
1690
1691- [ ] **Step 2: Implement in the hook**
1692
1693`cmd/gitbayd/hook.go`, the request becomes:
1694
1695```go
1696			resp, err := hookd.Ask(sock, hookd.Request{
1697				Hook:    args[0],
1698				RepoID:  repoID,
1699				UserID:  userID,
1700				Scope:   os.Getenv(hookd.EnvScope),
1701				Token:   os.Getenv(hookd.EnvToken),
1702				Updates: updates,
1703			}, func(emit func(hookd.RawCommit) error) error {
1704				return streamIncomingCommits(updates, emit)
1705			})
1706```
1707
1708- [ ] **Step 3: Build, vet, unit tests; one push e2e**
1709
1710Run: `go build ./... && go vet ./... && go test ./internal/sshd ./internal/hookd ./cmd/gitbayd -count=1`
1711Expected: PASS.
1712
1713Run: `go test ./e2e -run TestAuditAndHardening -count=1`
1714Expected: PASS. It pushes over SSH (including an oversized push
1715refused by receive-pack), so pre-receive and post-receive both go
1716through the token check end to end. This is the one e2e run for this
1717MR; CI runs the rest of the push tests.
1718
1719- [ ] **Step 4: Docs**
1720
1721`Architecture/03-Deployment.org`, hook.sock row:
1722
1723```org
1724| =<root>/hook.sock=   | Unix socket       | gitbayd    | on          | mode 0600; peer uid must be the daemon's (Linux); per-push token | =internal/hookd/hookd.go= |
1725```
1726
1727`Architecture/04-Trust-Boundaries.org`, TB5 row:
1728
1729```org
1730| TB5 | Z3 → Z1 hook socket                | ref updates, repository id, user id, key scope, push token, commit objects | the socket is mode 0600 and, on Linux, refuses a peer whose uid is not the daemon's; a request must carry the token sshd minted for its receive-pack (stored hashed in =push_tokens=) and name the same repository, account and scope. The daemon then decides with =policy.CheckPush= and =sig.VerifyCommit= (=internal/hookd/hookd.go=) |
1731```
1732
1733`Architecture/04-Trust-Boundaries.org`, step 2 of the push sequence
1734(line 57): append ", and a push token" after "key scope" in the list
1735of what `git receive-pack` runs with.
1736
1737`Architecture/10-Known-Gaps.org`: delete the `#282` row.
1738
1739- [ ] **Step 5: Commit, MR, merge**
1740
1741```bash
1742git add internal/sshd/sshd.go cmd/gitbayd/hook.go .gitbay/wiki/Architecture
1743git commit -S -m "sshd: mint a push token per receive-pack; hook sends it
1744
1745Closes #282"
1746git push -u origin hook-socket-auth
1747gitbay mr create --source hook-socket-auth --target main --title "hookd: authenticate the hook socket"
1748```
1749
1750Merge `--strategy ff` after CI, delete the branch both places. Deploy
1751with no push in flight (see runbook): a receive-pack started by the old
1752daemon has no token and its post-receive is refused by the new one.
1753
1754---
1755
1756# MR 5: audited refusals and a hash-chained audit log (branch `audit-refusals-chain`, closes #275)
1757
1758### Task 5.1: chained audit rows, the journal line, verification
1759
1760**Files:**
1761- Create: `internal/store/migrations/0070_audit_chain.up.sql`, `0070_audit_chain.down.sql`
1762- Modify: `internal/store/audit.go:1-19` (`Audit`)
1763- Modify: `internal/store/store.go:23-30` (`Store` gains `AuditJournal`)
1764- Create: `internal/store/auditchain_test.go`
1765
1766**Interfaces:**
1767- Produces:
1768  - `Store.AuditJournal *slog.Logger` — when set, every audit row is also logged there.
1769  - `type AuditChain struct { Rows, Unchained int; First, Last int64; LastHash string; BrokenAt int64; Reason string }`
1770  - `func (s *Store) VerifyAuditChain() (AuditChain, error)`
1771  - `Audit` signature unchanged.
1772
1773- [ ] **Step 1: Write the failing tests**
1774
1775`internal/store/auditchain_test.go`:
1776
1777```go
1778package store
1779
1780import (
1781	"bytes"
1782	"log/slog"
1783	"strings"
1784	"testing"
1785)
1786
1787func chainStore(t *testing.T) *Store {
1788	t.Helper()
1789	s := open(t)
1790	if err := s.MigrateUp(); err != nil {
1791		t.Fatal(err)
1792	}
1793	return s
1794}
1795
1796func TestAuditChainIntact(t *testing.T) {
1797	s := chainStore(t)
1798	s.Audit(0, "a", map[string]any{"n": 1})
1799	s.Audit(0, "b", nil)
1800	s.Audit(0, "c", map[string]any{"n": 3})
1801	res, err := s.VerifyAuditChain()
1802	if err != nil {
1803		t.Fatal(err)
1804	}
1805	if res.Rows != 3 || res.BrokenAt != 0 || res.First != 1 || res.Last != 3 || len(res.LastHash) != 64 {
1806		t.Fatalf("%+v", res)
1807	}
1808}
1809
1810func TestAuditChainDetectsAnEditedRow(t *testing.T) {
1811	s := chainStore(t)
1812	for _, a := range []string{"a", "b", "c"} {
1813		s.Audit(0, a, nil)
1814	}
1815	if _, err := s.DB.Exec("UPDATE audit_log SET action = 'x' WHERE id = 2"); err != nil {
1816		t.Fatal(err)
1817	}
1818	res, err := s.VerifyAuditChain()
1819	if err != nil {
1820		t.Fatal(err)
1821	}
1822	if res.BrokenAt != 2 || !strings.Contains(res.Reason, "contents") {
1823		t.Fatalf("%+v", res)
1824	}
1825}
1826
1827func TestAuditChainDetectsARemovedRow(t *testing.T) {
1828	s := chainStore(t)
1829	for _, a := range []string{"a", "b", "c"} {
1830		s.Audit(0, a, nil)
1831	}
1832	if _, err := s.DB.Exec("DELETE FROM audit_log WHERE id = 2"); err != nil {
1833		t.Fatal(err)
1834	}
1835	res, err := s.VerifyAuditChain()
1836	if err != nil {
1837		t.Fatal(err)
1838	}
1839	if res.BrokenAt != 3 || !strings.Contains(res.Reason, "previous hash") {
1840		t.Fatalf("%+v", res)
1841	}
1842}
1843
1844// Retention removes the oldest rows, and deleting an account nulls
1845// actor_id; neither is tampering.
1846func TestAuditChainSurvivesRetentionAndAccountDeletion(t *testing.T) {
1847	s := chainStore(t)
1848	uid, err := s.CreateUser("alice", false)
1849	if err != nil {
1850		t.Fatal(err)
1851	}
1852	s.Audit(0, "a", nil)
1853	s.Audit(uid, "b", nil)
1854	s.Audit(0, "c", nil)
1855	if _, err := s.DB.Exec("DELETE FROM audit_log WHERE id = 1"); err != nil {
1856		t.Fatal(err)
1857	}
1858	if _, err := s.DB.Exec("DELETE FROM users WHERE id = ?", uid); err != nil {
1859		t.Fatal(err)
1860	}
1861	res, err := s.VerifyAuditChain()
1862	if err != nil {
1863		t.Fatal(err)
1864	}
1865	if res.BrokenAt != 0 || res.First != 2 || res.Last != 3 {
1866		t.Fatalf("%+v", res)
1867	}
1868}
1869
1870// Rows written before migration 0070 carry no hash; the chain starts
1871// after them, and a hashless row after that start is a break.
1872func TestAuditChainLegacyRows(t *testing.T) {
1873	s := chainStore(t)
1874	if _, err := s.DB.Exec("INSERT INTO audit_log (action) VALUES ('legacy')"); err != nil {
1875		t.Fatal(err)
1876	}
1877	s.Audit(0, "a", nil)
1878	res, err := s.VerifyAuditChain()
1879	if err != nil {
1880		t.Fatal(err)
1881	}
1882	if res.Unchained != 1 || res.BrokenAt != 0 || res.First != 2 {
1883		t.Fatalf("%+v", res)
1884	}
1885	if _, err := s.DB.Exec("INSERT INTO audit_log (action) VALUES ('injected')"); err != nil {
1886		t.Fatal(err)
1887	}
1888	if res, _ = s.VerifyAuditChain(); res.BrokenAt != 3 {
1889		t.Fatalf("hashless row after the chain: %+v", res)
1890	}
1891}
1892
1893func TestAuditJournal(t *testing.T) {
1894	s := chainStore(t)
1895	var buf bytes.Buffer
1896	s.AuditJournal = slog.New(slog.NewTextHandler(&buf, nil))
1897	s.Audit(0, "cmd repo create", map[string]any{"argv": []string{"a/b"}})
1898	line := buf.String()
1899	for _, want := range []string{"msg=audit", "action=\"cmd repo create\"", "id=1", "hash="} {
1900		if !strings.Contains(line, want) {
1901			t.Fatalf("journal line %q lacks %q", line, want)
1902		}
1903	}
1904}
1905```
1906
1907- [ ] **Step 2: Run them and see them fail**
1908
1909Run: `go test ./internal/store -run 'TestAuditChain|TestAuditJournal' -count=1`
1910Expected: build failure, `s.VerifyAuditChain undefined`.
1911
1912- [ ] **Step 3: Implement**
1913
1914`0070_audit_chain.up.sql`:
1915
1916```sql
1917-- Each row carries the hash of the row before it. actor_ref is the actor
1918-- id as written: actor_id is set to NULL when the account is deleted,
1919-- and the hash must not change with it. Rows written before this
1920-- migration keep an empty hash; the chain starts after them.
1921ALTER TABLE audit_log ADD COLUMN actor_ref INTEGER NOT NULL DEFAULT 0;
1922ALTER TABLE audit_log ADD COLUMN prev_hash TEXT NOT NULL DEFAULT '';
1923ALTER TABLE audit_log ADD COLUMN hash TEXT NOT NULL DEFAULT '';
1924UPDATE audit_log SET actor_ref = COALESCE(actor_id, 0);
1925```
1926
1927`0070_audit_chain.down.sql`:
1928
1929```sql
1930ALTER TABLE audit_log DROP COLUMN hash;
1931ALTER TABLE audit_log DROP COLUMN prev_hash;
1932ALTER TABLE audit_log DROP COLUMN actor_ref;
1933```
1934
1935`internal/store/store.go`, `Store` gains:
1936
1937```go
1938	// AuditJournal, when set, receives a copy of every audit row. The
1939	// daemon sets it to its own logger, whose output the service
1940	// journal keeps outside the database.
1941	AuditJournal *slog.Logger
1942```
1943
1944(`store.go` imports `log/slog`.)
1945
1946`internal/store/audit.go`, replacing `Audit` (lines 5-19) and adding
1947the chain helpers; `AuditEntry`, `AuditFilter` and `AuditEntries` stay
1948as they are:
1949
1950```go
1951package store
1952
1953import (
1954	"crypto/sha256"
1955	"database/sql"
1956	"encoding/hex"
1957	"encoding/json"
1958	"errors"
1959	"time"
1960)
1961
1962// Audit appends to the security feed. Events are the product feed; this
1963// records who did what, from where, for an operator. actorID 0 means the
1964// host admin (gitbayd admin commands) or an unauthenticated source.
1965func (s *Store) Audit(actorID int64, action string, data map[string]any) {
1966	raw, err := json.Marshal(data)
1967	if err != nil {
1968		raw = []byte("{}")
1969	}
1970	id, createdAt, hash, err := s.appendAudit(actorID, action, string(raw), time.Now())
1971	if s.AuditJournal == nil {
1972		return
1973	}
1974	if err != nil {
1975		s.AuditJournal.Error("audit: append", "action", action, "err", err)
1976		return
1977	}
1978	s.AuditJournal.Info("audit", "id", id, "actor", actorID, "action", action,
1979		"data", string(raw), "created_at", createdAt, "hash", hash)
1980}
1981
1982// appendAudit writes one row and its chain hash in one transaction. The
1983// store begins every transaction IMMEDIATE, so two writers — the daemon
1984// and a gitbayd admin command, say — cannot both read the same last
1985// hash.
1986func (s *Store) appendAudit(actorID int64, action, data string, now time.Time) (int64, string, string, error) {
1987	tx, err := s.DB.Begin()
1988	if err != nil {
1989		return 0, "", "", err
1990	}
1991	defer tx.Rollback()
1992	var prev string
1993	err = tx.QueryRow("SELECT hash FROM audit_log ORDER BY id DESC LIMIT 1").Scan(&prev)
1994	if err != nil && !errors.Is(err, sql.ErrNoRows) {
1995		return 0, "", "", err
1996	}
1997	var actor any
1998	if actorID != 0 {
1999		actor = actorID
2000	}
2001	createdAt := fmtTime(now)
2002	res, err := tx.Exec(
2003		"INSERT INTO audit_log (actor_id, actor_ref, action, data_json, created_at, prev_hash) VALUES (?, ?, ?, ?, ?, ?)",
2004		actor, actorID, action, data, createdAt, prev)
2005	if err != nil {
2006		return 0, "", "", err
2007	}
2008	id, err := res.LastInsertId()
2009	if err != nil {
2010		return 0, "", "", err
2011	}
2012	hash := auditHash(prev, id, actorID, action, createdAt, data)
2013	if _, err := tx.Exec("UPDATE audit_log SET hash = ? WHERE id = ?", hash, id); err != nil {
2014		return 0, "", "", err
2015	}
2016	return id, createdAt, hash, tx.Commit()
2017}
2018
2019// auditHash covers every column an operator reads, plus the previous
2020// row's hash. A JSON array keeps field boundaries unambiguous.
2021func auditHash(prev string, id, actor int64, action, createdAt, data string) string {
2022	b, _ := json.Marshal([]any{prev, id, actor, action, createdAt, data})
2023	sum := sha256.Sum256(b)
2024	return hex.EncodeToString(sum[:])
2025}
2026
2027// AuditChain is what VerifyAuditChain found.
2028type AuditChain struct {
2029	Rows      int   // rows read
2030	Unchained int   // rows from before migration 0070, which carry no hash
2031	First     int64 // first chained row; its prev_hash is taken as given, since retention may have removed the row it names
2032	Last      int64
2033	LastHash  string
2034	BrokenAt  int64 // 0 when the chain is intact
2035	Reason    string
2036}
2037
2038// VerifyAuditChain recomputes every row's hash in id order and stops at
2039// the first row that does not match. It cannot see rows removed from
2040// the end of the table; the journal copy covers those.
2041func (s *Store) VerifyAuditChain() (AuditChain, error) {
2042	rows, err := s.DB.Query(`SELECT id, actor_ref, action, data_json, created_at, prev_hash, hash
2043		FROM audit_log ORDER BY id`)
2044	if err != nil {
2045		return AuditChain{}, err
2046	}
2047	defer rows.Close()
2048	var res AuditChain
2049	for rows.Next() {
2050		var (
2051			id, actor                          int64
2052			action, data, createdAt, prev, hash string
2053		)
2054		if err := rows.Scan(&id, &actor, &action, &data, &createdAt, &prev, &hash); err != nil {
2055			return res, err
2056		}
2057		res.Rows++
2058		switch {
2059		case hash == "" && res.First == 0:
2060			res.Unchained++
2061			continue
2062		case hash == "":
2063			res.BrokenAt, res.Reason = id, "row has no hash after the chain began"
2064		case res.First != 0 && prev != res.LastHash:
2065			res.BrokenAt, res.Reason = id, "previous hash does not match: a row before it was removed or changed"
2066		case auditHash(prev, id, actor, action, createdAt, data) != hash:
2067			res.BrokenAt, res.Reason = id, "row contents do not match its hash"
2068		}
2069		if res.BrokenAt != 0 {
2070			return res, nil
2071		}
2072		if res.First == 0 {
2073			res.First = id
2074		}
2075		res.Last, res.LastHash = id, hash
2076	}
2077	return res, rows.Err()
2078}
2079```
2080
2081The previous `Audit` ignored every error; it still returns nothing,
2082and reports an append failure only where there is a journal to report
2083it to.
2084
2085- [ ] **Step 4: Run the package**
2086
2087Run: `go test ./internal/store -count=1`
2088Expected: PASS, `TestMigrateUpDown` and `TestSweep*` included (the
2089retention sweep still deletes by `created_at`).
2090
2091- [ ] **Step 5: Commit**
2092
2093```bash
2094git add internal/store
2095git commit -S -m "store: hash-chained audit rows, journal copy, chain verification
2096
2097Ref #275"
2098```
2099
2100### Task 5.2: refused mutating commands are audited, rate-limited per actor
2101
2102**Files:**
2103- Create: `internal/control/auditrefusal.go`, `internal/control/auditrefusal_test.go`
2104- Modify: `internal/control/control.go:153-191` (`Dispatch` from the scope check to the end)
2105
2106**Interfaces:**
2107- Produces: `func AuditRefused(st *store.Store, actorID int64, action string, data map[string]any)` — records at most `refusalsPerMinute` (10) rows per actor per minute, then one `refused.throttled` row for the rest of that minute; a nil store records nothing.
2108- Dispatch audit actions: `"cmd <path>"` on success (unchanged), `"refused <path>"` on exit 4 or exit 3, for commands that are not `ReadOnly`, with data `{"argv", "source", "exit"}`.
2109
2110- [ ] **Step 1: Write the failing tests**
2111
2112`internal/control/auditrefusal_test.go`:
2113
2114```go
2115package control
2116
2117import (
2118	"testing"
2119
2120	"gitbay.org/gitbay/internal/protocol"
2121	"gitbay.org/gitbay/internal/store"
2122)
2123
2124func TestRefusedWritesAreAudited(t *testing.T) {
2125	refusals = &refusalLimiter{seen: map[int64]*refusalWindow{}}
2126	st, repo, _ := newQueueTestRepo(t)
2127	bobID, err := st.CreateUser("bob", false)
2128	if err != nil {
2129		t.Fatal(err)
2130	}
2131	bob := store.User{ID: bobID, Username: "bob"}
2132
2133	c, _ := pruneCtx(st, t.TempDir(), bob)
2134	if code := Dispatch(c, []string{"repo", "delete", repo.Path(), "--yes"}); code != protocol.ExitDenied {
2135		t.Fatalf("exit %d, want %d", code, protocol.ExitDenied)
2136	}
2137	// A refused read is not a write attempt.
2138	c, _ = pruneCtx(st, t.TempDir(), bob)
2139	if code := Dispatch(c, []string{"audit"}); code != protocol.ExitDenied {
2140		t.Fatalf("audit: exit %d", code)
2141	}
2142	got, err := st.AuditEntries(store.AuditFilter{ActionPrefix: "refused", Limit: 10})
2143	if err != nil {
2144		t.Fatal(err)
2145	}
2146	if len(got) != 1 || got[0].Action != "refused repo delete" || got[0].Actor != "bob" {
2147		t.Fatalf("entries: %+v", got)
2148	}
2149}
2150
2151func TestRefusalAuditIsRateLimited(t *testing.T) {
2152	refusals = &refusalLimiter{seen: map[int64]*refusalWindow{}}
2153	st, repo, _ := newQueueTestRepo(t)
2154	bobID, err := st.CreateUser("bob", false)
2155	if err != nil {
2156		t.Fatal(err)
2157	}
2158	for range refusalsPerMinute + 5 {
2159		c, _ := pruneCtx(st, t.TempDir(), store.User{ID: bobID, Username: "bob"})
2160		Dispatch(c, []string{"repo", "delete", repo.Path(), "--yes"})
2161	}
2162	refused, _ := st.AuditEntries(store.AuditFilter{ActionPrefix: "refused ", Limit: 100})
2163	throttled, _ := st.AuditEntries(store.AuditFilter{ActionPrefix: "refused.throttled", Limit: 100})
2164	if len(refused) != refusalsPerMinute || len(throttled) != 1 {
2165		t.Fatalf("%d refused rows, %d throttled rows", len(refused), len(throttled))
2166	}
2167}
2168```
2169
2170`AuditEntries` with prefix `"refused "` (trailing space) excludes
2171`refused.throttled`.
2172
2173- [ ] **Step 2: Run them and see them fail**
2174
2175Run: `go test ./internal/control -run 'TestRefusedWritesAreAudited|TestRefusalAuditIsRateLimited' -count=1`
2176Expected: build failure, `undefined: refusals`.
2177
2178- [ ] **Step 3: Implement the limiter**
2179
2180`internal/control/auditrefusal.go`:
2181
2182```go
2183package control
2184
2185import (
2186	"sync"
2187	"time"
2188
2189	"gitbay.org/gitbay/internal/store"
2190)
2191
2192// refusalsPerMinute bounds audit rows for refused writes per actor. A
2193// probe is what these rows record, and a loop of probes must not grow
2194// the table without bound.
2195const refusalsPerMinute = 10
2196
2197type refusalLimiter struct {
2198	mu   sync.Mutex
2199	seen map[int64]*refusalWindow
2200}
2201
2202type refusalWindow struct {
2203	start time.Time
2204	n     int
2205}
2206
2207var refusals = &refusalLimiter{seen: map[int64]*refusalWindow{}}
2208
2209// allow reports whether to record this refusal, and whether it is the
2210// first one past the limit in the current minute.
2211func (l *refusalLimiter) allow(actor int64, now time.Time) (record, firstDropped bool) {
2212	l.mu.Lock()
2213	defer l.mu.Unlock()
2214	if len(l.seen) > 4096 {
2215		for k, w := range l.seen {
2216			if now.Sub(w.start) >= time.Minute {
2217				delete(l.seen, k)
2218			}
2219		}
2220	}
2221	w := l.seen[actor]
2222	if w == nil || now.Sub(w.start) >= time.Minute {
2223		w = &refusalWindow{start: now}
2224		l.seen[actor] = w
2225	}
2226	w.n++
2227	return w.n <= refusalsPerMinute, w.n == refusalsPerMinute+1
2228}
2229
2230// AuditRefused records a refused attempt to change something. Past the
2231// per-actor limit it records one refused.throttled row a minute and
2232// drops the rest. Dispatcher tests run without a store.
2233func AuditRefused(st *store.Store, actorID int64, action string, data map[string]any) {
2234	if st == nil {
2235		return
2236	}
2237	switch record, first := refusals.allow(actorID, time.Now()); {
2238	case record:
2239		st.Audit(actorID, action, data)
2240	case first:
2241		st.Audit(actorID, "refused.throttled", map[string]any{"limit_per_minute": refusalsPerMinute})
2242	}
2243}
2244```
2245
2246- [ ] **Step 4: Route every Dispatch refusal through it**
2247
2248In `internal/control/control.go`, everything in `Dispatch` from the
2249scope check (line 154) to the end moves into `runChecked`, and
2250`Dispatch` ends:
2251
2252```go
2253	c.Argv = args
2254	code := runChecked(c, cmd, args)
2255	if !cmd.ReadOnly {
2256		data := map[string]any{"argv": auditArgs(args), "source": c.Source}
2257		switch code {
2258		case protocol.ExitOK:
2259			// Every successful mutating command lands in the audit log.
2260			c.Store.Audit(c.User.ID, "cmd "+joinPath(cmd.Path), data)
2261		case protocol.ExitDenied, protocol.ExitNotFound:
2262			// So does every refused one: probing leaves a trace.
2263			data["exit"] = code
2264			AuditRefused(c.Store, c.User.ID, "refused "+joinPath(cmd.Path), data)
2265		}
2266	}
2267	return code
2268}
2269
2270// runChecked applies the dispatcher's own gates, then runs the command.
2271func runChecked(c *Ctx, cmd Command, args []string) int {
2272	// A runner-scoped key reaches the runner protocol and nothing else, so
2273	// the key a CI host holds cannot administer the instance.
2274	if c.Scope != "full" && !(c.Scope == "runner" && cmd.Path[0] == "runner") {
2275		return c.fail(protocol.ExitDenied, "this key's scope (%s) does not allow control commands; use a key added with --scope full", c.Scope)
2276	}
2277	if c.ReadOnly && !cmd.ReadOnly {
2278		return c.fail(protocol.ExitDenied, "this token is read-only; %s modifies state — mint one with --scope full", joinPath(cmd.Path))
2279	}
2280	// The SSH listener refuses a disabled account before it gets here; the
2281	// API and the web reach Dispatch directly, so the check lives here too.
2282	if c.User.Disabled {
2283		return c.fail(protocol.ExitDenied, "this account is disabled; ask an instance admin to enable it")
2284	}
2285	// The admin noun is gated here as well as in each handler, so a new
2286	// admin command that forgets requireInstanceAdmin is still refused.
2287	if cmd.Path[0] == "admin" && !c.User.IsAdmin {
2288		return c.fail(protocol.ExitDenied, "admin commands are for instance admins; ask one")
2289	}
2290	if c.User.Pending && !pendingAllowed(cmd.Path) {
2291		return c.fail(protocol.ExitDenied,
2292			"your account is not active yet: verify your email first (email verify <code>, or ask for the mail again with email add)")
2293	}
2294	if code := limitWrites(c, cmd); code >= 0 {
2295		return code
2296	}
2297	if !cmd.ReadsStdin {
2298		c.Stdin = emptyReader{}
2299	}
2300	return cmd.Run(c, args)
2301}
2302```
2303
2304The gates and their messages are unchanged; only their position moves.
2305A successful command's audit data keeps exactly `argv` and `source`.
2306
2307- [ ] **Step 5: Run the package**
2308
2309Run: `go test ./internal/control -count=1`
2310Expected: PASS, including `TestAdminNounGatedInDispatch` and
2311`TestRefusalsHonourJSON` (nil store: `AuditRefused` returns early).
2312
2313- [ ] **Step 6: Commit**
2314
2315```bash
2316git add internal/control
2317git commit -S -m "control: audit refused mutating commands, rate-limited per actor
2318
2319Ref #275"
2320```
2321
2322### Task 5.3: refused pushes, the daemon's journal, `gitbayd admin audit verify`
2323
2324**Files:**
2325- Modify: `internal/sshd/sshd.go:355-360` (`Exec`, git transport case)
2326- Create: `internal/sshd/refusal_test.go`
2327- Create: `cmd/gitbayd/auditverify.go`
2328- Modify: `cmd/gitbayd/main.go:138-142` (`serveCmd`, after `openStore`), `:416` (`admin audit` registration)
2329- Modify: `e2e/audit_test.go` (new test `TestAuditChainVerify`)
2330
2331**Interfaces:**
2332- Consumes: `control.AuditRefused` (Task 5.2), `store.AuditJournal`, `store.VerifyAuditChain` (Task 5.1).
2333- Produces: `func auditVerifyCmd() *cobra.Command` in package `main`.
2334
2335- [ ] **Step 1: Write the failing sshd test**
2336
2337`internal/sshd/refusal_test.go`:
2338
2339```go
2340package sshd
2341
2342import (
2343	"bytes"
2344	"path/filepath"
2345	"strings"
2346	"testing"
2347
2348	"gitbay.org/gitbay/internal/config"
2349	"gitbay.org/gitbay/internal/control"
2350	"gitbay.org/gitbay/internal/protocol"
2351	"gitbay.org/gitbay/internal/store"
2352)
2353
2354// execFixture: alice owns the public alice/app; bob has no grant on it.
2355func execFixture(t *testing.T) (config.Config, *store.Store, store.User) {
2356	t.Helper()
2357	st, err := store.Open(filepath.Join(t.TempDir(), "gitbay.db"))
2358	if err != nil {
2359		t.Fatal(err)
2360	}
2361	t.Cleanup(func() { st.Close() })
2362	if err := st.MigrateUp(); err != nil {
2363		t.Fatal(err)
2364	}
2365	alice, err := st.CreateUser("alice", false)
2366	if err != nil {
2367		t.Fatal(err)
2368	}
2369	if _, err := st.CreateRepo("user", alice, "app", "public"); err != nil {
2370		t.Fatal(err)
2371	}
2372	bobID, err := st.CreateUser("bob", false)
2373	if err != nil {
2374		t.Fatal(err)
2375	}
2376	bob, err := st.UserByID(bobID)
2377	if err != nil {
2378		t.Fatal(err)
2379	}
2380	cfg := config.Default()
2381	cfg.Server.Root = t.TempDir()
2382	return cfg, st, bob
2383}
2384
2385func TestRefusedPushIsAudited(t *testing.T) {
2386	cfg, st, bob := execFixture(t)
2387	var out, errOut bytes.Buffer
2388	code := Exec(cfg, st, bob, "full", "SHA256:test", control.Term{}, "git-receive-pack alice/app",
2389		strings.NewReader(""), &out, &errOut, nil, nil)
2390	if code != protocol.ExitDenied {
2391		t.Fatalf("exit %d: %s", code, errOut.String())
2392	}
2393	got, err := st.AuditEntries(store.AuditFilter{ActionPrefix: "refused git-receive-pack", Limit: 5})
2394	if err != nil || len(got) != 1 || got[0].Actor != "bob" || !strings.Contains(got[0].Data, "alice/app") {
2395		t.Fatalf("entries %+v, %v", got, err)
2396	}
2397}
2398```
2399
2400- [ ] **Step 2: Run it and see it fail**
2401
2402Run: `go test ./internal/sshd -run TestRefusedPushIsAudited -count=1`
2403Expected: FAIL, `entries [] <nil>`.
2404
2405- [ ] **Step 3: Implement in `Exec`**
2406
2407```go
2408		case "git-upload-pack", "git-receive-pack", "git-upload-archive":
2409			if user.Pending {
2410				fmt.Fprintln(stderr, "your account is not active yet: verify your email first")
2411				return protocol.ExitDenied
2412			}
2413			code := runGit(cfg, st, user, scope, argv, stdin, stdout, stderr)
2414			if argv[0] == "git-receive-pack" && (code == protocol.ExitDenied || code == protocol.ExitNotFound) {
2415				control.AuditRefused(st, user.ID, "refused git-receive-pack",
2416					map[string]any{"argv": argv[1:], "source": source})
2417			}
2418			return code
2419```
2420
2421Run: `go test ./internal/sshd -count=1`
2422Expected: PASS.
2423
2424- [ ] **Step 4: Write the failing e2e test**
2425
2426Append to `e2e/audit_test.go` (imports gain `path/filepath` — already
2427there — and `gitbay.org/gitbay/internal/store`):
2428
2429```go
2430// The audit log is a hash chain: gitbayd admin audit verify passes on
2431// an untouched log and names the first row that was edited (#275).
2432func TestAuditChainVerify(t *testing.T) {
2433	t.Parallel()
2434	inst := startInstance(t)
2435	aliceKey := inst.newKey(t, "alice")
2436	inst.admin(t, "admin", "user", "create", "alice", "--key", aliceKey+".pub")
2437	if _, _, code := inst.ssh(t, aliceKey, "", "repo", "create", "alice/app"); code != 0 {
2438		t.Fatal("repo create failed")
2439	}
2440	if out := inst.admin(t, "admin", "audit", "verify"); !strings.Contains(out, "intact") {
2441		t.Fatalf("verify: %s", out)
2442	}
2443
2444	st, err := store.Open(filepath.Join(inst.root, "gitbay.db"))
2445	if err != nil {
2446		t.Fatal(err)
2447	}
2448	var id int64
2449	if err := st.DB.QueryRow("SELECT id FROM audit_log WHERE action = 'cmd repo create'").Scan(&id); err != nil {
2450		t.Fatal(err)
2451	}
2452	if _, err := st.DB.Exec("UPDATE audit_log SET data_json = '{}' WHERE id = ?", id); err != nil {
2453		t.Fatal(err)
2454	}
2455	st.Close()
2456	out := inst.forgedAdminErr(t, "admin", "audit", "verify")
2457	if !strings.Contains(out, fmt.Sprintf("row %d", id)) {
2458		t.Fatalf("verify after edit: %s", out)
2459	}
2460}
2461```
2462
2463(`fmt` is added to the file's imports.)
2464
2465- [ ] **Step 5: Run it and see it fail**
2466
2467Run: `go build ./... && go test ./e2e -run TestAuditChainVerify -count=1`
2468Expected: FAIL, `gitbayd [admin audit verify]: exit status 2` — the
2469host `audit` command reads `verify` as an unknown argument.
2470
2471- [ ] **Step 6: Implement the verify command and the journal**
2472
2473`cmd/gitbayd/auditverify.go`:
2474
2475```go
2476package main
2477
2478import (
2479	"fmt"
2480
2481	"github.com/spf13/cobra"
2482
2483	"gitbay.org/gitbay/internal/config"
2484)
2485
2486// auditVerifyCmd recomputes the audit log's hash chain. A break names
2487// the first row that does not match; rows removed from the end of the
2488// log leave no break, and the journal copy is the record for those.
2489func auditVerifyCmd() *cobra.Command {
2490	return &cobra.Command{
2491		Use:   "verify",
2492		Short: "check the audit log's hash chain",
2493		Args:  cobra.NoArgs,
2494		RunE: func(cmd *cobra.Command, args []string) error {
2495			cfg, err := config.Load(configPath)
2496			if err != nil {
2497				return err
2498			}
2499			st, err := openStore(cfg)
2500			if err != nil {
2501				return err
2502			}
2503			defer st.Close()
2504			res, err := st.VerifyAuditChain()
2505			if err != nil {
2506				return err
2507			}
2508			fmt.Printf("read %d rows (%d from before the chain)\n", res.Rows, res.Unchained)
2509			if res.BrokenAt != 0 {
2510				return fmt.Errorf("chain broken at row %d: %s", res.BrokenAt, res.Reason)
2511			}
2512			if res.Last == 0 {
2513				fmt.Println("no chained rows yet")
2514				return nil
2515			}
2516			fmt.Printf("chain intact from row %d to row %d\nlast hash %s\n", res.First, res.Last, res.LastHash)
2517			return nil
2518		},
2519	}
2520}
2521```
2522
2523`cmd/gitbayd/main.go`, in `adminCmd`, replace the `hostCmd("audit …")`
2524entry in `admin.AddCommand(…)` with a variable built before the call:
2525
2526```go
2527	auditCmd := hostCmd("audit [--limit n] [--json]", "print the security audit log, newest first", "audit")
2528	auditCmd.AddCommand(auditVerifyCmd())
2529```
2530
2531and pass `auditCmd` in its place. cobra resolves `verify` as the child
2532before the parent's passthrough arguments are considered, so
2533`gitbayd admin audit --limit 5` is unchanged.
2534
2535In `serveCmd`, after `defer st.Close()`:
2536
2537```go
2538			// The daemon's stderr is the service journal: a copy of each
2539			// audit row outside the database the daemon can write.
2540			st.AuditJournal = slog.Default()
2541```
2542
2543`gitbayd shell` and host `gitbayd admin` commands leave it unset:
2544their stderr is the SSH client or the operator's terminal (open
2545question 1).
2546
2547- [ ] **Step 7: Run it**
2548
2549Run: `go build ./... && go vet ./... && go test ./cmd/gitbayd ./internal/sshd ./internal/control ./internal/store -count=1 && go test ./e2e -run TestAuditChainVerify -count=1`
2550Expected: PASS.
2551
2552- [ ] **Step 8: Docs**
2553
2554`Admin.org`, under `* Audit and account control`, the first paragraph
2555becomes:
2556
2557```org
2558The audit log is the security feed (events are the product feed): every
2559successful mutating command with its argv and source credential (SSH key
2560fingerprint or API), every refused one (exit 3 or 4) as =refused
2561<command>=, refused pushes as =refused git-receive-pack=, registrations,
2562admin actions, force-pushes, and auth failures/throttling. Refusals are
2563recorded up to ten a minute per account; past that, one
2564=refused.throttled= row stands for the rest of the minute. Secrets
2565never appear — they travel on stdin, never in argv.
2566
2567Each row carries the SHA-256 of the row before it. =gitbayd admin audit
2568verify= recomputes the chain and names the first row that was edited or
2569whose predecessor was removed; it prints the last hash, which an
2570operator can note elsewhere. Retention removing the oldest rows is not
2571a break. Rows removed from the end leave no break, so the daemon also
2572logs every row to its journal (=journalctl -u gitbayd -g '^.*msg=audit'=),
2573outside the database it writes. Rows written before the chain existed
2574are counted and skipped.
2575```
2576
2577and add to the command block:
2578
2579```org
2580gitbayd admin audit verify           # check the hash chain; exit 1 names the first bad row
2581```
2582
2583`Architecture/09-Controls.org`, the two Logging rows:
2584
2585```org
2586| Denied attempts audited                     | in place | refused mutating commands and pushes, ten a minute per actor (=internal/control/auditrefusal.go=) |
2587| Audit log tamper resistance                 | partial  | hash chain checked by =gitbayd admin audit verify=; every row copied to the journal; the table itself is writable by the daemon user |
2588```
2589
2590`Architecture/06-Data-and-Cryptography.org`, the Audit and feed row
2591(line 21): append ", a hash chain (=prev_hash=, =hash=)" to its
2592description column.
2593
2594`Architecture/10-Known-Gaps.org`: delete the `#275` row.
2595
2596- [ ] **Step 9: Commit, MR, merge**
2597
2598```bash
2599git add internal/sshd cmd/gitbayd e2e/audit_test.go .gitbay/wiki
2600git commit -S -m "audit: refused pushes, journal copy, admin audit verify
2601
2602Closes #275"
2603git push -u origin audit-refusals-chain
2604gitbay mr create --source audit-refusals-chain --target main --title "audit: record refusals; hash-chain the log"
2605```
2606
2607Merge `--strategy ff` after CI, delete the branch both places.
2608
2609---
2610
2611# MR 6: one limit on pack generation (branch `pack-limit`, closes #262)
2612
2613### Task 6.1: `internal/packlimit`
2614
2615**Files:**
2616- Create: `internal/packlimit/packlimit.go`, `internal/packlimit/packlimit_test.go`
2617
2618**Interfaces:**
2619- Produces:
2620  - `var ErrBusy error`, `var ErrGone error`
2621  - `func New(max, per, queue int, wait time.Duration) *Limiter` — nil when `max <= 0` (no limit); `per <= 0` means no per-principal cap; `queue` is how many may wait (0: none).
2622  - `func (l *Limiter) Acquire(done <-chan struct{}, principal string) (release func(), err error)` — nil receiver never waits; `release` is idempotent.
2623
2624- [ ] **Step 1: Write the failing tests**
2625
2626```go
2627package packlimit
2628
2629import (
2630	"errors"
2631	"testing"
2632	"time"
2633)
2634
2635func TestGlobalCap(t *testing.T) {
2636	l := New(2, 0, 0, time.Second)
2637	r1, err1 := l.Acquire(nil, "a")
2638	r2, err2 := l.Acquire(nil, "b")
2639	if err1 != nil || err2 != nil {
2640		t.Fatal(err1, err2)
2641	}
2642	if _, err := l.Acquire(nil, "c"); !errors.Is(err, ErrBusy) {
2643		t.Fatalf("third with no queue: %v", err)
2644	}
2645	r1()
2646	r1() // a second release is a no-op
2647	r3, err := l.Acquire(nil, "c")
2648	if err != nil {
2649		t.Fatal(err)
2650	}
2651	if _, err := l.Acquire(nil, "d"); !errors.Is(err, ErrBusy) {
2652		t.Fatalf("double release freed two slots: %v", err)
2653	}
2654	r2()
2655	r3()
2656}
2657
2658func TestPerPrincipalCap(t *testing.T) {
2659	l := New(4, 1, 4, 50*time.Millisecond)
2660	ra, err := l.Acquire(nil, "a")
2661	if err != nil {
2662		t.Fatal(err)
2663	}
2664	defer ra()
2665	if _, err := l.Acquire(nil, "a"); !errors.Is(err, ErrBusy) {
2666		t.Fatalf("second for a: %v", err)
2667	}
2668	rb, err := l.Acquire(nil, "b")
2669	if err != nil {
2670		t.Fatalf("b blocked by a: %v", err)
2671	}
2672	rb()
2673}
2674
2675func TestWaiterGetsReleasedSlot(t *testing.T) {
2676	l := New(1, 0, 1, 5*time.Second)
2677	r1, _ := l.Acquire(nil, "a")
2678	got := make(chan error, 1)
2679	go func() {
2680		r, err := l.Acquire(nil, "b")
2681		if err == nil {
2682			r()
2683		}
2684		got <- err
2685	}()
2686	waitQueued(t, l, 1)
2687	r1()
2688	select {
2689	case err := <-got:
2690		if err != nil {
2691			t.Fatal(err)
2692		}
2693	case <-time.After(2 * time.Second):
2694		t.Fatal("waiter never got the slot")
2695	}
2696}
2697
2698func TestQueueIsBounded(t *testing.T) {
2699	l := New(1, 0, 1, 5*time.Second)
2700	r1, _ := l.Acquire(nil, "a")
2701	defer r1()
2702	go l.Acquire(nil, "b")
2703	waitQueued(t, l, 1)
2704	if _, err := l.Acquire(nil, "c"); !errors.Is(err, ErrBusy) {
2705		t.Fatalf("queue over its bound: %v", err)
2706	}
2707}
2708
2709// A principal cannot fill the queue on its own.
2710func TestPrincipalQueueIsBounded(t *testing.T) {
2711	l := New(1, 1, 8, 5*time.Second)
2712	r1, _ := l.Acquire(nil, "x")
2713	defer r1()
2714	go l.Acquire(nil, "a")
2715	waitQueued(t, l, 1)
2716	if _, err := l.Acquire(nil, "a"); !errors.Is(err, ErrBusy) {
2717		t.Fatalf("second waiter for a: %v", err)
2718	}
2719}
2720
2721func TestClientGoneWhileQueued(t *testing.T) {
2722	l := New(1, 0, 1, 5*time.Second)
2723	r1, _ := l.Acquire(nil, "a")
2724	defer r1()
2725	done := make(chan struct{})
2726	close(done)
2727	if _, err := l.Acquire(done, "b"); !errors.Is(err, ErrGone) {
2728		t.Fatalf("got %v, want ErrGone", err)
2729	}
2730}
2731
2732func TestWaitRunsOut(t *testing.T) {
2733	l := New(1, 0, 1, 20*time.Millisecond)
2734	r1, _ := l.Acquire(nil, "a")
2735	defer r1()
2736	if _, err := l.Acquire(nil, "b"); !errors.Is(err, ErrBusy) {
2737		t.Fatalf("got %v, want ErrBusy", err)
2738	}
2739}
2740
2741func TestNilLimiterNeverWaits(t *testing.T) {
2742	var l *Limiter
2743	if l = New(0, 1, 1, time.Second); l != nil {
2744		t.Fatal("max 0 should mean no limit")
2745	}
2746	r, err := l.Acquire(nil, "a")
2747	if err != nil {
2748		t.Fatal(err)
2749	}
2750	r()
2751}
2752
2753func waitQueued(t *testing.T, l *Limiter, n int) {
2754	t.Helper()
2755	deadline := time.Now().Add(2 * time.Second)
2756	for time.Now().Before(deadline) {
2757		l.mu.Lock()
2758		q := l.queued
2759		l.mu.Unlock()
2760		if q == n {
2761			return
2762		}
2763		time.Sleep(time.Millisecond)
2764	}
2765	t.Fatalf("queue never reached %d", n)
2766}
2767```
2768
2769- [ ] **Step 2: Run them and see them fail**
2770
2771Run: `go test ./internal/packlimit -count=1`
2772Expected: `no non-test Go files` / build failure.
2773
2774- [ ] **Step 3: Implement**
2775
2776`internal/packlimit/packlimit.go`:
2777
2778```go
2779// Package packlimit bounds concurrent git pack generation. upload-pack
2780// and upload-archive over SSH, smart HTTP and git:// draw on one
2781// budget: a global cap, a cap per principal (an account, or a client
2782// address on the anonymous transports), and a bounded queue whose
2783// waiters give up after a fixed wait or when the client goes away.
2784// Waiters are not served in order; the wait bounds how long any one
2785// of them waits.
2786package packlimit
2787
2788import (
2789	"errors"
2790	"sync"
2791	"time"
2792)
2793
2794var (
2795	ErrBusy = errors.New("the server is busy generating packs for other clients; try again in a minute")
2796	ErrGone = errors.New("client went away while queued")
2797)
2798
2799type Limiter struct {
2800	max, per, queue int
2801	wait            time.Duration
2802
2803	mu      sync.Mutex
2804	running int
2805	queued  int
2806	held    map[string]int // running, per principal
2807	waiting map[string]int // queued, per principal
2808	changed chan struct{}  // closed and replaced on every release
2809}
2810
2811// New returns a limiter, or nil — no limit — when max is not positive.
2812func New(max, per, queue int, wait time.Duration) *Limiter {
2813	if max <= 0 {
2814		return nil
2815	}
2816	return &Limiter{max: max, per: per, queue: queue, wait: wait,
2817		held: map[string]int{}, waiting: map[string]int{}, changed: make(chan struct{})}
2818}
2819
2820// Acquire takes a slot for principal, queueing when none is free.
2821// release is called once git has exited. done, when it closes, ends
2822// the wait.
2823func (l *Limiter) Acquire(done <-chan struct{}, principal string) (release func(), err error) {
2824	if l == nil {
2825		return func() {}, nil
2826	}
2827	l.mu.Lock()
2828	if l.fits(principal) {
2829		l.take(principal)
2830		l.mu.Unlock()
2831		return l.releaser(principal), nil
2832	}
2833	if l.queued >= l.queue || (l.per > 0 && l.waiting[principal] >= l.per) {
2834		l.mu.Unlock()
2835		return nil, ErrBusy
2836	}
2837	l.queued++
2838	l.waiting[principal]++
2839	l.mu.Unlock()
2840	defer func() {
2841		l.mu.Lock()
2842		l.queued--
2843		if l.waiting[principal]--; l.waiting[principal] == 0 {
2844			delete(l.waiting, principal)
2845		}
2846		l.mu.Unlock()
2847	}()
2848
2849	timer := time.NewTimer(l.wait)
2850	defer timer.Stop()
2851	for {
2852		l.mu.Lock()
2853		if l.fits(principal) {
2854			l.take(principal)
2855			l.mu.Unlock()
2856			return l.releaser(principal), nil
2857		}
2858		changed := l.changed
2859		l.mu.Unlock()
2860		select {
2861		case <-changed:
2862		case <-timer.C:
2863			return nil, ErrBusy
2864		case <-done:
2865			return nil, ErrGone
2866		}
2867	}
2868}
2869
2870func (l *Limiter) fits(principal string) bool {
2871	return l.running < l.max && (l.per <= 0 || l.held[principal] < l.per)
2872}
2873
2874func (l *Limiter) take(principal string) {
2875	l.running++
2876	l.held[principal]++
2877}
2878
2879func (l *Limiter) releaser(principal string) func() {
2880	var once sync.Once
2881	return func() {
2882		once.Do(func() {
2883			l.mu.Lock()
2884			defer l.mu.Unlock()
2885			l.running--
2886			if l.held[principal]--; l.held[principal] == 0 {
2887				delete(l.held, principal)
2888			}
2889			close(l.changed)
2890			l.changed = make(chan struct{})
2891		})
2892	}
2893}
2894```
2895
2896- [ ] **Step 4: Run it, with the race detector**
2897
2898Run: `go test -race ./internal/packlimit -count=3`
2899Expected: PASS.
2900
2901- [ ] **Step 5: Commit**
2902
2903```bash
2904git add internal/packlimit
2905git commit -S -m "packlimit: global and per-principal limit with a bounded queue
2906
2907Ref #262"
2908```
2909
2910### Task 6.2: config knobs
2911
2912**Files:**
2913- Modify: `internal/config/config.go:187-209` (`Limits`), `Validate` (after the `max_snippets_per_user` check, line 344-346)
2914- Test: `internal/config/config_test.go`
2915
2916**Interfaces:**
2917- Produces: `Limits.PackConcurrency`, `PackPerPrincipal`, `PackQueue int`, `PackQueueWait string`; `func (l Limits) PackLimits() (max, per, queue int, wait time.Duration)`; constants `DefaultPackConcurrency = 3`, `DefaultPackPerPrincipal = 2`, `DefaultPackQueue = 32`, `DefaultPackQueueWait = time.Minute`.
2918
2919- [ ] **Step 1: Write the failing tests**
2920
2921```go
2922func TestPackLimits(t *testing.T) {
2923	max, per, queue, wait := Limits{}.PackLimits()
2924	if max != DefaultPackConcurrency || per != DefaultPackPerPrincipal || queue != DefaultPackQueue || wait != DefaultPackQueueWait {
2925		t.Fatalf("defaults: %d %d %d %s", max, per, queue, wait)
2926	}
2927	max, per, queue, wait = Limits{PackConcurrency: -1, PackPerPrincipal: -1, PackQueue: -1, PackQueueWait: "5s"}.PackLimits()
2928	if max != 0 || per != 0 || queue != 0 || wait != 5*time.Second {
2929		t.Fatalf("off: %d %d %d %s", max, per, queue, wait)
2930	}
2931	max, per, queue, _ = Limits{PackConcurrency: 8, PackPerPrincipal: 3, PackQueue: 64}.PackLimits()
2932	if max != 8 || per != 3 || queue != 64 {
2933		t.Fatalf("set: %d %d %d", max, per, queue)
2934	}
2935}
2936```
2937
2938(`config_test.go` imports gain `time`.) And one `TestContradictions` case:
2939
2940```go
2941		{
2942			"bad pack_queue_wait",
2943			minimal + "\n[limits]\npack_queue_wait = \"soon\"\n",
2944			"limits.pack_queue_wait",
2945		},
2946```
2947
2948- [ ] **Step 2: Run them and see them fail**
2949
2950Run: `go test ./internal/config -run 'TestPackLimits|TestContradictions' -count=1`
2951Expected: build failure, `PackLimits undefined`.
2952
2953- [ ] **Step 3: Implement**
2954
2955Add to `Limits`:
2956
2957```go
2958	// PackConcurrency caps git pack generation (upload-pack and
2959	// upload-archive) running at once across SSH, smart HTTP and git://.
2960	// PackPerPrincipal caps it per account, or per client address on the
2961	// anonymous transports. PackQueue is how many may wait for a slot,
2962	// for at most PackQueueWait ("60s"). For the three counts 0 takes the
2963	// default and a negative value turns that bound off.
2964	PackConcurrency  int    `toml:"pack_concurrency"`
2965	PackPerPrincipal int    `toml:"pack_per_principal"`
2966	PackQueue        int    `toml:"pack_queue"`
2967	PackQueueWait    string `toml:"pack_queue_wait"`
2968```
2969
2970After `DefaultWriteRate`:
2971
2972```go
2973// Pack generation defaults for a four-core host: a full clone of a large
2974// repository runs git at about 1.5 cores (Performance wiki page).
2975const (
2976	DefaultPackConcurrency  = 3
2977	DefaultPackPerPrincipal = 2
2978	DefaultPackQueue        = 32
2979	DefaultPackQueueWait    = time.Minute
2980)
2981```
2982
2983After `Limits`:
2984
2985```go
2986// PackLimits resolves the pack_* settings for packlimit.New. A zero
2987// count is no bound.
2988func (l Limits) PackLimits() (max, per, queue int, wait time.Duration) {
2989	pick := func(v, def int) int {
2990		switch {
2991		case v == 0:
2992			return def
2993		case v < 0:
2994			return 0
2995		}
2996		return v
2997	}
2998	wait = DefaultPackQueueWait
2999	if d, err := time.ParseDuration(l.PackQueueWait); err == nil && d > 0 {
3000		wait = d
3001	}
3002	return pick(l.PackConcurrency, DefaultPackConcurrency),
3003		pick(l.PackPerPrincipal, DefaultPackPerPrincipal),
3004		pick(l.PackQueue, DefaultPackQueue), wait
3005}
3006```
3007
3008In `Validate`:
3009
3010```go
3011	if w := c.Limits.PackQueueWait; w != "" {
3012		if d, err := time.ParseDuration(w); err != nil || d <= 0 {
3013			errs = append(errs, fmt.Errorf("limits.pack_queue_wait %q must be a positive duration such as 60s", w))
3014		}
3015	}
3016```
3017
3018- [ ] **Step 4: Run the package**
3019
3020Run: `go test ./internal/config -count=1`
3021Expected: PASS.
3022
3023- [ ] **Step 5: Commit**
3024
3025```bash
3026git add internal/config
3027git commit -S -m "config: pack_concurrency, pack_per_principal, pack_queue, pack_queue_wait
3028
3029Ref #262"
3030```
3031
3032### Task 6.3: SSH transports acquire a slot and die with their client
3033
3034**Files:**
3035- Modify: `internal/gitutil/gitutil.go:35-56` (`Transport` takes a context)
3036- Modify: `internal/sshd/sshd.go:34-71` (`Server.packs`, `New`), `:299-313` (`runExec`), `:339-385` (`Exec`), `:387-468` (`runGit`)
3037- Modify: `cmd/gitbayd/system.go:97`
3038- Modify: `internal/sshd/sshd_test.go:65` (`New(cfg, st, nil)`)
3039- Modify: `internal/sshd/refusal_test.go` (Exec call gains `nil` packs; new busy test)
3040
3041**Interfaces:**
3042- Consumes: `packlimit.Limiter`, `packlimit.New` (Task 6.1).
3043- Produces:
3044  - `func Transport(ctx context.Context, service, repoPath string, stdin io.Reader, stdout, errW io.Writer, extraEnv []string, maxPack int64) error`
3045  - `func New(cfg config.Config, st *store.Store, packs *packlimit.Limiter) (*Server, error)`
3046  - `func Exec(cfg config.Config, st *store.Store, packs *packlimit.Limiter, user store.User, scope, source string, term control.Term, cmdline string, stdin io.Reader, stdout, stderr io.Writer, done, stopping <-chan struct{}) int`
3047
3048- [ ] **Step 1: Write the failing test**
3049
3050In `internal/sshd/refusal_test.go`, update `TestRefusedPushIsAudited`'s
3051call to `Exec(cfg, st, nil, bob, …)` and add (imports gain `time` and
3052`gitbay.org/gitbay/internal/packlimit`):
3053
3054```go
3055func TestCloneRefusedWhenPackSlotsAreFull(t *testing.T) {
3056	cfg, st, bob := execFixture(t)
3057	packs := packlimit.New(1, 0, 0, time.Second)
3058	hold, err := packs.Acquire(nil, "ip:elsewhere")
3059	if err != nil {
3060		t.Fatal(err)
3061	}
3062	defer hold()
3063	var out, errOut bytes.Buffer
3064	code := Exec(cfg, st, packs, bob, "full", "SHA256:test", control.Term{}, "git-upload-pack alice/app",
3065		strings.NewReader(""), &out, &errOut, nil, nil)
3066	if code != protocol.ExitFailure || !strings.Contains(errOut.String(), "busy") {
3067		t.Fatalf("exit %d: %q", code, errOut.String())
3068	}
3069}
3070```
3071
3072- [ ] **Step 2: Run it and see it fail**
3073
3074Run: `go test ./internal/sshd -run TestCloneRefusedWhenPackSlotsAreFull -count=1`
3075Expected: build failure, too many arguments to `Exec`.
3076
3077- [ ] **Step 3: Implement `Transport(ctx, …)`**
3078
3079```go
3080func Transport(ctx context.Context, service, repoPath string, stdin io.Reader, stdout, errW io.Writer, extraEnv []string, maxPack int64) error {
3081	// (argument building unchanged)
3082	cmd := exec.CommandContext(ctx, toolpath.Look("git"), args...)
3083	cmd.Env = append(os.Environ(), extraEnv...)
3084	cmd.Stdin = stdin
3085	cmd.Stdout = stdout
3086	cmd.Stderr = errW
3087	return cmd.Run()
3088}
3089```
3090
3091The doc comment gains: "ctx ending kills git."
3092
3093- [ ] **Step 4: Implement in sshd**
3094
3095`Server` gains `packs *packlimit.Limiter`; `New`:
3096
3097```go
3098func New(cfg config.Config, st *store.Store, packs *packlimit.Limiter) (*Server, error) {
3099	s := &Server{cfg: cfg, st: st, packs: packs, authLimiter: newRateLimiter(cfg.Limits.SSHAuthRate, time.Minute), conns: map[*conn]struct{}{}, stopping: make(chan struct{})}
3100```
3101
3102`runExec` passes it: `return Exec(s.cfg, s.st, s.packs, user, …, done, s.stopping)`.
3103
3104`Exec` takes `packs` after `st` and passes `packs, done, stopping`
3105to `runGit`:
3106
3107```go
3108			code := runGit(cfg, st, packs, user, scope, argv, stdin, stdout, stderr, done, stopping)
3109```
3110
3111`runGit` signature:
3112
3113```go
3114func runGit(cfg config.Config, st *store.Store, packs *packlimit.Limiter, user store.User, scope string, argv []string,
3115	stdin io.Reader, stdout, stderr io.Writer, done, stopping <-chan struct{}) int {
3116```
3117
3118and after the pull-mirror refusal (line 439), before `dir :=`:
3119
3120```go
3121	// Pack generation shares one budget with smart HTTP and git://.
3122	// receive-pack stays outside it: its post-receive runs after the
3123	// client has its report, and must not be queued or killed.
3124	ctx := context.Background()
3125	if !write {
3126		release, err := packs.Acquire(done, "user:"+strconv.FormatInt(user.ID, 10))
3127		if err != nil {
3128			fmt.Fprintln(stderr, err)
3129			return protocol.ExitFailure
3130		}
3131		defer release()
3132		var cancel context.CancelFunc
3133		ctx, cancel = context.WithCancel(ctx)
3134		defer cancel()
3135		go func() {
3136			select {
3137			case <-done:
3138				// done closes on a restart too; a clone already running
3139				// finishes then. Only a departed client ends it.
3140				select {
3141				case <-stopping:
3142				default:
3143					cancel()
3144				}
3145			case <-ctx.Done():
3146			}
3147		}()
3148	}
3149```
3150
3151and the transport call becomes
3152`gitutil.Transport(ctx, service, dir, stdin, stdout, stderr, env, maxPack)`.
3153
3154`internal/sshd/sshd.go` imports `gitbay.org/gitbay/internal/packlimit`.
3155
3156`cmd/gitbayd/system.go:97`:
3157
3158```go
3159			// Each forced command is its own process, so there is no
3160			// shared pack budget in system mode (see Admin, [limits]).
3161			code := sshd.Exec(cfg, st, nil, user, key.Scope, key.Fingerprint, control.ParseTerm(os.Getenv("GITBAY_TERM")), cmdline, os.Stdin, os.Stdout, os.Stderr, nil, nil)
3162```
3163
3164`internal/sshd/sshd_test.go:65`: `srv, err := New(cfg, st, nil)`.
3165
3166`cmd/gitbayd/main.go:208`: `srv, err := sshd.New(cfg, st, nil)`, so
3167this commit builds; Task 6.5 passes the shared limiter.
3168
3169- [ ] **Step 5: Run the packages**
3170
3171Run: `go build ./... && go vet ./... && go test ./internal/sshd ./internal/gitutil -count=1`
3172Expected: PASS.
3173
3174- [ ] **Step 6: Commit**
3175
3176```bash
3177git add internal/gitutil internal/sshd cmd/gitbayd/system.go cmd/gitbayd/main.go
3178git commit -S -m "sshd: upload-pack and upload-archive take a pack slot; killed when the client leaves
3179
3180Ref #262"
3181```
3182
3183### Task 6.4: smart HTTP and git:// acquire a slot; ls-refs does not
3184
3185**Files:**
3186- Modify: `internal/httpd/smart.go:25-38` (`Server.packs`, `New`), `:122-146` (`uploadPack`)
3187- Create: `internal/httpd/packlimit_test.go`
3188- Modify: `internal/gitd/gitd.go:22-27` (`Server.packs`, `New`), `:64-77` (`handle`)
3189- Create: `internal/gitd/gitd_test.go`
3190
3191**Interfaces:**
3192- Consumes: `packlimit` (Task 6.1).
3193- Produces: `func New(cfg config.Config, st *store.Store, packs *packlimit.Limiter) *Server` in both `httpd` and `gitd`; unexported `lsRefs(br *bufio.Reader) bool` in `httpd`.
3194
3195- [ ] **Step 1: Write the failing tests**
3196
3197`internal/httpd/packlimit_test.go`:
3198
3199```go
3200package httpd
3201
3202import (
3203	"net/http"
3204	"net/http/httptest"
3205	"path/filepath"
3206	"strings"
3207	"testing"
3208	"time"
3209
3210	"gitbay.org/gitbay/internal/config"
3211	"gitbay.org/gitbay/internal/packlimit"
3212	"gitbay.org/gitbay/internal/store"
3213)
3214
3215func busyServer(t *testing.T) *Server {
3216	t.Helper()
3217	st, err := store.Open(filepath.Join(t.TempDir(), "gitbay.db"))
3218	if err != nil {
3219		t.Fatal(err)
3220	}
3221	t.Cleanup(func() { st.Close() })
3222	if err := st.MigrateUp(); err != nil {
3223		t.Fatal(err)
3224	}
3225	uid, err := st.CreateUser("alice", false)
3226	if err != nil {
3227		t.Fatal(err)
3228	}
3229	if _, err := st.CreateRepo("user", uid, "app", "public"); err != nil {
3230		t.Fatal(err)
3231	}
3232	packs := packlimit.New(1, 0, 0, time.Second)
3233	hold, err := packs.Acquire(nil, "ip:elsewhere")
3234	if err != nil {
3235		t.Fatal(err)
3236	}
3237	t.Cleanup(hold)
3238	var cfg config.Config
3239	cfg.Server.Root = t.TempDir()
3240	return &Server{cfg: cfg, st: st, packs: packs, stopping: make(chan struct{})}
3241}
3242
3243func post(s *Server, body string) *httptest.ResponseRecorder {
3244	r := httptest.NewRequest("POST", "/alice/app/git-upload-pack", strings.NewReader(body))
3245	r.SetPathValue("owner", "alice")
3246	r.SetPathValue("repo", "app")
3247	w := httptest.NewRecorder()
3248	s.uploadPack(w, r)
3249	return w
3250}
3251
3252func TestUploadPackBusyIs503(t *testing.T) {
3253	w := post(busyServer(t), "0000")
3254	if w.Code != http.StatusServiceUnavailable || w.Header().Get("Retry-After") == "" {
3255		t.Fatalf("status %d, Retry-After %q", w.Code, w.Header().Get("Retry-After"))
3256	}
3257}
3258
3259// A protocol v2 ref listing generates no pack and is never queued.
3260func TestLsRefsBypassesTheLimit(t *testing.T) {
3261	w := post(busyServer(t), "0014command=ls-refs\n0000")
3262	if w.Code == http.StatusServiceUnavailable {
3263		t.Fatal("ls-refs was held to the pack limit")
3264	}
3265}
3266```
3267
3268The repository directory does not exist, so the ls-refs request's git
3269exits non-zero; the test asserts only that it was not refused.
3270
3271`internal/gitd/gitd_test.go`:
3272
3273```go
3274package gitd
3275
3276import (
3277	"fmt"
3278	"net"
3279	"path/filepath"
3280	"strings"
3281	"testing"
3282	"time"
3283
3284	"gitbay.org/gitbay/internal/config"
3285	"gitbay.org/gitbay/internal/packlimit"
3286	"gitbay.org/gitbay/internal/store"
3287)
3288
3289func TestBusyAnswersERR(t *testing.T) {
3290	st, err := store.Open(filepath.Join(t.TempDir(), "gitbay.db"))
3291	if err != nil {
3292		t.Fatal(err)
3293	}
3294	defer st.Close()
3295	if err := st.MigrateUp(); err != nil {
3296		t.Fatal(err)
3297	}
3298	uid, err := st.CreateUser("alice", false)
3299	if err != nil {
3300		t.Fatal(err)
3301	}
3302	repoID, err := st.CreateRepo("user", uid, "app", "public")
3303	if err != nil {
3304		t.Fatal(err)
3305	}
3306	if _, err := st.UpdateRepoSettings(repoID, func(rs *store.RepoSettings) { rs.GitDaemon = true }); err != nil {
3307		t.Fatal(err)
3308	}
3309	packs := packlimit.New(1, 0, 0, time.Second)
3310	hold, _ := packs.Acquire(nil, "ip:elsewhere")
3311	defer hold()
3312
3313	s := New(config.Config{Server: config.Server{Root: t.TempDir()}}, st, packs)
3314	client, server := net.Pipe()
3315	defer client.Close()
3316	go s.handle(server)
3317	req := "git-upload-pack /alice/app.git\x00host=x\x00"
3318	fmt.Fprintf(client, "%04x%s", len(req)+4, req)
3319	client.SetReadDeadline(time.Now().Add(5 * time.Second))
3320	line, err := readPktLine(client)
3321	if err != nil || !strings.HasPrefix(line, "ERR ") || !strings.Contains(line, "busy") {
3322		t.Fatalf("got %q, %v", line, err)
3323	}
3324}
3325```
3326
3327- [ ] **Step 2: Run them and see them fail**
3328
3329Run: `go test ./internal/httpd -run 'TestUploadPackBusyIs503|TestLsRefsBypassesTheLimit' -count=1; go test ./internal/gitd -run TestBusyAnswersERR -count=1`
3330Expected: build failures, `unknown field packs`.
3331
3332- [ ] **Step 3: Implement in httpd**
3333
3334`Server` gains `packs *packlimit.Limiter`; `New`:
3335
3336```go
3337func New(cfg config.Config, st *store.Store, packs *packlimit.Limiter) *Server {
3338	proxies, _ := cfg.HTTP.TrustedProxyNets() // validated at config load
3339	return &Server{cfg: cfg, st: st, packs: packs, apiLimit: newAPILimiter(cfg.Limits.APIRate), proxies: proxies,
3340		stopping: make(chan struct{})}
3341}
3342```
3343
3344`uploadPack`, from the gzip block to the end:
3345
3346```go
3347	body := io.Reader(r.Body)
3348	if r.Header.Get("Content-Encoding") == "gzip" {
3349		gz, err := gzip.NewReader(body)
3350		if err != nil {
3351			http.Error(w, "bad gzip body", http.StatusBadRequest)
3352			return
3353		}
3354		defer gz.Close()
3355		body = gz
3356	}
3357	br := bufio.NewReader(body)
3358	if !lsRefs(br) {
3359		// Waiting ends when the client leaves or the daemon stops, so a
3360		// queued clone does not hold up a restart's drain.
3361		release, err := s.packs.Acquire(s.until(r), "ip:"+s.clientIP(r))
3362		if err != nil {
3363			if errors.Is(err, packlimit.ErrBusy) {
3364				w.Header().Set("Retry-After", "30")
3365				http.Error(w, err.Error(), http.StatusServiceUnavailable)
3366			}
3367			return
3368		}
3369		defer release()
3370	}
3371	w.Header().Set("Content-Type", "application/x-git-upload-pack-result")
3372	w.Header().Set("Cache-Control", "no-cache")
3373	dir := control.RepoDir(s.cfg.Server.Root, repo.OwnerName, repo.Name)
3374	cmd := exec.CommandContext(r.Context(), toolpath.Look("git"), "upload-pack", "--stateless-rpc", dir)
3375	cmd.Env = append(os.Environ(), gitProtocolEnv(r)...)
3376	cmd.Stdin = br
3377	cmd.Stdout = w
3378	cmd.Run()
3379}
3380
3381// lsRefs reports whether a protocol v2 request is a ref listing, which
3382// generates no pack. Its first pkt-line is "command=ls-refs".
3383func lsRefs(br *bufio.Reader) bool {
3384	const want = "command=ls-refs"
3385	head, err := br.Peek(4 + len(want))
3386	return err == nil && string(head[4:]) == want
3387}
3388```
3389
3390Imports gain `bufio`, `errors`, and
3391`gitbay.org/gitbay/internal/packlimit`.
3392
3393- [ ] **Step 4: Implement in gitd**
3394
3395```go
3396type Server struct {
3397	cfg   config.Config
3398	st    *store.Store
3399	packs *packlimit.Limiter
3400}
3401
3402func New(cfg config.Config, st *store.Store, packs *packlimit.Limiter) *Server {
3403	return &Server{cfg: cfg, st: st, packs: packs}
3404}
3405```
3406
3407In `handle`, after the "repository not exported" check:
3408
3409```go
3410	host, _, _ := net.SplitHostPort(conn.RemoteAddr().String())
3411	release, err := s.packs.Acquire(nil, "ip:"+host)
3412	if err != nil {
3413		writeErr(conn, err.Error())
3414		return
3415	}
3416	defer release()
3417```
3418
3419`net.Pipe`'s address is `"pipe"`, which `SplitHostPort` rejects; `host`
3420is then empty and the principal is `"ip:"`, which is fine for the test.
3421
3422`cmd/gitbayd/main.go:225` and `:313`: `httpd.New(cfg, st, nil)` and
3423`gitd.New(cfg, st, nil)`, so this commit builds; Task 6.5 passes the
3424shared limiter.
3425
3426- [ ] **Step 5: Run the packages**
3427
3428Run: `go build ./... && go vet ./... && go test ./internal/httpd ./internal/gitd -count=1`
3429Expected: PASS.
3430
3431- [ ] **Step 6: Commit**
3432
3433```bash
3434git add internal/httpd internal/gitd cmd/gitbayd/main.go
3435git commit -S -m "httpd, gitd: pack generation takes a slot; ls-refs does not
3436
3437Ref #262"
3438```
3439
3440### Task 6.5: one limiter for the daemon; benchmark script; docs
3441
3442**Files:**
3443- Modify: `cmd/gitbayd/main.go` (before `sshd.New`, line 204-208; `httpd.New`, line 225; `gitd.New`, line 313)
3444- Create: `deploy/clonebench.sh`
3445- Modify: `.gitbay/wiki/Admin.org` (`** [limits]`), `.gitbay/wiki/Performance.org`,
3446  `.gitbay/wiki/Architecture/09-Controls.org`, `.gitbay/wiki/Architecture/10-Known-Gaps.org`
3447
3448**Interfaces:**
3449- Consumes: `config.Limits.PackLimits()` (Task 6.2), `packlimit.New` (Task 6.1), the three `New` signatures (Tasks 6.3, 6.4).
3450
3451- [ ] **Step 1: Wire the limiter**
3452
3453Before `errCh := make(chan error, 3)`:
3454
3455```go
3456			// One pack-generation budget for SSH, smart HTTP and git://.
3457			packs := packlimit.New(cfg.Limits.PackLimits())
3458```
3459
3460then `sshd.New(cfg, st, packs)`, `httpd.New(cfg, st, packs)`,
3461`gitd.New(cfg, st, packs).Serve(gln)`. Import
3462`gitbay.org/gitbay/internal/packlimit`.
3463
3464- [ ] **Step 2: Build, vet, touched packages**
3465
3466Run: `go build ./... && go vet ./... && go test ./cmd/gitbayd ./internal/sshd ./internal/httpd ./internal/gitd ./internal/packlimit ./internal/config ./internal/gitutil -count=1`
3467Expected: PASS.
3468
3469- [ ] **Step 3: Benchmark script**
3470
3471`deploy/clonebench.sh` (mode 0755):
3472
3473```sh
3474#!/bin/sh
3475# clonebench.sh <clone-url> <n>: start n full bare clones of <clone-url>
3476# at once and print each one's wall time and outcome, then the total.
3477# Run from a machine other than the server, against a public repository.
3478set -eu
3479url=$1
3480n=$2
3481dir=$(mktemp -d)
3482trap 'rm -rf "$dir"' EXIT
3483start=$(date +%s)
3484i=1
3485while [ "$i" -le "$n" ]; do
3486    (
3487        s=$(date +%s)
3488        if git clone --quiet --bare "$url" "$dir/$i.git" 2>"$dir/$i.err"; then
3489            echo "$i ok $(( $(date +%s) - s ))s"
3490        else
3491            echo "$i failed $(( $(date +%s) - s ))s: $(head -n 1 "$dir/$i.err")"
3492        fi
3493    ) &
3494    i=$((i + 1))
3495done
3496wait
3497echo "total $(( $(date +%s) - start ))s for $n clones"
3498```
3499
3500Run: `sh -n deploy/clonebench.sh`
3501Expected: no output (syntax ok).
3502
3503- [ ] **Step 4: Docs**
3504
3505`Admin.org`, `** [limits]`, add after the `max_bytes_per_user` bullet:
3506
3507```org
3508- =pack_concurrency= (3), =pack_per_principal= (2), =pack_queue= (32),
3509  =pack_queue_wait= (="60s"=) — git pack generation (clones, fetches,
3510  =git archive --remote=) over SSH, smart HTTP and git:// shares one
3511  budget: this many at once, this many per account (per client
3512  address when anonymous), and this many waiting for at most the wait.
3513  Past that an SSH client gets "the server is busy…" and exit 1, HTTP
3514  gets 503 with =Retry-After: 30=, git:// an =ERR= line. A queued
3515  client that disconnects leaves the queue; a running clone whose
3516  client disconnects is killed. Ref listings (info/refs, protocol v2
3517  =ls-refs=), pushes and web archives are outside the budget. For the
3518  three counts 0 means the default and a negative value turns that
3519  bound off. The defaults suit a four-core host; see [[Performance]].
3520  With =ssh.mode = "system"= each SSH session is its own process and
3521  SSH clones are not counted.
3522```
3523
3524`Performance.org`, the last paragraph of `* Why it holds` becomes:
3525
3526```org
3527The practical ceiling on this hardware is concurrent pack generation:
3528full clones of large repositories are CPU-bound in git itself (the 17s
3529clone ran git at ~156% CPU). =limits.pack_concurrency= bounds how many
3530run at once across SSH, HTTP and git://, with a queue behind it (see
3531[[Admin]], =[limits]=); the measurements below set its default.
3532```
3533
3534and a new section at the end:
3535
3536```org
3537* Concurrent clones
3538
3539Measured with =deploy/clonebench.sh https://gitbay.org/krz/gitbay.git <n>=
3540from a machine outside bay1 (four cores), before and after the pack
3541limit was deployed with its defaults (=pack_concurrency= 3,
3542=pack_per_principal= 2, =pack_queue= 32, =pack_queue_wait= 60s). All
3543clones in one run come from one address, so the per-principal cap
3544applies to them; the "limit off" run sets the counts to -1.
3545```
3546
3547The table itself is added by the operator from the runbook's #262
3548measurements, in the follow-up wiki MR described there.
3549
3550`Architecture/09-Controls.org`, the concurrency row:
3551
3552```org
3553| Concurrency limit on git pack generation    | in place | global, per-principal, bounded queue across SSH, HTTP and git:// (=internal/packlimit=); not in system SSH mode |
3554```
3555
3556`Architecture/10-Known-Gaps.org`: delete the `#262` row. The question
3557row "How many concurrent clones does the host sustain?" stays until the
3558runbook's numbers are on the Performance page.
3559
3560- [ ] **Step 5: Commit, MR, merge**
3561
3562```bash
3563chmod 0755 deploy/clonebench.sh
3564git add cmd/gitbayd/main.go deploy/clonebench.sh .gitbay/wiki
3565git commit -S -m "gitbayd: one pack-generation limit for SSH, HTTP and git://
3566
3567Closes #262"
3568git push -u origin pack-limit
3569gitbay mr create --source pack-limit --target main --title "git: limit concurrent pack generation across HTTP and SSH"
3570```
3571
3572Run the runbook's #262 "before" measurement against production before
3573deploying this MR. Merge `--strategy ff` after CI, delete the branch
3574both places.
3575
3576---
3577
3578# Runbook for the operator (cmc)
3579
3580The classifier refuses root ssh to bay1 from an assistant session; these
3581steps are run by hand. Operator ssh is `ssh -p 2222 root@gitbay.org`.
3582
35831. **Before merging MR 2 (#280).** `grep -A6 '^\[mail\]' /etc/gitbay/config.toml`
3584   on bay1. If `smtp_host` is not `localhost`/loopback, check the relay
3585   offers STARTTLS: `openssl s_client -starttls smtp -connect <smtp_host> -brief </dev/null`
3586   must complete a handshake. If it does not, either set
3587   `require_tls = false` in `[mail]` before deploying (and record why
3588   on the Admin page), or switch to the relay's implicit-TLS port with
3589   `tls = "implicit"`. After deploying, `gitbay dashboard --json | jq .queues`
3590   shows no mail failures after the next notification.
35912. **Before merging MR 3 (#279).** `git --version` on bay1 must be
3592   2.37 or later (`http.curloptResolve`). If not, upgrade git first;
3593   mirrors fail with an unknown-config error otherwise. After
3594   deploying, `gitbay repo mirror sync krz/gitbay` then
3595   `gitbay repo mirror list krz/gitbay` shows the GitHub push mirror
3596   with no error.
35973. **Deploying MR 4 (#282).** Check `gitbay build list` and the
3598   journal for a push in progress, then `make deploy`. After it:
3599   `stat -c '%a %U' /var/lib/gitbay/hook.sock` shows `600 gitbay`;
3600   push a commit to a scratch repository and confirm it lands and its
3601   push event appears in `gitbay feed`.
36024. **After deploying MR 5 (#275).** On bay1, as the gitbay user:
3603   `sudo -u gitbay gitbayd --config /etc/gitbay/config.toml admin audit verify`
3604   prints "chain intact" with the unchained count equal to the rows
3605   written before the upgrade. `journalctl -u gitbayd -g 'msg=audit' -n 5`
3606   shows the rows the verify run's own session produced. Record the
3607   printed last hash somewhere off the host (a note in the
3608   password manager) if a manual anchor is wanted.
36095. **MR 6 (#262) benchmark.** From the laptop, before deploying MR 6:
3610   `for n in 1 2 4 8; do sh deploy/clonebench.sh https://gitbay.org/krz/gitbay.git $n; done`,
3611   and on bay1 `uptime` during the n=8 run. After deploying MR 6
3612   (defaults), repeat, and additionally n=16 and n=40 (40 exceeds
3613   concurrency + queue from one address with per-principal 2 and shows
3614   the refusals). Record per n: total wall time, slowest clone,
3615   refused count, load average. Put the table under
3616   `* Concurrent clones` in `.gitbay/wiki/Performance.org` on a branch
3617   `wiki-clone-benchmark`, remove the "How many concurrent clones"
3618   question row from `Architecture/10-Known-Gaps.org`, and open an MR
3619   with `Ref #262`. If the numbers show the web staying slow with 3
3620   concurrent clones, lower `DefaultPackConcurrency` in the same MR
3621   and say so on the Admin page.
36226. **After MR 1 (#281).** `openssl s_client -connect gitbay.org:443 -tls1_1 </dev/null`
3623   fails; `-tls1_2` and `-tls1_3` succeed.
3624
3625# Release notes for whoever tags these
3626
3627- mail: `mail.require_tls` defaults on for non-local relays; a relay
3628  without STARTTLS stops receiving mail unless `require_tls = false`.
3629  New `mail.tls = "implicit"` for port 465.
3630- mirror: git ≥ 2.37 required on the server; mirrors no longer follow
3631  redirects.
3632- hookd: pushes in flight across the upgrade lose their post-receive
3633  effects; deploy with none running.
3634- audit: schema 0070 adds the chain; rows before it are reported as
3635  unchained by `gitbayd admin audit verify`.
3636- limits: new `pack_*` settings with non-zero defaults; a burst of
3637  clones now queues and, past the queue, is refused with 503 / exit 1.
3638
3639# Decisions and remaining questions
3640
3641Decided 2026-09-28:
3642
3643- **receive-pack stays outside the pack limit.**
3644- **bay1's relay and git**, checked: git 2.47.3 (`http.curloptResolve`
3645  needs 2.37); relay is AWS mail manager on port 587 and negotiates
3646  STARTTLS (TLS 1.3, certificate verified). MRs 2 and 3 are not blocked;
3647  runbook steps 1 and 2 stay as the check for other operators.
3648
3649Remaining:
3650
36511. **System SSH mode.** With `ssh.mode = "system"` every session is a
3652   separate `gitbayd shell` process, so (a) the pack limiter cannot
3653   count SSH clones across sessions — this plan passes `nil` and says
3654   so on the Admin page — and (b) audit rows written there are not
3655   copied to the journal, because that process's stderr is the SSH
3656   client. The same applies to host `gitbayd admin …` commands. gitbay.org
3657   runs embedded mode; the plan documents the limit and adds nothing
3658   for system mode.
36592. **Default pack limits.** 3 / 2 / 32 / 60s are an estimate from the
3660   one measured full clone (~1.5 cores). The runbook's benchmark
3661   decides whether they stand.
3662
3663# Self-review
3664
3665- Coverage against the issues: #281 MinVersion + Admin (MR 1). #280
3666  require_tls default by locality, implicit TLS (MR 2). #279 resolve
3667  and check before each sync, pin for git, http(s) only per
3668  `ValidateURL` (MR 3). #282 chmod 0600, SO_PEERCRED behind build
3669  tags, per-push token minted in `runGit` and required by hookd, works
3670  in both SSH modes through SQLite (MR 4). #275 refusals audited with
3671  per-actor limit, hash chain with `actor_ref`, `gitbayd admin audit
3672  verify`, journal copy from the daemon (MR 5). #262 global and
3673  per-principal limit, bounded queue, cancellation while queued (done /
3674  request context / stop) and while running (SSH ctx, HTTP request
3675  context), ls-refs and info/refs outside, knobs with 4-core defaults,
3676  benchmark script and Performance section, results via runbook (MR 6).
3677- Signatures used across tasks: `packlimit.New(max, per, queue int, wait time.Duration)`
3678  and `Limits.PackLimits() (max, per, queue int, wait time.Duration)`
3679  match; `Acquire(done <-chan struct{}, principal string)` is called
3680  with `done` (SSH), `s.until(r)` (HTTP), `nil` (git://, tests).
3681  `Exec` gains `packs` in MR 6 only; MR 5's test call is updated in
3682  Task 6.3 Step 1. `CreatePushToken` returns the raw token and
3683  `DeletePushToken` takes the raw token in both sshd and tests.
3684- Migrations 0069/0070 are within plan 3's range; renumber if another
3685  plan has taken them by then.
3686- No new route, template, ReadOnly command, control command or stdin
3687  reader, so no registry rows.