Commit 0666c9f622

0666c9f622171df00f8e2c79a13d011522039624

parent: 607ba55d8f

Verified · cmc ci/build: success ci/test: success

cmc <hello@cleberg.net> · 2026-09-28 21:34 UTC

audit: refused pushes, journal copy, admin audit verify

Closes #275

Layout: unified · split

.gitbay/wiki/Admin.org +29 −3
@@ -238,12 +238,38 @@ meaningful — an unverified address never produces a =verified= badge.
238 238
239The audit log is the security feed (events are the product feed): every 239The audit log is the security feed (events are the product feed): every
240successful mutating command with its argv and source credential (SSH key 240successful mutating command with its argv and source credential (SSH key
241fingerprint or API), registrations, admin actions, force-pushes, and 241fingerprint or API), every refused one (exit 3 or 4) as =refused
242auth failures/throttling. Secrets never appear — they travel on stdin, 242<command>=, refused pushes as =refused git-receive-pack=, registrations,
243never in argv. 243admin actions, force-pushes, and auth failures/throttling. A refusal row
244keeps the flag names and the first positional, not the values.
245Refusals are recorded up to ten a minute per account and 600 a minute
246across the instance; past either, one =refused.throttled= row stands
247for the rest of that minute. The caps bound the embedded listener, the
248web and the API; under =ssh.mode = "system"= each =gitbayd shell=
249connection counts separately. Secrets never appear — they travel on
250stdin, never in argv.
251
252Each row carries the SHA-256 of the row before it. =gitbayd admin audit
253verify= opens the store as other admin commands do, applying pending
254migrations, so run it with the binary that matches the daemon. It
255recomputes the chain and exits 1 naming the first row that was
256edited or whose predecessor was removed. Retention removing the oldest
257rows is not a break. Rows written before the chain existed are counted
258and skipped; when every row is such a row, verify warns and exits 1,
259since clearing the hash columns looks the same. After an upgrade that
260clears with the first new audit row.
261
262Removing the newest rows leaves no break, and neither do rows written
263afterwards under the freed ids. The database cannot show either. The
264daemon logs every row it writes to its journal, outside the database
265(=journalctl -u gitbayd -g 'INFO audit '=), and verify prints the last
266id and hash: compare them with the newest journal line. Rows written by
267host =gitbayd admin= commands, and by =gitbayd shell= when =ssh.mode =
268"system"=, are not copied to the journal.
244 269
245#+begin_src sh 270#+begin_src sh
246gitbayd admin audit [--actor u|-] [--action prefix] [--since 24h|7d|date] [--limit n] [--json] 271gitbayd admin audit [--actor u|-] [--action prefix] [--since 24h|7d|date] [--limit n] [--json]
272gitbayd admin audit verify # check the hash chain; exit 1 names the first bad row
247ssh git@<host> audit ... # the same, from an admin session 273ssh git@<host> audit ... # the same, from an admin session
248ssh git@<host> admin user list [--state active|pending|disabled|admin] 274ssh git@<host> admin user list [--state active|pending|disabled|admin]
249ssh git@<host> admin user show <name> # keys, emails, orgs, tokens, sessions 275ssh git@<host> admin user show <name> # keys, emails, orgs, tokens, sessions
.gitbay/wiki/Architecture/06-Data-and-Cryptography.org +1 −1
@@ -18,7 +18,7 @@ content (as confidential as the repository), *O* operational.
18| Integrations | =webhooks= (secret), =webhook_deliveries=, =mirrors= (username, token) | C | *plaintext* secrets and tokens | 18| Integrations | =webhooks= (secret), =webhook_deliveries=, =mirrors= (username, token) | C | *plaintext* secrets and tokens |
19| Notifications | =notifications= (mail queue), =inbox=, =push_devices= (APNs token), =push_queue= | P | device tokens in clear | 19| Notifications | =notifications= (mail queue), =inbox=, =push_devices= (APNs token), =push_queue= | P | device tokens in clear |
20| Signatures | =commit_signatures=, =settings.key_epoch= | O | verification cache | 20| Signatures | =commit_signatures=, =settings.key_epoch= | O | verification cache |
21| Audit and feed | =audit_log=, =events= | O, P | actor ids, pruned argv, fingerprints and IPs in some audit rows | 21| Audit and feed | =audit_log=, =events= | O, P | actor ids, pruned argv, fingerprints and IPs in some audit rows, a hash chain (=prev_hash=, =hash=) |
22| Dependencies | =dep_checks=, =dep_reports= | O | | 22| Dependencies | =dep_checks=, =dep_reports= | O | |
23 23
24Outside the database: 24Outside the database:
.gitbay/wiki/Architecture/09-Controls.org +2 −2
@@ -68,8 +68,8 @@ chapter names of OWASP ASVS 4.0 where one fits.
68|---------------------------------------------+----------+------------------------------------------------------------------| 68|---------------------------------------------+----------+------------------------------------------------------------------|
69| Security-relevant writes audited | in place | every successful mutating command (=control.go=) | 69| Security-relevant writes audited | in place | every successful mutating command (=control.go=) |
70| Authentication failures audited | in place | =auth.failed=, =auth.throttled= | 70| Authentication failures audited | in place | =auth.failed=, =auth.throttled= |
71| Denied attempts audited | gap | refused commands are not recorded (#275) | 71| Denied attempts audited | in place | refused mutating commands and pushes, ten a minute per actor, 600 in all (=internal/control/auditrefusal.go=) |
72| Audit log tamper resistance | gap | same database, writable by the daemon user (#275) | 72| Audit log tamper resistance | partial | hash chain checked by =gitbayd admin audit verify=; every row the daemon writes copied to its journal; the table is writable by the daemon user, and removing the newest rows (or reusing their ids) shows only by comparing verify's last id and hash with the journal |
73 73
74** Communications and integrations (V9, V10, V12) 74** Communications and integrations (V9, V10, V12)
75 75
.gitbay/wiki/Architecture/10-Known-Gaps.org +6 −1
@@ -18,10 +18,15 @@ what the 2026-09-27 review found; remove a row when its issue closes.
18| #262 | Availability | No limit on concurrent git pack generation | high | 18| #262 | Availability | No limit on concurrent git pack generation | high |
19| #273 | Data at rest | CI secrets, webhook secrets and mirror tokens are stored in clear in SQLite | high | 19| #273 | Data at rest | CI secrets, webhook secrets and mirror tokens are stored in clear in SQLite | high |
20| #274 | Backups | The local backup archive is not encrypted | medium | 20| #274 | Backups | The local backup archive is not encrypted | medium |
21| #275 | Audit | Refused writes are not audited; the audit table is writable by the daemon user | medium |
22| #298 | SSRF | =repo import --from= fetches without an address check | medium | 21| #298 | SSRF | =repo import --from= fetches without an address check | medium |
23| #297 | Credentials | A browser session can mint tokens and keys that outlive it | low | 22| #297 | Credentials | A browser session can mint tokens and keys that outlive it | low |
24 23
24* Not filed
25
26| Area | Gap | Severity |
27|-------+-------------------------------------------------------------------------------------------------------------+----------|
28| Audit | Removing the newest audit rows, or writing new rows under their freed ids, is not detectable from the database; only comparing =gitbayd admin audit verify='s last id and hash with the daemon's journal shows it. Rows written by =gitbayd shell= (=ssh.mode = "system"=) and host admin commands have no journal copy, and the refusal caps are per process, so under that mode each connection counts separately | low |
29
25* Questions an auditor will ask that have no answer yet 30* Questions an auditor will ask that have no answer yet
26 31
27| Question | Status | 32| Question | Status |
.gitbay/wiki/Threat-Model.org +8
@@ -233,6 +233,14 @@ assume has been checked.
233- Backups are consistent per the DB-snapshot-first ordering but are not a 233- Backups are consistent per the DB-snapshot-first ordering but are not a
234 single atomic snapshot; a few orphaned git objects are possible and 234 single atomic snapshot; a few orphaned git objects are possible and
235 harmless (see [[Admin]]). 235 harmless (see [[Admin]]).
236- The audit log lives in the database the daemon writes, so anyone with
237 the daemon user's access can change it. The hash chain makes an edited
238 or removed row show as a break under =gitbayd admin audit verify=,
239 except at the end: removing the newest rows, and writing new rows
240 under their freed ids, leaves a valid chain. Only comparing verify's
241 last id and hash with the daemon's journal copy shows it, and rows
242 written outside the daemon (=gitbayd shell= under =ssh.mode =
243 "system"=, host =gitbayd admin= commands) have no journal copy.
236- A global signature-verification epoch over-invalidates the cache on any 244- A global signature-verification epoch over-invalidates the cache on any
237 trust-input change. Correct, not a leak; a performance tradeoff. 245 trust-input change. Correct, not a leak; a performance tradeoff.
238- A build's secrets are environment variables inside its container, so 246- A build's secrets are environment variables inside its container, so
CHANGELOG.org +15
@@ -54,6 +54,21 @@ must add =--scope full=. Existing tokens keep their scope.
54 daemon acts. *Operators:* deploy with no push in flight, since a 54 daemon acts. *Operators:* deploy with no push in flight, since a
55 receive-pack started by the old daemon has no token and its 55 receive-pack started by the old daemon has no token and its
56 post-receive will be refused by the new one (#282). 56 post-receive will be refused by the new one (#282).
57- Audit rows are hash-chained, each carrying the SHA-256 of the one
58 before it (migration 0064), and the daemon logs a copy of every row it
59 writes to its journal. =gitbayd admin audit verify= prints the row
60 count and the last id and hash, and exits 1 naming the first row that
61 was edited or whose predecessor was removed. Removing the newest rows
62 shows only by comparing that last id and hash with the journal (#275).
63- Refused mutating commands (exit 3 or 4) are audited as =refused
64 <command>=, and refused pushes as =refused git-receive-pack=, keeping
65 flag names and the target but no values; ten a minute per account and
66 600 across the instance, counted per process, past which one
67 =refused.throttled= row stands for the rest of the minute. Under
68 =ssh.mode = "system"= each =gitbayd shell= connection counts
69 separately (#275).
70- Audit retention deletes by id, up to the newest row older than the
71 retention, so a clock step back cannot leave a gap in the chain (#275).
57 72
58* v1.36.0 — 2026-09-23 73* v1.36.0 — 2026-09-23
59 74
cmd/gitbayd/auditverify.go added +53
@@ -0,0 +1,53 @@
1package main
2
3import (
4 "fmt"
5
6 "github.com/spf13/cobra"
7
8 "gitbay.org/gitbay/internal/config"
9)
10
11// auditVerifyCmd recomputes the audit log's hash chain. A break names
12// the first row that does not match. Rows removed from the end of the
13// log, and rows written after that under the reused ids, leave no
14// break: the last id and hash printed here are what an operator
15// compares with the daemon's journal copy to see either.
16func auditVerifyCmd() *cobra.Command {
17 return &cobra.Command{
18 Use: "verify",
19 Short: "check the audit log's hash chain",
20 Args: cobra.NoArgs,
21 RunE: func(cmd *cobra.Command, args []string) error {
22 cfg, err := config.Load(configPath)
23 if err != nil {
24 return err
25 }
26 st, err := openStore(cfg)
27 if err != nil {
28 return err
29 }
30 defer st.Close()
31 res, err := st.VerifyAuditChain()
32 if err != nil {
33 return err
34 }
35 fmt.Printf("rows %d\nunchained %d\n", res.Rows, res.Unchained)
36 if res.BrokenAt != 0 {
37 return fmt.Errorf("chain broken at row %d: %s", res.BrokenAt, res.Reason)
38 }
39 // Every row unchained is what dropping and re-adding the hash
40 // columns leaves. It is also an upgrade from before migration
41 // 0064 with no row written since; the first new row clears it.
42 if res.Rows > 0 && res.Unchained == res.Rows {
43 return fmt.Errorf("no row carries a hash: the chain columns were cleared, or no audit row has been written since the upgrade")
44 }
45 if res.Last == 0 {
46 fmt.Println("no rows yet")
47 return nil
48 }
49 fmt.Printf("first %d\nlast %d\nlast hash %s\nchain intact\n", res.First, res.Last, res.LastHash)
50 return nil
51 },
52 }
53}
cmd/gitbayd/main.go +7 −1
@@ -140,6 +140,10 @@ func serveCmd() *cobra.Command {
140 return err 140 return err
141 } 141 }
142 defer st.Close() 142 defer st.Close()
143 // The daemon's stderr is the service journal: a copy of each
144 // audit row outside the database the daemon can write. Rows
145 // are logged at Info, which the default handler always emits.
146 st.AuditJournal = slog.Default()
143 147
144 // Regenerate hook scripts so a moved binary self-heals, then 148 // Regenerate hook scripts so a moved binary self-heals, then
145 // start the hook policy socket. 149 // start the hook policy socket.
@@ -406,6 +410,8 @@ func adminCmd() *cobra.Command {
406 ) 410 )
407 configCmd := &cobra.Command{Use: "config", Short: "the configuration in effect"} 411 configCmd := &cobra.Command{Use: "config", Short: "the configuration in effect"}
408 configCmd.AddCommand(configShowCmd()) 412 configCmd.AddCommand(configShowCmd())
413 auditCmd := hostCmd("audit [--limit n] [--json]", "print the security audit log, newest first", "audit")
414 auditCmd.AddCommand(auditVerifyCmd())
409 admin.AddCommand( 415 admin.AddCommand(
410 userCmd, 416 userCmd,
411 emailCmd, 417 emailCmd,
@@ -414,7 +420,7 @@ func adminCmd() *cobra.Command {
414 hostCmd("invite --email <address>", "issue a registration invite and email its code", "admin", "invite"), 420 hostCmd("invite --email <address>", "issue a registration invite and email its code", "admin", "invite"),
415 hostCmd("stats [--json]", "instance statistics: counts and per-repository disk usage", "admin", "stats"), 421 hostCmd("stats [--json]", "instance statistics: counts and per-repository disk usage", "admin", "stats"),
416 hostCmd("runners [--json]", "runner accounts: last poll, scope, the build each holds", "admin", "runners"), 422 hostCmd("runners [--json]", "runner accounts: last poll, scope, the build each holds", "admin", "runners"),
417 hostCmd("audit [--limit n] [--json]", "print the security audit log, newest first", "audit"), 423 auditCmd,
418 backupCmd(), 424 backupCmd(),
419 gcCmd(), 425 gcCmd(),
420 adminMigrateCommitRefsCmd(), 426 adminMigrateCommitRefsCmd(),
e2e/audit_test.go +47
@@ -2,10 +2,13 @@ package e2e
2 2
3import ( 3import (
4 "crypto/rand" 4 "crypto/rand"
5 "fmt"
5 "os" 6 "os"
6 "path/filepath" 7 "path/filepath"
7 "strings" 8 "strings"
8 "testing" 9 "testing"
10
11 "gitbay.org/gitbay/internal/store"
9) 12)
10 13
11func TestAuditAndHardening(t *testing.T) { 14func TestAuditAndHardening(t *testing.T) {
@@ -115,3 +118,47 @@ func TestAuditAndHardening(t *testing.T) {
115 t.Fatal("throttle audited more than once per window") 118 t.Fatal("throttle audited more than once per window")
116 } 119 }
117} 120}
121
122// The audit log is a hash chain: gitbayd admin audit verify passes on
123// an untouched log and names the first row that was edited (#275).
124func TestAuditChainVerify(t *testing.T) {
125 t.Parallel()
126 inst := startInstance(t)
127 aliceKey := inst.newKey(t, "alice")
128 inst.admin(t, "admin", "user", "create", "alice", "--key", aliceKey+".pub")
129 if _, _, code := inst.ssh(t, aliceKey, "", "repo", "create", "alice/app"); code != 0 {
130 t.Fatal("repo create failed")
131 }
132 if out := inst.admin(t, "admin", "audit", "verify"); !strings.Contains(out, "chain intact") {
133 t.Fatalf("verify: %s", out)
134 }
135 // The passthrough parent still takes its own flags.
136 if out := inst.admin(t, "admin", "audit", "--limit", "5"); !strings.Contains(out, "repo create") {
137 t.Fatalf("audit --limit: %s", out)
138 }
139
140 st, err := store.Open(filepath.Join(inst.root, "gitbay.db"))
141 if err != nil {
142 t.Fatal(err)
143 }
144 var id int64
145 if err := st.DB.QueryRow("SELECT id FROM audit_log WHERE action = 'cmd repo create'").Scan(&id); err != nil {
146 t.Fatal(err)
147 }
148 if _, err := st.DB.Exec("UPDATE audit_log SET data_json = '{}' WHERE id = ?", id); err != nil {
149 t.Fatal(err)
150 }
151 out := inst.forgedAdminErr(t, "admin", "audit", "verify")
152 if !strings.Contains(out, fmt.Sprintf("chain broken at row %d:", id)) {
153 t.Fatalf("verify after edit: %s", out)
154 }
155
156 // With every hash cleared no row is chained, which is not a pass.
157 if _, err := st.DB.Exec("UPDATE audit_log SET prev_hash = '', hash = ''"); err != nil {
158 t.Fatal(err)
159 }
160 st.Close()
161 if out := inst.forgedAdminErr(t, "admin", "audit", "verify"); !strings.Contains(out, "no row carries a hash") {
162 t.Fatalf("verify with hashes cleared: %s", out)
163 }
164}
internal/sshd/refusal_test.go added +68
@@ -0,0 +1,68 @@
1package sshd
2
3import (
4 "bytes"
5 "path/filepath"
6 "strings"
7 "testing"
8
9 "gitbay.org/gitbay/internal/config"
10 "gitbay.org/gitbay/internal/control"
11 "gitbay.org/gitbay/internal/protocol"
12 "gitbay.org/gitbay/internal/store"
13)
14
15// execFixture: alice owns the public alice/app; bob has no grant on it.
16func execFixture(t *testing.T) (config.Config, *store.Store, store.User) {
17 t.Helper()
18 st, err := store.Open(filepath.Join(t.TempDir(), "gitbay.db"))
19 if err != nil {
20 t.Fatal(err)
21 }
22 t.Cleanup(func() { st.Close() })
23 if err := st.MigrateUp(); err != nil {
24 t.Fatal(err)
25 }
26 alice, err := st.CreateUser("alice", false)
27 if err != nil {
28 t.Fatal(err)
29 }
30 if _, err := st.CreateRepo("user", alice, "app", "public"); err != nil {
31 t.Fatal(err)
32 }
33 bobID, err := st.CreateUser("bob", false)
34 if err != nil {
35 t.Fatal(err)
36 }
37 bob, err := st.UserByID(bobID)
38 if err != nil {
39 t.Fatal(err)
40 }
41 cfg := config.Default()
42 cfg.Server.Root = t.TempDir()
43 return cfg, st, bob
44}
45
46// A refused push leaves one row holding the target, the key and the exit
47// code, whether runGit refused it or the account is not yet active.
48func TestRefusedPushIsAudited(t *testing.T) {
49 for _, pending := range []bool{false, true} {
50 cfg, st, bob := execFixture(t)
51 bob.Pending = pending
52 key := store.SSHKey{Scope: "full", Fingerprint: "SHA256:test"}
53 var out, errOut bytes.Buffer
54 code := Exec(cfg, st, bob, key, control.Term{}, "git-receive-pack alice/app",
55 strings.NewReader(""), &out, &errOut, nil, nil, nil)
56 if code != protocol.ExitDenied {
57 t.Fatalf("pending %v: exit %d: %s", pending, code, errOut.String())
58 }
59 got, err := st.AuditEntries(store.AuditFilter{ActionPrefix: "refused git-receive-pack", Limit: 5})
60 if err != nil || len(got) != 1 || got[0].Actor != "bob" {
61 t.Fatalf("pending %v: entries %+v, %v", pending, got, err)
62 }
63 want := `{"argv":["alice/app"],"exit":4,"source":"SHA256:test"}`
64 if got[0].Data != want {
65 t.Fatalf("pending %v: data %s, want %s", pending, got[0].Data, want)
66 }
67 }
68}
internal/sshd/sshd.go +11 −2
@@ -466,11 +466,20 @@ func Exec(cfg config.Config, st *store.Store, user store.User, key store.SSHKey,
466 if len(argv) > 0 { 466 if len(argv) > 0 {
467 switch argv[0] { 467 switch argv[0] {
468 case "git-upload-pack", "git-receive-pack", "git-upload-archive": 468 case "git-upload-pack", "git-receive-pack", "git-upload-archive":
469 code := protocol.ExitDenied
469 if user.Pending { 470 if user.Pending {
470 fmt.Fprintln(stderr, "your account is not active yet: verify your email first") 471 fmt.Fprintln(stderr, "your account is not active yet: verify your email first")
471 return protocol.ExitDenied 472 } else {
473 code = runGit(cfg, st, user, key.Scope, argv, stdin, stdout, stderr, revoked)
474 }
475 // A refused push is a refused write, audited like one. runGit
476 // refuses only with the path as the one argument, so argv[1:]
477 // holds no value beyond the target.
478 if argv[0] == "git-receive-pack" && (code == protocol.ExitDenied || code == protocol.ExitNotFound) {
479 control.AuditRefused(st, user.ID, "refused git-receive-pack",
480 map[string]any{"argv": argv[1:], "source": key.Fingerprint, "exit": code})
472 } 481 }
473 return runGit(cfg, st, user, key.Scope, argv, stdin, stdout, stderr, revoked) 482 return code
474 case "git-lfs-authenticate": 483 case "git-lfs-authenticate":
475 // Part of the git transport, not the control plane: usable by 484 // Part of the git transport, not the control plane: usable by
476 // git-scoped and deploy keys, with the transports' access rules. 485 // git-scoped and deploy keys, with the transports' access rules.
internal/store/auditchain_test.go +22 −6
@@ -2,6 +2,7 @@ package store
2 2
3import ( 3import (
4 "bytes" 4 "bytes"
5 "encoding/json"
5 "fmt" 6 "fmt"
6 "log/slog" 7 "log/slog"
7 "strings" 8 "strings"
@@ -242,12 +243,27 @@ func TestAuditChainLegacyRows(t *testing.T) {
242func TestAuditJournal(t *testing.T) { 243func TestAuditJournal(t *testing.T) {
243 s := chainStore(t) 244 s := chainStore(t)
244 var buf bytes.Buffer 245 var buf bytes.Buffer
245 s.AuditJournal = slog.New(slog.NewTextHandler(&buf, nil)) 246 s.AuditJournal = slog.New(slog.NewJSONHandler(&buf, nil))
246 s.Audit(0, "cmd repo create", map[string]any{"argv": []string{"a/b"}}) 247 s.Audit(0, "cmd repo create", map[string]any{"argv": []string{"a/b"}})
247 line := buf.String() 248 lines := strings.Split(strings.TrimRight(buf.String(), "\n"), "\n")
248 for _, want := range []string{"msg=audit", "action=\"cmd repo create\"", "id=1", "hash="} { 249 if len(lines) != 1 {
249 if !strings.Contains(line, want) { 250 t.Fatalf("journal lines %q", lines)
250 t.Fatalf("journal line %q lacks %q", line, want) 251 }
251 } 252 var got map[string]any
253 if err := json.Unmarshal([]byte(lines[0]), &got); err != nil {
254 t.Fatal(err)
255 }
256 var id float64
257 var data, createdAt, hash string
258 if err := s.DB.QueryRow("SELECT id, data_json, created_at, hash FROM audit_log").Scan(&id, &data, &createdAt, &hash); err != nil {
259 t.Fatal(err)
260 }
261 want := map[string]any{
262 "level": "INFO", "msg": "audit", "id": id, "actor": float64(0), "action": "cmd repo create",
263 "data": data, "created_at": createdAt, "hash": hash,
264 }
265 delete(got, "time")
266 if fmt.Sprint(got) != fmt.Sprint(want) {
267 t.Fatalf("journal line %v, want %v", got, want)
252 } 268 }
253} 269}