Commit 7e5de5325c

7e5de5325cb233405dcf88e4ccabf7405a149144

parent: 1481a12712

Verified · cmc ci/build: success ci/test: success

cmc <hello@cleberg.net> · 2026-09-28 09:16 UTC

deploy, wiki: provision and document the secret key file

Closes #273

Layout: unified · split

.gitbay/wiki/Admin.org +42
@@ -18,8 +18,14 @@ install -m 755 gitbayd /usr/local/bin/
18adduser --system --group --home /var/lib/gitbay --shell /usr/sbin/nologin gitbay 18adduser --system --group --home /var/lib/gitbay --shell /usr/sbin/nologin gitbay
19install -d -o gitbay -g gitbay -m 750 /var/lib/gitbay 19install -d -o gitbay -g gitbay -m 750 /var/lib/gitbay
20gitbayd --config /etc/gitbay/config.toml check-config 20gitbayd --config /etc/gitbay/config.toml check-config
21gitbayd --config /etc/gitbay/config.toml admin secrets init
22chown gitbay:gitbay /etc/gitbay/secret.key
21#+end_src 23#+end_src
22 24
25=admin secrets init= refuses when the key file already exists, so a
26reinstall on the same host should skip it — =deploy/install.sh= does
27this with a file check before running it.
28
23=deploy/= in the source tree has a cloud-init file, a hardened systemd 29=deploy/= in the source tree has a cloud-init file, a hardened systemd
24unit, and a nightly backup timer. Run as the unprivileged =gitbay= user; 30unit, and a nightly backup timer. Run as the unprivileged =gitbay= user;
25the unit's =AmbientCapabilities=CAP_NET_BIND_SERVICE= covers ports 31the unit's =AmbientCapabilities=CAP_NET_BIND_SERVICE= covers ports
@@ -57,6 +63,10 @@ validation still prints, followed by the contradiction.
57 keys, ACME cache all live here. 63 keys, ACME cache all live here.
58- =site_url= (required) — canonical =https://host=; drives ACME, clone 64- =site_url= (required) — canonical =https://host=; drives ACME, clone
59 URLs, mail links. 65 URLs, mail links.
66- =secret_key_file= (default =/etc/gitbay/secret.key=) — the keys that
67 seal CI secrets, webhook secrets, mirror tokens and push device
68 tokens in the database. Must be outside =root=, mode 0600, readable
69 by the daemon's user. See "Secret key" below.
60- =source_repo= (optional, =owner/name=) — the repository this instance 70- =source_repo= (optional, =owner/name=) — the repository this instance
61 develops itself in. Startup warns when the running build's commit is 71 develops itself in. Startup warns when the running build's commit is
62 not on that repository's default branch, which is how a binary built 72 not on that repository's default branch, which is how a binary built
@@ -503,6 +513,38 @@ snapshots sit beside them and the data stays referenced. Snapshot IDs
503change; their times do not. Done for krz/keycask (formerly rust-pass) 513change; their times do not. Done for krz/keycask (formerly rust-pass)
504on 2026-09-18, across 22 snapshots. 514on 2026-09-18, across 22 snapshots.
505 515
516** Secret key
517
518CI secrets, webhook secrets, mirror tokens and APNs device tokens are
519stored sealed: AES-256-GCM under a key in =server.secret_key_file=,
520each value prefixed with the id of the key that sealed it
521(=gbs1:<id>:=). The key file is not in the database, not under
522=server.root=, and therefore in neither the local archives nor the
523main restic repository. It must be copied off the host separately;
524without it a restored database's secrets cannot be opened, and
525gitbayd refuses to start against them.
526
527#+begin_src sh
528gitbayd admin secrets init # once; deploy/install.sh does it on first install
529gitbayd admin secrets check # open every value, count by key
530gitbayd admin secrets rotate # new key, reseal, retire the old one (as root)
531#+end_src
532
533- Missing file: every gitbayd process that opens the database refuses
534 to run and names the path, including =serve= and, in system mode,
535 =authorized-keys=. =migrate= does not need it.
536- Wrong key: =serve= stops at startup naming the first row that does
537 not open; =secrets check= does the same without starting anything.
538- Upgrade: the first start after the upgrade seals every value still
539 in clear and logs =sealed secret values=.
540- Rotation: =rotate= adds a key, reseals every value under it in one
541 transaction, then removes the old keys. Run it as root, since it
542 replaces the key file in =/etc/gitbay=; the file keeps its owner.
543 The daemon re-reads the file when it changes, so it needs no
544 restart. Copy the new file off the host afterwards.
545- Push devices are looked up by the SHA-256 of their token
546 (=push_devices.token_hash=), since two seals of one token differ.
547
506* Upgrades 548* Upgrades
507 549
508Replace the binary, restart the unit. Migrations apply automatically and 550Replace the binary, restart the unit. Migrations apply automatically and
.gitbay/wiki/Architecture/03-Deployment.org +1
@@ -50,6 +50,7 @@ a database check.
50| =<root>/acme= | ACME account key and certificates | autocert defaults | 50| =<root>/acme= | ACME account key and certificates | autocert defaults |
51| =<root>/hooks= | generated hook scripts | 0755 | 51| =<root>/hooks= | generated hook scripts | 0755 |
52| =/etc/gitbay/config.toml= | configuration, including SMTP password | 0640 (cloud-init) | 52| =/etc/gitbay/config.toml= | configuration, including SMTP password | 0640 (cloud-init) |
53| =/etc/gitbay/secret.key= | keys sealing secret columns | 0600, owner =gitbay= (=deploy/install.sh=) |
53| =/var/backups/gitbay= | backup archives | 0750 (cloud-init) | 54| =/var/backups/gitbay= | backup archives | 0750 (cloud-init) |
54 55
55* Outbound connections from gitbayd 56* Outbound connections from gitbayd
.gitbay/wiki/Architecture/06-Data-and-Cryptography.org +10 −8
@@ -2,7 +2,7 @@
2 2
3* Data inventory 3* Data inventory
4 4
5Schema: =internal/store/migrations/=, 59 migrations. Classification: 5Schema: =internal/store/migrations/=, 66 migrations. Classification:
6*C* credential or secret, *P* personal data, *R* private repository 6*C* credential or secret, *P* personal data, *R* private repository
7content (as confidential as the repository), *O* operational. 7content (as confidential as the repository), *O* operational.
8 8
@@ -14,9 +14,9 @@ content (as confidential as the repository), *O* operational.
14| Collaboration | =issues=, =issue_*=, =merge_requests=, =mr_*=, =labels=, =milestones=, =mentions= | R | bodies of issues, comments and reviews | 14| Collaboration | =issues=, =issue_*=, =merge_requests=, =mr_*=, =labels=, =milestones=, =mentions= | R | bodies of issues, comments and reviews |
15| Releases, snippets | =releases=, =release_assets=, =snippets=, =snippet_files= | R | | 15| Releases, snippets | =releases=, =release_assets=, =snippets=, =snippet_files= | R | |
16| CI | =builds= (includes logs), =build_schedules=, =runner_repos=, =runner_seen= | R | build logs can echo anything a step prints | 16| CI | =builds= (includes logs), =build_schedules=, =runner_repos=, =runner_seen= | R | build logs can echo anything a step prints |
17| CI secrets | =build_secrets= | C | *plaintext* | 17| CI secrets | =build_secrets= | C | sealed (AES-256-GCM) |
18| Integrations | =webhooks= (secret), =webhook_deliveries=, =mirrors= (username, token) | C | *plaintext* secrets and tokens | 18| Integrations | =webhooks= (secret), =webhook_deliveries=, =mirrors= (username, token) | C | webhook secret and mirror token sealed |
19| Notifications | =notifications= (mail queue), =inbox=, =push_devices= (APNs token), =push_queue= | P | device tokens in clear | 19| Notifications | =notifications= (mail queue), =inbox=, =push_devices= (APNs token), =push_queue= | P | device tokens sealed; looked up by SHA-256 |
20| Signatures | =commit_signatures=, =settings.key_epoch= | O | verification cache | 20| Signatures | =commit_signatures=, =settings.key_epoch= | O | verification cache |
21| Audit and feed | =audit_log=, =events= | O, P | actor ids, pruned argv, fingerprints and IPs in some audit rows, a hash chain (=prev_hash=, =hash=) | 21| Audit and feed | =audit_log=, =events= | O, P | actor ids, pruned argv, fingerprints and IPs in some audit rows, a hash chain (=prev_hash=, =hash=) |
22| Dependencies | =dep_checks=, =dep_reports= | O | | 22| Dependencies | =dep_checks=, =dep_reports= | O | |
@@ -31,6 +31,7 @@ Outside the database:
31| TLS keys (ACME) | =<root>/acme= | C | 31| TLS keys (ACME) | =<root>/acme= | C |
32| SMTP password | =/etc/gitbay/config.toml= | C | 32| SMTP password | =/etc/gitbay/config.toml= | C |
33| APNs signing key (.p8) | path in =push.key_file= | C | 33| APNs signing key (.p8) | path in =push.key_file= | C |
34| Secret key file | =server.secret_key_file= (=/etc/gitbay/secret.key=) | C |
34| Backups | =/var/backups/gitbay=, offsite | all of the above | 35| Backups | =/var/backups/gitbay=, offsite | all of the above |
35 36
36No table stores client IP addresses as a column. The daemon writes a 37No table stores client IP addresses as a column. The daemon writes a
@@ -42,14 +43,15 @@ throttling (=internal/sshd/sshd.go=).
42| Item | Protection | 43| Item | Protection |
43|---------------------------------------+----------------------------------------------------------------| 44|---------------------------------------+----------------------------------------------------------------|
44| API tokens, sessions, login links, email codes, invites | SHA-256 of a 256-bit random value; the value is shown once and never stored (=internal/store/sessions.go=) | 45| API tokens, sessions, login links, email codes, invites | SHA-256 of a 256-bit random value; the value is shown once and never stored (=internal/store/sessions.go=) |
45| CI secrets, webhook secrets, mirror tokens, APNs device tokens | stored in clear in SQLite; protection is filesystem permissions and the rule that values are write-only through the interface | 46| CI secrets, webhook secrets, mirror tokens, APNs device tokens | AES-256-GCM under a key file outside the database and outside =server.root=; additional data binds table, column and row (=internal/seal=, =internal/store/secrets.go=) |
46| SQLite file | mode 0640, directory 0750 | 47| SQLite file | mode 0640, directory 0750 |
47| Backups | the local archive is not encrypted; restic encrypts the offsite copy | 48| Backups | the local archive is not encrypted; restic encrypts the offsite copy |
48| Disk | no application-level encryption; any disk encryption is the host's | 49| Disk | no application-level encryption; any disk encryption is the host's |
49 50
50The code base contains no symmetric encryption. A database or backup 51The database file or a backup read by anyone other than the =gitbay=
51file read by anyone other than the =gitbay= user discloses every CI 52user discloses no CI secret, webhook secret, mirror token or device
52secret, webhook secret and mirror token. 53token without the key file, which neither carries. Rotation:
54=gitbayd admin secrets rotate= (Admin wiki).
53 55
54* In transit 56* In transit
55 57
.gitbay/wiki/Architecture/09-Controls.org +1 −1
@@ -56,7 +56,7 @@ chapter names of OWASP ASVS 4.0 where one fits.
56| Control | Status | Evidence | 56| Control | Status | Evidence |
57|---------------------------------------------+----------+------------------------------------------------------------------| 57|---------------------------------------------+----------+------------------------------------------------------------------|
58| TLS for all authenticated HTTP | in place | ACME or certificate files; HSTS | 58| TLS for all authenticated HTTP | in place | ACME or certificate files; HSTS |
59| Secrets encrypted at rest | gap | CI secrets, webhook secrets, mirror tokens stored in clear (#273) | 59| Secrets encrypted at rest | in place | AES-256-GCM, key file outside the database and the main backups (=internal/seal=) |
60| Secrets kept out of argv, logs and output | in place | =ReadsStdin=, pruned audit argv, write-only secret commands | 60| Secrets kept out of argv, logs and output | in place | =ReadsStdin=, pruned audit argv, write-only secret commands |
61| Local backups encrypted | gap | tar.gz in clear; offsite copy encrypted by restic (#274) | 61| Local backups encrypted | gap | tar.gz in clear; offsite copy encrypted by restic (#274) |
62| Data retention configurable | in place | =[retention]= (=internal/config/config.go=) | 62| Data retention configurable | in place | =[retention]= (=internal/config/config.go=) |
.gitbay/wiki/Architecture/10-Known-Gaps.org −1
@@ -14,7 +14,6 @@ what the 2026-09-27 review found; remove a row when its issue closes.
14| #260 | CI network | Builds share the runner's source address; no egress policy | medium | 14| #260 | CI network | Builds share the runner's source address; no egress policy | medium |
15| #261 | Various | Migration foreign-key check after commit; three web writes bypass dispatch; documentation drift | medium | 15| #261 | Various | Migration foreign-key check after commit; three web writes bypass dispatch; documentation drift | medium |
16| #262 | Availability | No limit on concurrent git pack generation | high | 16| #262 | Availability | No limit on concurrent git pack generation | high |
17| #273 | Data at rest | CI secrets, webhook secrets and mirror tokens are stored in clear in SQLite | high |
18| #274 | Backups | The local backup archive is not encrypted | medium | 17| #274 | Backups | The local backup archive is not encrypted | medium |
19| #298 | SSRF | =repo import --from= fetches without an address check | medium | 18| #298 | SSRF | =repo import --from= fetches without an address check | medium |
20| #297 | Credentials | A browser session can mint tokens and keys that outlive it | low | 19| #297 | Credentials | A browser session can mint tokens and keys that outlive it | low |
CHANGELOG.org +13
@@ -123,6 +123,19 @@ for the eighteen commands whose CLI path differs from the registry's
123 browser (#269). 123 browser (#269).
124- Untrusted builds (merge requests from forks) get a fresh HOME removed after the build and no secrets; trusted builds keep a per-repository home under =<workdir>/trusted-home=. Deploy gitbayd before the runner; the old shared homes under the runner's workdir can be deleted. (#255) 124- Untrusted builds (merge requests from forks) get a fresh HOME removed after the build and no secrets; trusted builds keep a per-repository home under =<workdir>/trusted-home=. Deploy gitbayd before the runner; the old shared homes under the runner's workdir can be deleted. (#255)
125- =status set= refuses =ci/= contexts, which belong to the instance's builds. Build results are reused only from trusted builds on the same image. =repo settings require-contexts= names status contexts that must report green; setting any turns require-checks on, and one not yet reported counts as pending. (#258) 125- =status set= refuses =ci/= contexts, which belong to the instance's builds. Build results are reused only from trusted builds on the same image. =repo settings require-contexts= names status contexts that must report green; setting any turns require-checks on, and one not yet reported counts as pending. (#258)
126*Upgrade note.* gitbayd needs =server.secret_key_file= (default
127=/etc/gitbay/secret.key=) and refuses to start without it. Before
128replacing the binary, run =gitbayd admin secrets init= as root and
129=chown gitbay:gitbay /etc/gitbay/secret.key= (=deploy/install.sh= does
130both when the file is missing). The first start seals the stored
131secrets. Back the key file up separately: =admin backup= archives do
132not carry it (see the Admin wiki, "Secret key"). Downgrading to an
133earlier release after values are sealed is not supported: an older
134gitbayd reads a sealed value's =gbs1:...= prefix as the literal
135secret.
136- CI secrets, webhook secrets, mirror tokens and push device tokens are
137 stored sealed with AES-256-GCM (#273). =gitbayd admin secrets
138 init|rotate|check=.
126- Untrusted builds (merge requests from forks) get a fresh HOME 139- Untrusted builds (merge requests from forks) get a fresh HOME
127 removed after the build and no secrets; trusted builds keep a 140 removed after the build and no secrets; trusted builds keep a
128 per-repository home under =<workdir>/trusted-home=. Deploy gitbayd 141 per-repository home under =<workdir>/trusted-home=. Deploy gitbayd
deploy/install.sh +6
@@ -14,6 +14,12 @@ ssh -p "$port" "root@$host" '
14 chmod 755 /usr/local/bin/gitbayd.new 14 chmod 755 /usr/local/bin/gitbayd.new
15 mv /usr/local/bin/gitbayd.new /usr/local/bin/gitbayd 15 mv /usr/local/bin/gitbayd.new /usr/local/bin/gitbayd
16 /usr/local/bin/gitbayd --config /etc/gitbay/config.toml check-config --no-host-checks 16 /usr/local/bin/gitbayd --config /etc/gitbay/config.toml check-config --no-host-checks
17 # The key that seals secrets in the database. Created on the first
18 # install, never replaced here; gitbayd refuses to start without it.
19 if [ ! -e /etc/gitbay/secret.key ]; then
20 /usr/local/bin/gitbayd --config /etc/gitbay/config.toml admin secrets init
21 chown gitbay:gitbay /etc/gitbay/secret.key
22 fi
17 systemctl restart gitbayd 23 systemctl restart gitbayd
18 sleep 1 24 sleep 1
19 systemctl --no-pager --lines=5 status gitbayd 25 systemctl --no-pager --lines=5 status gitbayd
e2e/backup_test.go +1 −1
@@ -16,7 +16,7 @@ import (
16// secretsCheckOneSealed matches "admin secrets check" reporting the one 16// secretsCheckOneSealed matches "admin secrets check" reporting the one
17// build secret set in TestAdminBackup as sealed under some key, e.g. 17// build secret set in TestAdminBackup as sealed under some key, e.g.
18// "build_secrets.value: key 98e412e4 1". 18// "build_secrets.value: key 98e412e4 1".
19var secretsCheckOneSealed = regexp.MustCompile(`build_secrets\.value: key \S+ 1`) 19var secretsCheckOneSealed = regexp.MustCompile(`(?m)build_secrets\.value: key \S+ 1$`)
20 20
21func TestAdminBackup(t *testing.T) { 21func TestAdminBackup(t *testing.T) {
22 t.Parallel() 22 t.Parallel()