.gitbay/wiki/Architecture/07-CI-and-Supply-Chain.org
93 lines · 7026 bytes
1#+title: CI and supply chain
2
3[[file:diagrams/07-ci-flow.svg]]
4
5* Pipeline definition
6
7=.gitbay/ci.yml= at the pushed commit (=internal/ci/ci.go=):
8
9| Limit / rule | Value |
10|------------------------------+---------------------------------------------------------------|
11| jobs per file | 10 |
12| steps per job | 50, each at most 4096 bytes |
13| path filters | 50 each for =paths= and =paths-ignore= |
14| job name | =^[a-z0-9][a-z0-9_-]{0,39}$= |
15| image | a restricted reference; it becomes a podman argument, so no whitespace or shell characters (=ci.go=) |
16| triggers | push, merge request, =schedule= (cron), =tags= (glob) |
17
18A file that does not parse sets a =ci/config= failure status on the
19commit instead of failing silently.
20
21* Build lifecycle
22
231. *Queue.* The post-receive hook calls =queueJobs=
24 (=internal/control/build.go=). Each job gets a =ci/<job>= status:
25 =pending= when queued, =skipped= when path filters exclude it, or
26 =success= copied from an earlier build of the same tree (#177).
27 Merge requests from forks are queued against the target repository
28 with =trusted = false=.
292. *Claim.* A runner calls =runner next= over SSH
30 (=build.go=). Allowed for a =runner=-scoped key or an admin; a
31 runner key claims only for repositories it is attached to with
32 =repo runner add=. Untrusted builds are claimable only by a runner
33 started with =-untrusted= (=internal/store/builds.go=). The
34 claim returns id, repository, job, commit, ref, steps, image and —
35 for trusted builds only — the repository's secrets (=build.go=).
363. *Run.* The runner clones over SSH into =build-<id>=, starts a
37 container and runs each step with =podman exec … sh -c <step>=
38 (=cmd/gitbay-runner/isolate.go=).
394. *Log.* =runner log <id>= streams stdin into the build row; the server
40 ends the stream if the build is cancelled (=build.go=).
415. *Result.* =runner done <id> success|failure= sets the status,
42 records an event and mails the repository's watchers a log tail on
43 failure (=build.go=).
446. *Reap.* The scheduler fails a running build whose log stream closed
45 more than 2 minutes ago, or that started more than 90 minutes ago
46 (=internal/store/builds.go=).
47
48Who may do what:
49
50| Action | Requirement |
51|------------------------------------+------------------------------------------------|
52| =build list/show/log/jobs= | read on the repository |
53| =build trigger=, =build cancel= | write on the repository |
54| =repo secret set/remove/list= | admin on the repository |
55| =repo runner add/remove= | admin on the repository |
56| =runner next/log/done= | =runner= key attached to the repository, or admin |
57| =status set= | write on the repository (any context name; #258) |
58
59* Runner isolation
60
61| Control | Implementation |
62|----------------------------+----------------------------------------------------------------------------|
63| Isolation mode | =podman= by default; =none= must be chosen explicitly and logs a warning; an unknown value or missing prerequisites refuse start (=isolate.go=) |
64| Container runtime | rootless podman under the =ci-runner= user and its subordinate uid range |
65| Image | =--pull=never=; images are built by the operator (=deploy/Containerfile.ci=) and referenced by tag |
66| Workspace | =<workdir>/build-<id>=, removed after the build; workdir must be 0700 and owned by the runner (=main.go=) |
67| Build home | =<workdir>/home/<owner>/<name>=, one per repository, mounted read-write, shared by trusted and untrusted builds of that repository (#255) |
68| Secrets | env file 0600 outside the workspace, or =--env NAME= for multi-line values |
69| Resources | per-build cgroup with =memory.max= and =cpu.max= written by the runner; unit-level =MemoryMax=6G=, =CPUQuota=300%= |
70| Network | podman default (pasta); outbound unrestricted (#260) |
71| Shutdown | SIGTERM stops claiming and drains in-flight builds; the unit uses =KillMode=mixed= |
72
73* Integrations
74
75| Integration | Trigger | Security properties |
76|-------------+----------------------+--------------------------------------------------------------------------------|
77| Webhooks | recorded events | SSRF checks at save and connect time, no redirects, HMAC-SHA256 signature, 5 attempts with exponential backoff, response body capped at 4 KiB (=internal/webhook/webhook.go=) |
78| Mirrors | schedule | address check at save; token via =GIT_ASKPASS= script (0700); heads and tags only; 10-minute timeout (=internal/mirror/mirror.go=) |
79| Dependency checks | schedule, opt-in | fixed registry hosts; package names restricted (=internal/deps/registry.go=) |
80
81* The project's own supply chain
82
83| Stage | Control |
84|----------------+---------------------------------------------------------------------------------------------|
85| Source | krz/gitbay on the instance itself; signed commits required, fast-forward merges only; =require-mr= on =main= |
86| Dependencies | 15 direct Go modules (=go.mod=); pure-Go SQLite (=modernc.org/sqlite=), no cgo |
87| CI | =build= (build, vet) and =test= (full suite against real git, ssh, sshd, gpg) on every push; =vuln= (govulncheck) nightly and before release (=.gitbay/ci.yml=) |
88| Static checks | =deploy/audit.sh=: vet, govulncheck, short fuzz runs of the pkt-line, commit, signature, PGP key and tokenizer parsers |
89| Build | =CGO_ENABLED=0 -trimpath -ldflags='-s -w -buildid='= for reproducible binaries; the commit is stamped in (=deploy/release.sh=, =Makefile=) |
90| Release | =SHA256SUMS= for every binary; a minisign signature of the manifest when the release key is present (optional) |
91| Distribution | release assets on the forge; Homebrew formula in krz/homebrew-tap built from the tag; push mirror to GitHub (read-only copy) |
92| Deploy | =make deploy= refuses a dirty tree, then copies, checks config and restarts over operator SSH |
93| CI image | built on the host from =deploy/Containerfile.ci= (=golang:1.27-trixie= plus git-lfs, gnupg, openssh, python3, sqlite3); tagged, never pulled at build time |