.gitbay/wiki/Architecture/07-CI-and-Supply-Chain.org

 1#+title: CI and supply chain
 2
 3[[file:diagrams/07-ci-flow.svg]]
 4
 5* Pipeline definition
 6
 7=.gitbay/ci.yml= at the pushed commit (=internal/ci/ci.go=):
 8
 9| Limit / rule                 | Value                                                         |
10|------------------------------+---------------------------------------------------------------|
11| jobs per file                | 10                                                            |
12| steps per job                | 50, each at most 4096 bytes                                   |
13| path filters                 | 50 each for =paths= and =paths-ignore=                        |
14| job name                     | =^[a-z0-9][a-z0-9_-]{0,39}$=                                  |
15| image                        | a restricted reference; it becomes a podman argument, so no whitespace or shell characters (=ci.go=) |
16| triggers                     | push, merge request, =schedule= (cron), =tags= (glob)         |
17
18A file that does not parse sets a =ci/config= failure status on the
19commit instead of failing silently.
20
21* Build lifecycle
22
231. *Queue.* The post-receive hook calls =queueJobs=
24   (=internal/control/build.go=). Each job gets a =ci/<job>= status:
25   =pending= when queued, =skipped= when path filters exclude it, or
26   =success= copied from an earlier trusted build of the same tree on the same image (#177, #258).
27   Merge requests from forks are queued against the target repository
28   with =trusted = false=.
292. *Claim.* A runner calls =runner next= over SSH
30   (=build.go=). Allowed for a =runner=-scoped key or an admin; a
31   runner key claims only for repositories it is attached to with
32   =repo runner add=. Untrusted builds are claimable only by a runner
33   started with =-untrusted= (=internal/store/builds.go=). The
34   claim returns id, repository, job, commit, ref, steps, image, the
35   build's trust, and — for trusted builds only — the repository's secrets
36   (=build.go=).
373. *Run.* The runner clones over SSH into =build-<id>=, starts a
38   container and runs each step with =podman exec … sh -c <step>=
39   (=cmd/gitbay-runner/isolate.go=).
404. *Log.* =runner log <id>= streams stdin into the build row; the server
41   ends the stream if the build is cancelled (=build.go=).
425. *Result.* =runner done <id> success|failure [--step <n>] [--reason
43   <text>]= records where a failed build stopped, sets the status,
44   records an event and mails the repository's watchers a log tail on
45   failure (=build.go=).
466. *Reap.* The scheduler fails a running build whose log stream closed
47   more than 2 minutes ago, or that started more than 90 minutes ago
48   (=internal/store/builds.go=).
49
50Who may do what:
51
52| Action                             | Requirement                                    |
53|------------------------------------+------------------------------------------------|
54| =build list/show/log/jobs=         | read on the repository                         |
55| =build trigger=, =build cancel=    | write on the repository                        |
56| =repo secret set/remove/list=      | admin on the repository                        |
57| =repo runner add/remove=           | admin on the repository                        |
58| =runner next/log/done=             | =runner= key attached to the repository, or admin |
59| =status set=                       | write on the repository; =ci/*= contexts refused (=status.go=) |
60
61* Runner isolation
62
63| Control                    | Implementation                                                             |
64|----------------------------+----------------------------------------------------------------------------|
65| Isolation mode             | =podman= by default; =none= must be chosen explicitly and logs a warning; an unknown value or missing prerequisites refuse start (=isolate.go=) |
66| Container runtime          | rootless podman under the =ci-runner= user and its subordinate uid range   |
67| Image                      | =--pull=never=; images are built by the operator (=deploy/Containerfile.ci=) and referenced by tag |
68| Workspace                  | =<workdir>/build-<id>=, removed after the build; workdir must be 0700 and owned by the runner (=main.go=) |
69| Build home                 | trusted: =<workdir>/trusted-home/<owner>/<name>=, one per repository, persistent; untrusted: =<workdir>/build-<id>-home=, removed with the build (=main.go=) |
70| Secrets                    | env file 0600 outside the workspace, or =--env NAME= for multi-line values |
71| Resources                  | per-build cgroup with =memory.max= and =cpu.max= written by the runner; unit-level =MemoryMax=6G=, =CPUQuota=300%= |
72| Network                    | pasta; a loopback runner's builds run with =--no-map-gw= (=main.go=); host limited by user (=gitbay-runner-egress.nft=) and by build cgroup, trusted or untrusted (=gitbay-runner-builds.nft=, #260) |
73| Shutdown                   | SIGTERM stops claiming and drains in-flight builds; the unit uses =KillMode=mixed= |
74
75* Integrations
76
77| Integration | Trigger              | Security properties                                                            |
78|-------------+----------------------+--------------------------------------------------------------------------------|
79| Webhooks    | recorded events      | SSRF checks at save and connect time, no redirects, HMAC-SHA256 signature, 5 attempts with exponential backoff, response body capped at 4 KiB (=internal/webhook/webhook.go=) |
80| Mirrors     | schedule             | address check at save and before each sync, git pinned to the checked addresses, no redirects; token via =GIT_ASKPASS= script (0700); heads and tags only; 10-minute timeout (=internal/mirror/mirror.go=) |
81| Dependency checks | schedule, opt-in | fixed registry hosts; package names restricted (=internal/deps/registry.go=) |
82
83* The project's own supply chain
84
85| Stage          | Control                                                                                     |
86|----------------+---------------------------------------------------------------------------------------------|
87| Source         | krz/gitbay on the instance itself; signed commits required, fast-forward merges only; =require-mr= on =main= |
88| Dependencies   | 15 direct Go modules (=go.mod=); pure-Go SQLite (=modernc.org/sqlite=), no cgo              |
89| CI             | =build= (build, vet) and =test= (full suite against real git, ssh, sshd, gpg) on every push; =vuln= (govulncheck) nightly and before release (=.gitbay/ci.yml=) |
90| Static checks  | =deploy/audit.sh=: vet, govulncheck, short fuzz runs of the pkt-line, commit, signature, PGP key and tokenizer parsers |
91| Build          | =CGO_ENABLED=0 -trimpath -ldflags='-s -w -buildid='= for reproducible binaries; the commit is stamped in (=deploy/release.sh=, =Makefile=) |
92| Release        | binaries gzipped; =SHA256SUMS= for every archive; a minisign signature of the manifest when the release key is present (optional) |
93| Distribution   | release assets on the forge; Homebrew formula in krz/homebrew-tap built from the tag; push mirror to GitHub (read-only copy) |
94| Deploy         | =make deploy= refuses a dirty tree, then copies, checks config and restarts over operator SSH |
95| CI image       | built on the host from =deploy/Containerfile.ci= (=golang:1.27-trixie= plus git-lfs, gnupg, openssh, python3, sqlite3); tagged, never pulled at build time |