.gitbay/wiki/CI.org

  1#+title: CI: what a push queues
  2
  3Three mechanisms decide what a push does to CI, and they interact:
  4
  5- *Dedupe.* A job's result is a property of the commit's tree. A commit
  6  that already has a passed, queued or running build for a job is not
  7  queued again; a commit whose tree already passed a job gets that
  8  result as its status, naming the build it came from (#177). Only a
  9  trusted build counts, and for tree reuse only one on the image the
 10  job names: a fork's green build does not stand for the repository's
 11  own, so its commit is built again when it lands on a branch (#258). A
 12  job that names no =image:= is compared as naming none: its reuse does
 13  not notice the runner's default image changing, because reuse is
 14  decided when the push is queued, before any runner claims the build,
 15  and runners can differ in their default. Name the image in =ci.yml=
 16  to tie reuse to it; after an operator changes a runner's =-image=,
 17  =build trigger= builds a job afresh, since a triggered build is never
 18  reused. When a trusted and an untrusted build of the same commit both
 19  run, the one that finishes last sets =ci/<job>= on that commit. A
 20  failed, cancelled or abandoned build does not count: that commit runs
 21  again.
 22- *Path filters.* =paths= and =paths-ignore= on a job are evaluated
 23  against the files the push changed. The diff base is the old tip when
 24  it is an ancestor of the new one, and the merge base with the default
 25  branch otherwise: a new branch, or a force push after a rebase (#176).
 26  A filter that cannot be evaluated, because there is no diff base or
 27  the diff fails, runs the job rather than skipping it. A filter that
 28  excludes the push records a =skipped= status, which =require-checks=
 29  accepts (#172).
 30- *The reaper.* A build a runner claimed and never reported is failed by
 31  the scheduler's tick: two minutes after its log stream ended, or at
 32  the build deadline if no stream was ever seen (#179). Its status reads
 33  =build abandoned=.
 34
 35Which runner takes a build is the fourth: a build is claimed only by a
 36runner attached to its repository (or one polling with a full-scope
 37admin key), and an untrusted build only by one started with
 38=-untrusted=. A repository with no runner attached queues builds
 39nothing claims. See the Users page. Among what a runner may claim it
 40takes the oldest pending build, across every repository it serves;
 41a repository that must never wait on another gets a runner of its own
 42(decided in krz/gitbay#207). There is no cap on how many
 43builds an account queues or how often a schedule fires, and none is
 44planned (krz/gitbay#206): a tick queues nothing while the job's last
 45build is pending or running, so a repository with no runner holds one
 46row per scheduled job, and a schedule with a runner attached spends
 47its owner's compute, not the instance's.
 48
 49Scheduled jobs run on their cron against the default branch, never on
 50push; a default-branch push registers or updates them. Tag jobs run on
 51a matching tag push and nothing else.
 52
 53A running build is followed with =build log <owner/name> <n> --follow=:
 54the stored log, then output as the runner sends it, then the outcome
 55as =build <n> <status>= on stderr once the build ends. The exit code is
 560 whatever the outcome. A follow of a build still queued after ten
 57minutes ends on its own and says so on stderr, since nothing reaps a
 58queued build. The build page does the same without JavaScript while a
 59build is queued or running; =?follow=0= renders it once. An account
 60holds at most eight follows open, and signed-out viewers share one
 61account's eight. Over the JSON API the command answers when the build
 62ends, with the whole log. Read access is checked again every two
 63seconds: a follower who loses it is told the repository is not found.
 64A restart ends every open follow with a message saying so, rather
 65than holding the drain; follow again once the daemon is back.
 66
 67A failed build names the step it stopped at: the log's last line reads
 68=step 3/3 failed: exit 1=, and =build show= prints =failed step= (=3/3
 69go test ./... (exit 1)=) and =duration=. =build log <owner/name> <n>
 70--step failed= prints only that step's output, =--step 2= another one
 71(=0= is the clone before the first step), and =--tail 40= the last forty
 72lines of whichever was chosen; neither combines with =--follow=. The
 73build page folds the finished log into one section per step, opens the
 74failed one and links to it from the top as "Jump to failure". Builds
 75from before this reported no step; their last section is taken as the
 76failed one.
 77
 78* What a build can reach
 79
 80Builds have outbound internet access, trusted and untrusted alike. On
 81the runner's host an nftables table limits them to the forge's public
 82ports 22, 80 and 443, which closes the operator's sshd. The forge is
 83reached at the address in =GITBAY_SSH=: on a runner that polls the
 84daemon over loopback, =169.254.1.2=, which pasta translates to the
 85host's public address. See the Threat-Model page, "What a build can
 86reach", for how and why (krz/gitbay#260).
 87
 88* The table
 89
 90One =ci.yml= with five jobs, each isolating one rule:
 91
 92#+begin_src yaml
 93jobs:
 94  plain:
 95    steps: [echo plain]
 96  onapp:
 97    paths: [app/**]
 98    steps: [echo onapp]
 99  notapp:
100    paths-ignore: [app/**]
101    steps: [echo notapp]
102  nightly:
103    schedule: "0 3 * * *"
104    steps: [echo nightly]
105  release:
106    tags: "v*"
107    steps: [echo release]
108#+end_src
109
110Every push below changes =app/x= and nothing else, so a push whose
111filters are evaluated queues =onapp= and skips =notapp=, and a push
112whose filters cannot be evaluated queues both. Each cell is what the
113push produced for that job; the commit's =ci/<job>= status follows:
114
115- =queued=: a build, and a =pending= status until the runner reports.
116  =queued, untrusted= is a build without the repository's secrets.
117- =skipped=: no build, and a =skipped= status naming the filter.
118- =reused=: no build, and a =success= status naming the earlier build
119  with the same tree.
120- =registered=: no build; the job's schedule is (re)registered.
121- =abandoned=: the build failed and the status reads =build abandoned=.
122- =—=: nothing new. A status the commit already had stands.
123
124| push | plain | onapp | notapp | nightly | release | ci/config |
125|---+---+---+---+---+---+---|
126| first push of the default branch | queued | queued | queued | registered | — | — |
127| push to the default branch | queued | queued | skipped | registered | — | — |
128| push to another branch | queued | queued | skipped | — | — | — |
129| a new branch, no old sha | queued | queued | skipped | — | — | — |
130| rebase onto a moved default branch, old not an ancestor | queued | queued | skipped | — | — | — |
131| rewritten commit, same tree as a passed build | reused | reused | skipped | — | — | — |
132| fast-forward of a commit built on another branch | — | — | — | registered | — | — |
133| a commit whose earlier build failed, on a new branch | queued | queued | — | — | — | — |
134| merge request head from a fork | queued, untrusted | queued, untrusted | queued, untrusted | — | — | — |
135| tag push | — | — | — | — | queued | — |
136| schedule tick on the default branch | — | — | — | queued | — | — |
137| schedule tick while the last scheduled build is still pending | — | — | — | — | — | — |
138| claimed builds whose runner vanished | abandoned | abandoned | — | — | — | — |
139| push to the default branch with an old sha that cannot be diffed | queued | queued | queued | registered | — | — |
140| push with a broken ci.yml | — | — | — | — | — | failure |
141| push with no ci.yml | — | — | — | — | — | — |
142
143Rows worth a second look:
144
145- The first push of the default branch has no diff base at all: the
146  merge base of the tip with itself is the tip. Every filtered job
147  runs. The same holds when the old sha cannot be diffed on the default
148  branch; on any other branch the merge base takes over and the filters
149  apply.
150- A rewritten commit with the same tree reuses =plain= and =onapp=
151  because the tree check comes before the filter. =notapp= never had a
152  passed build to reuse, and the push changed nothing, so the filter
153  skips it again.
154- A fast-forward of a commit built elsewhere queues nothing: the
155  builds belong to the commit, not the branch. The default-branch push
156  still registers the schedule.
157- A failed build does not stand for its commit. The same commit pushed
158  to another branch runs again; =notapp= keeps its skipped status.
159- A merge request head from a fork has no diff base and, unlike a
160  branch push, no merge-base fallback: every job runs, without secrets.
161  Filtering a head down to no jobs would make it unmergeable under
162  =require-checks= (#172).
163- A context named in =repo settings require-contexts= must be one that
164  reports on merge request heads. A schedule-only or tag-only job never
165  reports there, so the gate stays pending (#258).
166
167=TestPushShapes= in =internal/hookd= runs every row against real git
168and the store, and =TestPushShapesTableOnWiki= checks that this page
169carries each row as the code has it. A row that changes fails there
170first.