.gitbay/wiki/CI.org

  1#+title: CI: what a push queues
  2
  3Three mechanisms decide what a push does to CI, and they interact:
  4
  5- *Dedupe.* A job's result is a property of the commit's tree. A commit
  6  that already has a passed, queued or running build for a job is not
  7  queued again; a commit whose tree already passed a job gets that
  8  result as its status, naming the build it came from (#177). A failed,
  9  cancelled or abandoned build does not count: that commit runs again.
 10- *Path filters.* =paths= and =paths-ignore= on a job are evaluated
 11  against the files the push changed. The diff base is the old tip when
 12  it is an ancestor of the new one, and the merge base with the default
 13  branch otherwise: a new branch, or a force push after a rebase (#176).
 14  A filter that cannot be evaluated, because there is no diff base or
 15  the diff fails, runs the job rather than skipping it. A filter that
 16  excludes the push records a =skipped= status, which =require-checks=
 17  accepts (#172).
 18- *The reaper.* A build a runner claimed and never reported is failed by
 19  the scheduler's tick: two minutes after its log stream ended, or at
 20  the build deadline if no stream was ever seen (#179). Its status reads
 21  =build abandoned=.
 22
 23Which runner takes a build is the fourth: a build is claimed only by a
 24runner attached to its repository (or one polling with a full-scope
 25admin key), and an untrusted build only by one started with
 26=-untrusted=. A repository with no runner attached queues builds
 27nothing claims. See the Users page. Among what a runner may claim it
 28takes the oldest pending build, across every repository it serves;
 29a repository that must never wait on another gets a runner of its own
 30(decided in krz/gitbay#207). There is no cap on how many
 31builds an account queues or how often a schedule fires, and none is
 32planned (krz/gitbay#206): a tick queues nothing while the job's last
 33build is pending or running, so a repository with no runner holds one
 34row per scheduled job, and a schedule with a runner attached spends
 35its owner's compute, not the instance's.
 36
 37Scheduled jobs run on their cron against the default branch, never on
 38push; a default-branch push registers or updates them. Tag jobs run on
 39a matching tag push and nothing else.
 40
 41A running build is followed with =build log <owner/name> <n> --follow=:
 42the stored log, then output as the runner sends it, then the outcome
 43as =build <n> <status>= on stderr once the build ends. The exit code is
 440 whatever the outcome. A follow of a build still queued after ten
 45minutes ends on its own and says so on stderr, since nothing reaps a
 46queued build. The build page does the same without JavaScript while a
 47build is queued or running; =?follow=0= renders it once. An account
 48holds at most eight follows open, and signed-out viewers share one
 49account's eight. Over the JSON API the command answers when the build
 50ends, with the whole log. Read access is checked again every two
 51seconds: a follower who loses it is told the repository is not found.
 52A restart ends every open follow with a message saying so, rather
 53than holding the drain; follow again once the daemon is back.
 54
 55* The table
 56
 57One =ci.yml= with five jobs, each isolating one rule:
 58
 59#+begin_src yaml
 60jobs:
 61  plain:
 62    steps: [echo plain]
 63  onapp:
 64    paths: [app/**]
 65    steps: [echo onapp]
 66  notapp:
 67    paths-ignore: [app/**]
 68    steps: [echo notapp]
 69  nightly:
 70    schedule: "0 3 * * *"
 71    steps: [echo nightly]
 72  release:
 73    tags: "v*"
 74    steps: [echo release]
 75#+end_src
 76
 77Every push below changes =app/x= and nothing else, so a push whose
 78filters are evaluated queues =onapp= and skips =notapp=, and a push
 79whose filters cannot be evaluated queues both. Each cell is what the
 80push produced for that job; the commit's =ci/<job>= status follows:
 81
 82- =queued=: a build, and a =pending= status until the runner reports.
 83  =queued, untrusted= is a build without the repository's secrets.
 84- =skipped=: no build, and a =skipped= status naming the filter.
 85- =reused=: no build, and a =success= status naming the earlier build
 86  with the same tree.
 87- =registered=: no build; the job's schedule is (re)registered.
 88- =abandoned=: the build failed and the status reads =build abandoned=.
 89- =—=: nothing new. A status the commit already had stands.
 90
 91| push | plain | onapp | notapp | nightly | release | ci/config |
 92|---+---+---+---+---+---+---|
 93| first push of the default branch | queued | queued | queued | registered | — | — |
 94| push to the default branch | queued | queued | skipped | registered | — | — |
 95| push to another branch | queued | queued | skipped | — | — | — |
 96| a new branch, no old sha | queued | queued | skipped | — | — | — |
 97| rebase onto a moved default branch, old not an ancestor | queued | queued | skipped | — | — | — |
 98| rewritten commit, same tree as a passed build | reused | reused | skipped | — | — | — |
 99| fast-forward of a commit built on another branch | — | — | — | registered | — | — |
100| a commit whose earlier build failed, on a new branch | queued | queued | — | — | — | — |
101| merge request head from a fork | queued, untrusted | queued, untrusted | queued, untrusted | — | — | — |
102| tag push | — | — | — | — | queued | — |
103| schedule tick on the default branch | — | — | — | queued | — | — |
104| schedule tick while the last scheduled build is still pending | — | — | — | — | — | — |
105| claimed builds whose runner vanished | abandoned | abandoned | — | — | — | — |
106| push to the default branch with an old sha that cannot be diffed | queued | queued | queued | registered | — | — |
107| push with a broken ci.yml | — | — | — | — | — | failure |
108| push with no ci.yml | — | — | — | — | — | — |
109
110Rows worth a second look:
111
112- The first push of the default branch has no diff base at all: the
113  merge base of the tip with itself is the tip. Every filtered job
114  runs. The same holds when the old sha cannot be diffed on the default
115  branch; on any other branch the merge base takes over and the filters
116  apply.
117- A rewritten commit with the same tree reuses =plain= and =onapp=
118  because the tree check comes before the filter. =notapp= never had a
119  passed build to reuse, and the push changed nothing, so the filter
120  skips it again.
121- A fast-forward of a commit built elsewhere queues nothing: the
122  builds belong to the commit, not the branch. The default-branch push
123  still registers the schedule.
124- A failed build does not stand for its commit. The same commit pushed
125  to another branch runs again; =notapp= keeps its skipped status.
126- A merge request head from a fork has no diff base and, unlike a
127  branch push, no merge-base fallback: every job runs, without secrets.
128  Filtering a head down to no jobs would make it unmergeable under
129  =require-checks= (#172).
130
131=TestPushShapes= in =internal/hookd= runs every row against real git
132and the store, and =TestPushShapesTableOnWiki= checks that this page
133carries each row as the code has it. A row that changes fails there
134first.