Commit 8ab6240513
Verified · cmc
Layout: unified · split
.gitbay/wiki/Admin.org +10 −9
| @@ -925,16 +925,19 @@ at real ones.* Every deploy that switched the whole instance to | |||
| 925 | containers and failed took CI down with it. Instead: create a throwaway | 925 | containers and failed took CI down with it. Instead: create a throwaway |
| 926 | repository the runner account can read (public, or granted read — a | 926 | repository the runner account can read (public, or granted read — a |
| 927 | private one is "not found" to the runner and the build stays pending), | 927 | private one is "not found" to the runner and the build stays pending), |
| 928 | give it one job that names the CI image, and deploy the runner with | 928 | give it one job that names the CI image, attach the runner's key to it, |
| 929 | =-repos= naming only that repository. The production unit, with its real | 929 | and deploy the runner with =-repos= naming only that repository. The |
| 930 | hardening, then claims nothing else; other repositories' builds queue | 930 | production unit, with its real hardening, then claims nothing else; |
| 931 | until =-repos= is switched back, which is a pause, not an outage. | 931 | other repositories' builds queue until =-repos= is removed again, which |
| 932 | is a pause, not an outage. | ||
| 932 | 933 | ||
| 933 | #+begin_src sh | 934 | #+begin_src sh |
| 934 | gitbay repo create cmc/ci-smoke # then push a .gitbay/ci.yml naming the image | 935 | gitbay repo create cmc/runner-scratch # then push a .gitbay/ci.yml naming the image |
| 935 | sed -i 's#-repos krz/gitbay #-repos cmc/ci-smoke #' /etc/systemd/system/gitbay-runner.service.d/override.conf | 936 | # on the host: |
| 937 | gitbay repo runner add cmc/runner-scratch < /var/lib/gitbay-runner/.ssh/id_ed25519.pub | ||
| 938 | sed -i 's#^ExecStart=/usr/local/bin/gitbay-runner #&-repos cmc/runner-scratch #' /etc/systemd/system/gitbay-runner.service.d/override.conf | ||
| 936 | systemctl daemon-reload && systemctl restart gitbay-runner | 939 | systemctl daemon-reload && systemctl restart gitbay-runner |
| 937 | gitbay build log cmc/ci-smoke 1 # green: switch -repos back, redeploy | 940 | gitbay build log cmc/runner-scratch 1 # green: remove -repos, redeploy, delete the scratch repository |
| 938 | #+end_src | 941 | #+end_src |
| 939 | 942 | ||
| 940 | *Do not deploy an isolating runner to a host that has not been | 943 | *Do not deploy an isolating runner to a host that has not been |
| @@ -950,8 +953,6 @@ management and =ReadWritePaths= for podman's store under | |||
| 950 | make read-only. Those paths are prefixed =-= so they are ignored when | 953 | make read-only. Those paths are prefixed =-= so they are ignored when |
| 951 | absent: the drop-in installs on unprepared hosts too, and a unit that | 954 | absent: the drop-in installs on unprepared hosts too, and a unit that |
| 952 | refused to start would stop every build. | 955 | refused to start would stop every build. |
| 953 | The nightly canary on =cmc/ci-smoke= only runs if the runner's =-repos= | ||
| 954 | names that repository too; a scoped runner claims nothing else. | ||
| 955 | =gitbay-runner-prune.timer= prunes unused images weekly, as the runner's | 956 | =gitbay-runner-prune.timer= prunes unused images weekly, as the runner's |
| 956 | user: rootless storage belongs to that user, and root's prune would not | 957 | user: rootless storage belongs to that user, and root's prune would not |
| 957 | see it. An unpruned image store on a 40GB host is a slow outage. | 958 | see it. An unpruned image store on a 40GB host is a slow outage. |
deploy/gitbay-runner.override.conf +5 −3
| @@ -34,9 +34,11 @@ After=gitbay-runner-egress.service | |||
| 34 | [Service] | 34 | [Service] |
| 35 | # The runner polls as a non-admin account with a runner-scoped key, and | 35 | # The runner polls as a non-admin account with a runner-scoped key, and |
| 36 | # claims only the repositories that key is attached to (`repo runner | 36 | # claims only the repositories that key is attached to (`repo runner |
| 37 | # add`): krz/gitbay and cmc/ci-smoke. The attachments are the boundary, | 37 | # add`): krz/gitbay, krz/hutch, krz/keycask, krz/orgo, krz/skunky-art |
| 38 | # so ExecStart names no -repos. cmc/ci-smoke is the nightly isolation | 38 | # and cmc/cleberg.net. The attachments are the boundary, so ExecStart |
| 39 | # canary; keep it attached or its scheduled build waits forever. | 39 | # names no -repos. To validate a runner change, create a scratch |
| 40 | # repository, attach this key to it, and run with -repos naming only | ||
| 41 | # that repository until the change is proven (Admin wiki, CI runner). | ||
| 40 | # | 42 | # |
| 41 | # Two layers of resource caps. MemoryMax and CPUQuota bound the unit — | 43 | # Two layers of resource caps. MemoryMax and CPUQuota bound the unit — |
| 42 | # the runner and every build together — which is what keeps the forge | 44 | # the runner and every build together — which is what keeps the forge |