store, control: fail a build whose log stream ended with no outcome !321

merged merged by cmc on 2026-09-07 06:36 UTC · krz/gitbay:reap-on-log-close-179 into main

Discussion

cmc

Second of three for #179.

ReapStaleBuilds keyed only on started_at past a fixed deadline, which is why build 1037 sat running for over half an hour after its runner was restarted out from under it. The server already has a better signal: the runner streams the log over one ssh session and reports the outcome right after closing it. A stream that ends with no outcome following means the runner is gone.

runRunnerLog now records log_closed_at when the stream ends (migration 0046, running builds only). The reaper fails a running build whose stream closed more than StaleLogGrace (two minutes) ago — long enough to outlast !320's thirty seconds of report retries — and keeps the deadline rule for a build whose stream was never seen.

TestReapStaleBuildsAfterLogClosed covers inside and past the grace period, a build with an open stream staying untouched, and marking being a no-op once the build is no longer running. TestRunnerLogMarksStreamClosed drives runRunnerLog with a stream that ends and checks the mark lands while the build stays running for the report.

Stacked on !320.

Ref #179

retargeted from runner-report-retry-179 to main

2026-09-07 05:07 UTC