Some jobs not dispatching

Incident Report for Buildkite

Resolved

Between 2026-02-12 01:33 and 04:49 UTC, an elevated error rate in job dispatch was observed. This affected a subset of jobs. Affected jobs were unable to be dispatched during the disruption window, but were eventually dispatched due to automatic retries. A subset of pipeline step uploads were also affected; the agent would check for successful step upload over a 5-minute period before timing out.

This occurred due to a bug in a configuration change that resulted in some Sidekiq instances not receiving all required database connection configuration. The configuration change was subsequently reverted, with further investigation underway.

Posted Feb 12, 2026 - 01:33 UTC