High job dispatch latency

Incident Report for Buildkite

Postmortem

This was one of five incidents with a common root cause. The post mortem is available here.

Posted Jun 23, 2022 - 07:39 UTC

Resolved

This incident has been resolved, and the system is performing normally. Storage latency on a core database was elevated, and we worked closely with our vendor to resolve it.
Posted May 07, 2022 - 13:05 UTC

Update

The issue has been identified and service is beginning to return to normal.
Posted May 07, 2022 - 12:30 UTC

Update

We are continuing to investigate degraded queue performance.
Posted May 07, 2022 - 12:08 UTC

Investigating

We are seeing high dispatch latency, we are investigating.
Posted May 07, 2022 - 11:30 UTC
This incident affected: Job Queue.
← Current Status Powered by Atlassian Statuspage