Jobs failing to run

Incident Report for Buildkite

Resolved

The system appears to have caught back up to real-time, and load has returned to normal. Apologies for the interruption.

Posted Jul 08, 2020 - 19:58 UTC

Monitoring

A fix has finished deploying and we are seeing the system start to catch back up to real-time.

Posted Jul 08, 2020 - 19:23 UTC

Identified

We believe we have identified the cause and are deploying a fix. The issue is also blocking webhook processing, but no payloads have been lost and the relevant actions will still occur once the system returns to health.

Posted Jul 08, 2020 - 19:07 UTC

Investigating

We are currently seeing an issue preventing jobs from being accepted or run by agents.

Posted Jul 08, 2020 - 18:24 UTC

This incident affected: Job Queue and SCM Integrations.