Internal error spike – No customer impact

Incident Report for Buildkite

Resolved

In analyzing the impact, we have determined that the only impacted organizations are internal to Buildkite, used only for monitoring purposes.

There was no impact to customers in this case.
Posted Jan 24, 2025 - 00:24 UTC

Monitoring

The rollback has completed, which has resolved the error. Any impacted inbound webhooks have now been reprocessed successfully.
Posted Jan 24, 2025 - 00:09 UTC

Identified

We have performed a rollback, and are seeing error rates returning to baseline levels.
Posted Jan 23, 2025 - 23:57 UTC

Investigating

We are observing elevated error rates on webhook ingestion. We're currently investigating the issue, and will provide an update soon.
Posted Jan 23, 2025 - 23:38 UTC
This incident affected: SCM Integrations.