Build failures and decreased performance
Incident Report for Buildkite
Resolved
We identified a routine database index creation task which created unexpected problems, and caused problems with build creation and job allocation. The task was identified and stopped, and performance returned to normal. We'll continue to investigate how the task caused unexpected problems, and ways of preventing this from reoccurring.
Posted Dec 02, 2019 - 02:52 UTC
Monitoring
A fix has been implemented and we are monitoring the results.
Posted Dec 02, 2019 - 02:17 UTC
Identified
We're investigating issues with build creation failures and agent job allocation.
Posted Dec 02, 2019 - 02:13 UTC
This incident affected: Web, REST API, and Job Queue.