Build notifications not sent for 15 minutes
Incident Report for Buildkite
Resolved
On 2020-08-27 for fifteen minutes between 01:40 and 01:55 UTC, some build notifications were not sent.

Also during that time windows some trigger steps were incorrectly marked as failed even though the pipeline they triggered had passed.

Our automated monitoring alerted us to the issue, and we traced it to a concurrency issue in a recent deploy, which we rolled back.

We have since made improvements to the error handling around this condition.
Posted Aug 26, 2020 - 15:40 UTC