Elevated Latency on Notifications
Incident Report for Buildkite
We continue to implement further mitigations to improve notification latency
Posted Feb 02, 2023 - 21:53 UTC
Latency is now negligible for all notification types. We are continuing to actively monitor the situation.
Posted Feb 02, 2023 - 21:03 UTC
We continue to see notification latency improve, and we are continuing to actively monitor the situation
Posted Feb 02, 2023 - 20:35 UTC
We're seeing notification latency improve, but we continue to proactively throttle some notification types
Posted Feb 02, 2023 - 20:00 UTC
We are seeing elevated latency of up to an hour on step-level statuses. These have been deprioritized in order to manage latency across other notification types.

We continue to investigate the underlying root causes.
Posted Feb 02, 2023 - 19:30 UTC
This incident affected: Notifications (GitHub Commit Status Notifications, Email Notifications, Slack Notifications, Webhook Notifications).