Elevated response time in the Agent API

Incident Report for Buildkite

Postmortem

This was part of a wider incident and the full write up is available here: https://www.buildkitestatus.com/incidents/txxkzf4r262c

Posted Jan 17, 2025 - 03:21 UTC

Resolved

Incident is resolved as we no longer see elevated latency issues after applying the necessary mitigations.
Posted Jan 07, 2025 - 20:38 UTC

Monitoring

We are observing latency is returning to normal levels after the mitigation and we are actively monitoring it.
Posted Jan 07, 2025 - 20:12 UTC

Identified

We applied some mitigations and are observing improvements in the elevated latency but continuing to investigate this further.
Posted Jan 07, 2025 - 19:54 UTC

Update

We are seeing elevated latency with Agent API and team is investigating the issue.
Posted Jan 07, 2025 - 19:18 UTC

Investigating

We've spotted that something has gone wrong. We're currently investigating the issue, and will provide an update soon.
Posted Jan 07, 2025 - 18:49 UTC
This incident affected: Agent API.