Performance problems on the Agent API

Incident Report for Buildkite

Resolved

Everything is looking fine and dandy!
Posted Mar 08, 2016 - 00:35 UTC

Update

We've upgraded our main database cluster with more memory and processing power (by 8x). We'll be rolling out more API servers throughout the day. But for now we're seeing good numbers from our reports showing that the Agent API should be back to it's usual self.
Posted Mar 07, 2016 - 23:56 UTC

Monitoring

A fix has been implemented and we are monitoring the results.
Posted Mar 07, 2016 - 23:29 UTC

Identified

We're now working on rolling out fixes to our performance problems from this morning. We'll keep this status entry updated with how we're tracking.
Posted Mar 07, 2016 - 23:14 UTC

Investigating

We're getting reports of timeouts in agents trying to access our Agents API. As a result, builds are taking longer than usual to get scheduled. We're investigating the problem now and trying to resolve it ASAP.
Posted Mar 07, 2016 - 21:59 UTC
← Current Status Powered by Atlassian Statuspage