Agent API Connection Issues
Incident Report for Buildkite
Resolved
Graphs have returned to normal, and jobs and agents should be running again as normal. Please reach out to support if you have any stuck jobs
Posted Oct 16, 2019 - 12:00 AEDT
Monitoring
We've removed the database that was having the issue, and graphs are starting to look better now. We're still all hands on deck, and keeping an eye on systems.
Posted Oct 16, 2019 - 11:22 AEDT
Update
We've identified an issue with one of the databases responsible for storing log chunks from agents. We've started the process to decommission that database, which we hope will bring things back to normal.
Posted Oct 16, 2019 - 10:59 AEDT
Update
We are continuing to investigate this issue.
Posted Oct 16, 2019 - 10:42 AEDT
Investigating
We've received reports that some agents are loosing connections and builds are hanging. We're looking into what the issue might be.
Posted Oct 16, 2019 - 10:32 AEDT
This incident affected: Agent API and Job Queue.