Resolved -
We have seen a full recovery of service, and have a good understanding of the underlying cause. We will publish a post-incident review next week.
Feb 27, 02:30 UTC
Monitoring -
We've seen recovery for the remaining subset of customers. We will continue to monitor.
Feb 27, 00:57 UTC
Investigating -
We're seeing ongoing latency impact across for a subset of customers. Some customers are seeing signs of improvement, but we are continuing to investigate the issue.
Feb 26, 23:41 UTC
Monitoring -
We're seeing signs of recovery and will continue to monitor.
Feb 26, 19:36 UTC
Identified -
Some customers are experiencing increased latency for jobs being assigned to agents. We have identified the cause and are working on mitigations.
Feb 26, 19:10 UTC
Resolved -
Response times have returned to normal. This incident is now resolved.
Feb 26, 02:44 UTC
Monitoring -
We've increased the compute available to the secrets service, and have seen response times return to normal levels.
Feb 26, 00:53 UTC
Investigating -
We're observing increased latency on secrets endpoints for a subset of our customers. We're currently investigating and will provide status updates as they become available.
Feb 26, 00:43 UTC
Resolved -
The cause of pipeline upload failures and dispatch latency that was impacting some of our customers has been resolved.
Feb 17, 21:05 UTC
Update -
We're seeing signs of recovery. Our team is continuing to monitor the situation.
Feb 17, 20:47 UTC
Update -
We've identified that this issue is also impacting pipeline uploads for customers that are affected. This is an isolated issue and not all customers are experiencing impact.
Feb 17, 20:30 UTC
Investigating -
Some customers are experiencing increased latency for jobs being assigned to agents. We are investigating and will provide an update in 30 minutes.
Feb 17, 20:09 UTC