Resolved. No outages. Normal performance restored for WebUI. All remaining traffic cleared.
Posted Sep 01, 2021 - 10:21 PDT
We are continuing to monitor the situation as traffic continues clearing from our background job queues.
Posted Aug 31, 2021 - 13:26 PDT
The Coveralls API and Web UI are now available with no reported outages. Traffic is still high in background queues so performance remains degraded in terms of build times. Users can continue submitting jobs to the API, but build pages may continue displaying the "Data Tables" error for some time today.
Posted Aug 31, 2021 - 07:48 PDT
We are experiencing intermittent outages with Web UI and API. We are continuing to address these issues.
Posted Aug 30, 2021 - 15:46 PDT
Some customers are reporting a service outage. We are monitoring.
Posted Aug 30, 2021 - 12:48 PDT
We have temporarily increased processing resources on all background jobs to speed clearing. Queues are now clearing at a faster-than-normal rate.
To assist clearing, we have temporarily paused processing on ten (10) repos with an excessive number of jobs in queue and have notified the associated subscription email for each project. If believe you may have an affected project, you can check the email associated with your subscription; you will have already received a message from firstname.lastname@example.org.
Affected projects will be un-paused and resume processing when general traffic has cleared by 80%. In the worst case, they will be unpaused at the end of the US PST working day, at which time they'll have relatively unfettered resources and should process quickly thereafter.
Posted Aug 30, 2021 - 11:31 PDT
We are continuing to monitor for any further issues.
Posted Aug 30, 2021 - 10:44 PDT
We are experiencing degraded performance in the Coveralls Web UI this morning due to an unusually high number of jobs in background processing queues.
We are monitoring and will manually clear queues where possible. To do so, we may pause some projects with excessive numbers of concurrent jobs and postpone their processing until most general traffic has cleared.
We will notify customers with projects affected by this procedure, so you needn't contact support to check the status of your project unless you receive an email from us.
All other projects will return to normal performance asap.
Posted Aug 30, 2021 - 10:39 PDT
This incident affected: Coveralls.io Web and Coveralls.io API.