24 Aug 2016
21:30 UTC
Paul Cammish
All affected servers show now have returned to normal operation.
24 Aug 2016
21:29 UTC
Paul Cammish
At 21:56, our alerting systems identified a problem with the Cloud Server platform. On investigation, this was found to be head 25 which had crashed unexpectedly. Affected customers will have seen their servers disappear and reappear around 20-30 minutes later after a reboot.
This is unfortunate as a large majority of the Cloud Servers on this head were moved from head1 earlier in the week which had been experiencing issues, and head 25 (at the time) was chosen as a replacement due to its high reliability. Therefore, there is a significant chance that Cloud Server customers who were affected by the issue earlier in the week will have also been affected by this outage.
Our engineers will be investigating this in detail as a priority, as it is possible that we have uncovered a bug somewhere in the virtualization or kernel which is being inadvertently triggered by one of the guests.
We will be directly contacting the customers affected by both this and the previous outage on head 1 in the next few days with further information.