Some Cloud Servers may have problems writing to disks.

Expected resolution: 29 Jun 2017, 16:00 UTC
Return to issues
Issue status: Resolved Date:

29 Jun 2017
21:56 UTC

Posted by:

Peter Taphouse

As of around 14:00 today, we've started getting alerts that some Cloud Servers are experiencing problems with disk IO for one of the storage pools on tail56.

This will manifest as either very slow disk throughput on SATA/SSD storage, or simply problems accessing files, and may cause processes or servers to crash, and affected servers may fail to restart.

All affected disks are being migrated to other tails at the moment, and affected servers should return to normal as they are moved.

We will will update this post as soon once there are further updates.

Update at 14:51 - So far 25% of the disks have been migrated

Update at 15:18 - Over 50% of the disks have now been migrated.

Update at 15:44 - The remaining 30% of disks are being migrated.

Update at 16:57 - There are now only 3 disks left to move which are being migrated now - we expect these to complete shortly, and we are manually checking for any affected machines which are not responding to pings.

Updated at 22:55 - All disk migrations completed before 17:15 and checks on any affected VMs are complete.

Issue status: Investigating Date:

29 Jun 2017
15:59 UTC

Posted by:

Paul Cammish

As of around 14:00 today, we've started getting alerts that some Cloud Servers are experiencing problems with disk IO for one of the storage pools on tail56.

This will manifest as either very slow disk throughput on SATA/SSD storage, or simply problems accessing files, and may cause processes or servers to crash, and affected servers may fail to restart.

All affected disks are being migrated to other tails at the moment, and affected servers should return to normal as they are moved.

We will will update this post as soon once there are further updates.

Update at 14:51 - So far 25% of the disks have been migrated

Update at 15:18 - Over 50% of the disks have now been migrated.

Update at 15:44 - The remaining 30% of disks are being migrated.

Update at 16:57 - There are now only 3 disks left to move which are being migrated now - we expect these to complete shortly, and we are manually checking for any affected machines which are not responding to pings.

Return to issues

Issue still not addressed? Please contact support.