29 Nov 2016
14:20 UTC
David Edwards
The small number of VM owners affected have been contacted, so resolving now .
27 Nov 2016
02:49 UTC
Chris Cottam
Those discs that discs that we've been able to transfer off have been migrated to other tails.
There are still some disks of an unknown state
27 Nov 2016
01:50 UTC
Peter Taphouse
The copy from the damaged array is taking longer than I had estimated, though it looks like it should be finished and we'll be back running by 3am.
There are a small number of discs that we have been unable to recover and we will be directly contacting you this evening if it affects you.
26 Nov 2016
23:19 UTC
Peter Taphouse
Apologies for the lack of updates.
Two thirds of the machines on this tail were live-migrated off to other tails earlier this evening.
There are currently 25 virtual machines with disks still affected by this issue.
The RAID card in the tail malfunctioned rejecting three drives simultaneously - we believe we have now managed to re-assemble the array, but are unsure of the state of the filesystems on it currently. We are keeping the machines offline while we try to copy the data off to other tails.
26 Nov 2016
12:46 UTC
David Edwards
We're aware of an issue on tail 19 in Manchester. Affected clients will see reduced I/O performance, or file system access issues. Just looking at moving affected clients to another tail.