31 Jan 2016
15:24 UTC
Bytemark Engineer
At around 3pm today, while tracking down some performance problems, I authorised some testing against BigV's API service. As an unforeseen consequence of this action, the heads and tails located in York were disconnected from our command and control infrastructure.
Customers will have been unable to manage their VMs or create new ones between 3pm - 3:15pm - already-running VMs will have been unaffected, and the issue is resolved now. I'll be performing works over the coming week to prevent a recurrence of the unforeseen failure, as well as fixing a few other problems that the testing threw a bit more light upon.
Apologies for the interruption - as ever, do get in touch via the usual channels if you'd like more information, or if you're experiencing any problems.