Data Centre Power Failure Update

Over the past few days since the power failure in the hosting comapnies data centre in Leeds I have been monitoring all of my clients web sites especially those that rely on MySQL databases as well as my own and everything appears to be stable although some sites may be slow to load, due to the massive amount of traffic flowing in and out of the data centre in the aftermath of this event.

From the information provided by the hosting company responsible for the data centre the power failure was caused by a fault in a voltage sensing device which triggered an alarm and the power system then shut down power in order to protect the system and then the UPS power supply kicked in 9 seconds later and power was restored.

The problem is that computers don’t like it when you shut off the power supply, and when you switch the power back on they don’t just carry on as if nothing happened, someone has to push the power button and hard reboot the machine, in this case the system admins had to hard reboot 15.000 servers, which has to be done in a certain order and is not a trivial task by any means.

It is now 5 days since the power failure and there are still many servers still down and customers sites offline and the system admins are still working to get them online, which clearly demonstrates the gravity of this event and its effect, fortunately all my sites and my clients were back online within hours, so its impact was minimal in that respect, I can only apologise for any inconvenience this caused. 

Posted in Web Hosting Services.