Yesterday I got a few calls from clients reporting that they could not send or recieve emails and that their web sites were down, when I checked I found this site was offline as was my hosting store Source Host and the system status page was also offline along with all of my clients web sites every site I tried to browse returned a 404 error.

When I checked the hosting company’s own system status page it stated that they had suffered a power failure mid afternoon within the data centre which had the effect of shutting down all the machines in one of the halls within the data centre, the system administrators were quickly on hand to start working restarting all the servers resolving any issues.

By late afternoon the admins had got the mail servers up and running and email traffic was begining to flow, I started to recieve emails sent 2 to 3 hours earlier, fortunately no email data had been lost during the time the mail servers went dark, but the webmail interface has been disabled while they rebuild the servers from backups.

I spent the rest of the evening monitoring the situation keeping an eye on the system status page and the companies FaceBook page for updates and slowly my clients sites started to come online as the admins continued working to restart all the servers and get everything back online, and get the system stable again as quickly as possible.

After a while all my clients sites that do not rely on a MySQL database started to come back online slowly mine included, but all the sites that use MySQL databases were still returning 404 or 503 errors, but after a couple more hours the Cube Cart stores came online and then all the WordPress sites, it was 10:30pm before all of my clients sites and my own were back online much to everyones relief.

Hopefully things will become more stable over the next few hours as the system admins get everything under control again.

I apologise for any inconvenience that this may have caused.