Wednesday, January 14, 2015

Network Issues

There is a network issue at the data center. Engineers are currently working on the issue, and I will post more information on their progress as it becomes available.

Thank you for your patience.

Update 8:47pm PST I haven't been able to pry much information on what's going on from the techs at the data center, but I did get this much:
Our team is currently working with engineers to correct an I/O degradation issue that is occurring during an upgrade of the Reston, VA data center cluster system software. We hope to have a resolution shortly, and appreciate your patience as this issue is addressed and resolved.

--- UPDATE ---

We are still making progress as our engineers correct the issues experienced during the upgrade process, and to bring affected customer environments back online. Again, thank you for your patience during this time.
So in a nutshell, they were upgrading the network and something went wrong. likely they had to power the entire system down to do the repairs, then bring it back up again, a little at a time. Websites run on PHP or HTML are displaying correctly, but any sites that require a database connection/MySQL (WordPress, forums, our support system) are down. I've tried restarting the MySQL server, but they have everything locked, so there's nothing more I can do.

The engineers are working on the problem, and it shouldn't be too much longer. Please check this status blog for updates. This blog should always be your first stop when there's a major server incident. Thank you for your patience.

Update 9:27pm PST  MySQL has been successfully restarted. Our support ticket system is back up. All account databases are being checked and repaired as necessary.

Update 9:50pm PST All databases have been checked and repaired as necessary. If you notice any database errors, please submit a support ticket.