Server down

The server is down with an "Internal Server 500 error", which means something's been misconfigured, although we haven't configured anything recently. I have an emergency ticket in with the data center, and will update the progress as soon as I have any news.

12:58pm (PT): Data center is still working on the issue, and have escalated the ticket. The server has been rebooted.

2:34pm (PT): I requested a progress update at 1:58pm, they have yet to reply. I will continue to monitor/prod until this is resolved.

4:12pm (PT): Still no word back, other than "We are still working on this." I've escalated the ticket, gone over their heads and submitted a ticket directly with the Engineering dept of the Data Center.

5:11pm (PT): FIXED! All services are back up and running! The only lagging I'm seeing is when the site's DNS is 100% external (meaning you have another company running your DNS).

This is never recommended, but what *is* recommended is having backup, or secondary DNS, with us as primary. That way, you're covered, no matter what. This will also keep your email flowing, etc.

There's not much that can be done when there's a hardware failure (as this appears to have been, but the specifics are unconfirmed), and we are pushing the data center for an answer as to what happened. The plan to switch data centers is being moved up, and customers will be contacted as their sites are moved. We plan to have "cross-over time" meaning the old server won't be shut off until all DNS is fully switched over to the new server.

We expect no downtime (further hardware failures on the current server not withstanding). The new server company advertises "ZERO downtime during hardware failures." It's something I hope to never have to put to the test, but no technology is ever 100%.

We apologize for this downtime, and are constantly striving to provide better service to our valued customers. We appreciate your patience.