MySQL Outage
As many of you had noticed we had an issue today with the database servers. We are still investigating to isolate the exact cause. Outage was about 22 minutes.
Essentially the mysql cluster that spends it time lazily hovering around 5% usage filled up a table_cache, started writing to disk at the rate of 3gb/minute, and then when it ran out of space for temp files, fell over. The team at Firehost.com was johnny on the spot and were working to restore service within seconds of the alarm.
Future System Upgrades
We were saving this news for later, but this seems to be an ideal time to share.
We have been working with Firehost, and contracted system experts to design an entirely new, high performance and highly scalable vps-cloud server architecture that will move into a testing phase shortly after we move datacenters (March 26th) .
Brand new hardware, in a brand new load balanced vps-cloud configuration, in a new datacenter that will increase performance and reliability 10-fold for customers as well as give us the solid base to scale and grow our business.
We have mastered the art of managing WordPress for our customers, our next trick is delivering the performance at scale.
Customers: look for a detailed update and notices on the datacenter move by blog post, email, and your dashboard soon.