09-27-11 | Blog Post
Last Saturday, top SEO company SEOmoz reported unscheduled downtime of their paid web crawler service while hosting a number of their applications with Amazon Web Services (AWS). The web crawler service, also known as Crawl Diagnostics, is a time-sensitive monitoring service that tracks any potential search engine indexing issues with your site. Similar to Diagnostics in Google Webmaster Tools, the service can tell you if a page on your site has a crawl error or detect any malware.
Discount Cloud Computing
While SEOmoz normally pays a fixed rate per hour for their cloud computing services, they also take advantage of a discount billing model called Spot Instances. One major risk attached to this purchasing method is that you can be outbid by another person willing to pay more for the computers hosting your services. Meaning, at any moment, you could be kicked off and forced to rebuild your servers.
And that’s precisely what happened when it came to their web crawler service they offer to customers willing to pay for PRO SEO monitoring and analytic services on their site.
According to Amazon’s Elastic Compute Cloud site:
“The Spot Price changes periodically based on supply and demand, and customers whose bids meet or exceed it gain access to the available Spot Instances.” And by ‘gain access’ they mean completely replace your servers.
“If you have flexibility in when your applications can run, Spot Instances can significantly lower your Amazon EC2 costs.” Did SEOmoz foresee an only-sometimes-available paid service model for their e-commerce business?
Backup and Disaster Recovery
While they claim they haven’t lost any data, they acknowledge the web crawler service will be inaccessible for at least one week while they rebuild and load terabytes of 9 months of data into their NoSQL data store.
Majestic SEO, a SEO company based in the UK, also recently reported downtime on Friday, but their two phase disaster recovery plan first successfully recovered the API used by their premium members and resellers and then the rest of the site for public users, putting them back up within the same day. While they acknowledge the problem was found to be outside their systems, there’s no confirmation that their ‘third party’ could be identified as Amazon.
However, they do state their commitment to higher hosting standards: “We have taken pride in our attempt to provide 24/7 availability, and continue to invest heavily in redundant equipment in order to ensure our customers receive the best possible service,” which is a slightly different approach than attempting to score discount hosting at the customers’ expense.
Their Solution
SEOmoz’s future solution appears to be better payment strategy while continuing Amazon’s Spot Instance pricing plan – they plan to bid slightly higher for their service hosts ($3 instead of $2 per hour, to compensate for the unexpected .51 cent spike in bids). They also plan to distribute hosts across multiple availability zones, meaning they’ll avoid putting all their eggs in one basket this time around in hopes of salvaging at least half of their service hosts in a different region.
They also plan to use a mix of On-Demand and Spot Instance pricing, acknowledging the tradeoff of fixed and higher pricing for better odds of uptime.
Reactions
SEOmoz mentions that Amazon gave them no notice before booting them to make space for a higher bidder. However, that seems to be the nature of the discount pricing model, and by no means a result of deception. SEOmoz wanted the cheap prices at the expense of reliablility, and they should have taken note of potential consequences beforehand – as stated on Amazon’s site, “If you’re running Spot Instances and your maximum price no longer exceeds the current Spot Price, your instances will be terminated.” Amazon does offer a status monitoring service that can track termination in their management console.
In a comment from a member of SEOmoz’s staff, they also dismiss the incident as just “life in the cloud” guaranteeing “very little assurance of anything.” However, not all hosting or cloud computing providers are going to reformat their hard drives and give your space to another tenant without notification. Perhaps a large and impersonal company accepting bids for computing will, but there are a lot of alternative IT vendors out there providing smarter computing options and better customer support.
Although they may be dealing with some server hosting issues, they are still a fantastic resource for SEO tools, and we are big fans of their outstanding PRO member services. And while their efforts in transparency and honest communication are also commendable, SEOmoz could benefit from reconsidering their hosting strategy and provider. Outages and unavailable services (especially for a week’s time) can mean lost business and unhappy customers. If your service is important enough to sell at a premium, isn’t it important enough to support with quality, reliable hosting services?
