Which Cloud Provider?

We created this page because we’ve found ourselves constantly running through the pros and cons of physical versus virtual servers and laboring through the benefits and pitfalls of the plethora of no-name virtual hosting providers out there. We now save ourselves and our clients time by pointing them to this page.

As a result of our trials and tribulations over the years, we’ve come to rely on three different cloud providers for a variety of reasons that we will outline below. Each has strong points and weaknesses. We hope our thoughts on this topic can aid in the decision making process.

The three cloud providers we routinely utilize and evaluate for clients are: Rackspace, Linode, and Amazon. This is because we’ve found these three providers offer a good mix of price, performance, reliability, and features. Although competitive offerings evolve, we’ve evaluated and passed on other providers for tending to fall short in these areas and generally recommend passing on lesser names too. How do the top three providers stack up?

Ease of use

It isn’t terribly challenging to get up and running on Amazon EC2 for a technically savvy and experienced system administrator. However, it’s a big stumbling block for the neophyte with a website many consider cryptic and challenging to navigate and “option overload” including features that are hard to decipher based on their names. There are also a lot of “gotchas” when running in EC2 that even seasoned administrators won’t be familiar with.

Rackspace Cloud has a very streamlined process for signing up and quickly launching servers. The worst part about Rackspace is a phone verification requirement when initially signing up for a new account. We’ve ended up playing phone tag with them for a day on occasion. Once past that, click through a few screens with clear and concise options and a new server is up and running. The Rackspace Cloud web console has radically improved over the past few years, evolving into the fully featured tool one would expect from such a large hosting company.

Linode falls somewhere between Rackspace and Amazon. It’s definitely easier to get started on than Amazon, but is a bit more low-level than Rackspace, and has many options which could be intimidating (and potentially dangerous) to someone who is not a system administrator. One minor frustration with signing up with Linode is that one must purchase a server before an account can be created to poke around. With Rackspace and Amazon, one can sign up and browse through their system to get the feel of things before committing to a purchase.

Reliability

The Rackspace Cloud has had one well known public outage (which also affected Slicehost). That’s fewer than Amazon and certainly fewer than the many fly-by-night VPS providers whose outages would never be known by many. The one Rackspace outage did not affect any of our servers – our oldest server at the time had been running uninterrupted for nearly three years. We have seen an occasional glitch with individual systems, but we’ve never seen a significant outage that impacted multiple systems.

Linode, on the other hand, has a now-distant history of several public outages per year. They have had outages ranging from losing a number of nodes to losing entire data centers for some number of hours. That said, most of this seems to have been put behind them in the last few years. Not to jinx ourselves, but other than one recent Denial-of-Service attack on their Dallas data center, we’ve never experienced an unplanned outage on linode.

Amazon EC2 appears to have relatively few major outages, but when they have one, it’s bad. Almost everyone is aware of the April 2012 outage that impacted the entire East region and took out several major social sites for the better part of a day. This seems to be par for the course for EC2 – a major outage every year or so. They have a disadvantage in that, as the largest cloud provider, any outage impacts a larger portion of the Internet and is more widely publicized.

Beyond major outages, we’ve found individual machine reliability to be somewhat lower at Amazon. It fits the deployment model, in that Amazon tends to be for large redundant deployments and Linode and Rackspace seem to be utilized more for small or single-server deployments where reliability is more critical.

Performance

Rackspace and especially Linode servers are significantly faster than Amazon’s EC2 servers and most of the tier 2 VPS providers we have experience with. It’s noticeable immediately upon login to a smaller virtual server (e.g., 512MB). The key difference between Rackspace and Amazon is that Rackspace CPU cycles are minimum guaranteed and Amazon’s are maximum. Rackspace and Linode servers will also burst if CPU is available on the physical box which, in our experience, it normally (but not always) is. Further, Rackspace and Linode virtual servers have substantially more local disk throughput than EC2 instances. Keep in mind that, when running tasks that require performance, one must take disk I/O into account when pricing out cloud solutions. Otherwise, more or larger instances may be needed to run the application – which could easily change the apparent pricing advantage between providers.

Although both easily trounce Amazon, there’s still a large discrepancy between Linode and Rackspace. Linode readily outperforms Rackspace in most workloads. Further, both Linode and Rackspace now offer SSD storage which easily outperforms Amazon’s network-attached EBS volumes. Amazon has recently rolled out SSDs on newer instance types, but these are only for ephemeral storage which can be harder to safely take advantage of. Note that one possible advantage in Amazon EC2 demonstrated well in the charts below is very predictable performance, which can be a feature in some cases.

Persistent

This is no longer the major factor in decision making it was a few years ago, since Amazon long-ago introduced persistent storage with EBS system volumes. However, it’s worth mentioning that, on Amazon, one can either have blazing-fast SSD local storage or storage that survives reboots, but not both.

This means if a server crashes, or it’s powered down (but not merely on a reboot), everything on disk is lost. People come up with sometimes elaborate workarounds. For example, some replicate data across numerous instances in an attempt to prevent data loss.

Another complicating factor is that, since EBS volumes are network-attached, they are both throttled for throughput and, unless the EBS optimization feature (another extra cost option) is enabled, are competing with normal network traffic for bandwidth that is limited by instance type.

EBS is also an extra cost on top of base server rate at $0.05 per GB month and $0.05 per million I/O requests. One can also pay even more for provisioned IOPS (PIOPS) EBS volumes that will perform well, but not nearly as well as SSD.

As mentioned above, both Rackspace Cloud and Linode have persistent local disk storage on SSD.

Some readers may be thinking, “Who in their right mind would use Amazon?” See Features below.

Pricing

Cloud pricing almost always beats dedicated server hosting for small projects – frequently by an astonishing amount. We rarely recommend dedicated hosting unless custom, high-performance, or very large RAM or disk capacity hardware is needed. Every time we’ve run the price comparison between Linode and tier 2 cloud providers, we always end up thinking “Who would pay more for a lesser provider?!” After managing large data centers that demand multi-year contracts with complicated pricing schemes for every imaginable item, it’s easy to appreciate having no contract and a pricing structure that couldn’t be simpler.

Although this has been different in the past and certainly may change again in the future, Linode is substantially cheaper for most uses cases than Rackspace, and both are cheaper than Amazon when taking performance into account. Both Linode and Rackspace provide a generous amount of free bandwidth which makes pricing simpler to calculate. Both now charge about the same for their own internal backup system.

Run the numbers and perform some benchmarking. Both almost always beat Amazon in bang for the buck.

Backups

Rackspace offers file-level backups with compression, de-duplication, and encryption based on Jungle Disk. It requires installing and configuring a client. Backups can be manually initiated or scheduled hourly, daily, or weekly. This is a nice feature for small companies that don’t need or can’t justify the complexity of a a “real” backup solution. Since the agent runs on the server, it can backup encrypted partitions or strange filesystems. The cost is $10 per month plus storage at $0.10 per GB and bandwidth at $0.12 per GB

Linode allows four backup slots. Three slots auto-rotate and one slot is reserved for manual backups. These backups are file system level backups: disk partition(s) are mounted by Linode and backed up. If a server utilizes anything but Linode-managed ext3/ext4 partitions, they won’t work. Encrypted volume? Forget about it. We find it a bit creepy to have volumes with sensitive data mounted by someone else – not that other cloud providers can’t do this.

Pricing is tiered by Linode plan size and is currently 25% of the monthly rate, or $10/mo for a 4GB server.

With Amazon, one can take EBS snapshots and backup to S3, or roll a custom solution with S3, EBS, or something else. S3 snapshots are a very simple solution (especially with our EC2 Snapshot script). However, this is not a trivial solution to recover a few files from. Recovery requires creating a volume from the snapshot and mounting it. S3 has become inexpensive at $0.03 per GB and EBS isn’t much more at $0.05.

There is no fee beyond S3 or EBS costs, but administrators will be managing them manually as they are not automated with a few clicks like Linode or Rackspace. Keep this in mind, as it’s a compelling reason for many budding system administrators to procrastinate on backups – a potentially fatal mistake.

Operating System Selection

The only operating system we miss when using Rackspace is OpenSolaris (thanks for destroying it, Oracle!). In addition to the common flavors of Linux, they also offer Windows Server 2008 and 2012 in both 32 and 64 bit. It’s not a comprehensive selection, but all the Linux and Windows spins we would consider (and some we wouldn’t!) are available. They also offer a “pv-grub” solution that can be requested, where a custom kernel and distribution can be uploaded. It’s not for the faint of heart or weak of skill.

Linode offers a more limited selection of Linux distributions compared to Rackspace, but still offers anything we would reasonably consider. There are no licensed RedHat options and Windows is not supported. It is called Linode, after all! They do support pv-grub to enable running other distributions. We’ve even authored an article on how to use pv-grub to encrypt an entire Linode.

In Amazon’s EC2, there are an infinite number of Amazon Machine Images (AMIs) to pick from and a customer Xen image can be uploaded. However, this provides a potentially overwhelming number of choices for a novice user, and maintaining a own custom Xen image is something that shouldn’t be undertaken without serious consideration of the additional maintenance one will be committing to.

SLA and Support

The Rackspace Cloud is backed by Rackspace’s Fanatical Support™ for 24x7x365 technical support at no extra cost. Support is extremely responsive and more skilled than average. Rather than just being an option, one is actively encouraged to chat with a support engineer.

With Amazon EC2, one is largely on their own unless they choose to pay the greater of $100 per month or 10% of EC2 costs for Business level support. As a company scales up, it pays more in support costs even if not utilizing support more. For an experienced system administrator this will probably matter less as it won’t be utilized often. However, for companies lacking administration skills, this can be a great resource.

Linode support falls somewhere in the middle. It’s not really fanatical, but it’s there and they’re competent. Support is handled through an internal ticketing system like Rackspace, but they don’t encourage starting a live chat with technical support.

Features

We haven’t yet come up with a reason to choose a no-name, fly-by-night, or tier 2 cloud provider. However, there are some unique reasons to consider Amazon EC2 over Rackspace or Linode. Below are a few that make the decision for us. Note that we’re not including other extremely useful Amazon features that can be utilized from other cloud providers (e.g. SQS, CloudFront, S3, Route 53). We’re also skipping services that can be useful, but aren’t significant enough to drive a decision for us (e.g. Elasticache, Redshift).

Virtual Private Cloud (VPC). Launch all servers in a completely private network with full control over all network ingress and egress. This might be the most important feature we rely on for deploying more than a few servers.

Elastic Block Storage (EBS). Scale storage almost infinitely at $0.05 per gigabyte month and $0.05 per 1 million I/O requests. Automate snapshots of these volumes. Provisioned IOPS (PIOPS) are also available at an extra cost for impressive performance. Rackspace has Cloud Storage, but it’s expensive and not as flexible. Extra storage can be added to a specific Linode, but it’s extremely expensive ($1 per gigabyte) and very limited. It would be a challenge to deploy a 10TB MySQL instance on Linode, but not on Amazon.

Elastic Load Balancing (ELB). As with EBS, Linode and Rackspace offer similar solutions, but they’re more expensive and not as flexible. This can be both external (publicly addressable) or internal load balancing for services inside a VPC.

Relational Database Service (RDS). Deploy redundant databases in multiple data centers with a few clicks in a GUI. Automate replication and backups. If utilizing an open source database, this is all for only a slight upcharge over self-managing the same size instances.

Identity and Access Management (IAM). An unmatched system for provisioning users and groups with an impressive fine-grained access control not found anywhere else. For a large team, this is a significant benefit. For a single system administrator, it’s almost worthless.

Auto Scaling. Start or terminate instances based on definable conditions. Scale a web cluster up during peaks, and back down at night. Rackspace has a similar but limited offering.

It’s worth looking at Amazon’s list of services and deciding if hosting with Amazon gives provides a strategic advantage that can’t be duplicated with Rackspace or Linode. Remember that some of these services (such as load balancing) can be replicated in any cloud, but involve more effort to deploy and maintain.

One negative of Amazon we’ve stumbled over more than once is the lack of a system console. Both Rackspace and Linode provide a javascript console that will enable both seeing errors logged only to console and performing valiant resuscitation efforts on a dead server. In sharp contrast, Amazon has no such feature. One may occasionally be stuck with an unbootable server that can’t be saved. As with Linode (but not Rackspace), a system disk can be mounted on another server for debugging purposes, but it’s not quite the same. Linode is king for consoles and server recovery options.

When deciding between Linode and Rackspace, we find that Linode has more low-level system features than Rackspace, but the reverse is true for major features like auto-scaling and system bootstrapping. The Linode administration system is a bit more powerful – self-provision extra IPs, extra RAM, extra disk space and have it up and running immediately. Tweak disk space allocation and customize the partitioning scheme, which can’t (easily) be done on Rackspace. System consoles can be accessed via ssh without logging in to the web console – very handy when a critical system is dead and the Rackspace‘s web console is lagging.

Summary

One fallacy we run into frequently with people’s perception of operating in the cloud is that “the cloud” is magically fault-tolerant. This is far from the truth. In fact, cloud servers can be less reliable than physical servers. When running in the cloud, one should begin planning with redundancy in mind. Even simple websites should run with two systems either in an active/active or failover configuration. Beyond that, if uptime is critical, begin thinking about spreading systems across multiple facilities (“availability zones”, “regions”, “data centers”), using multiple cloud providers (for example, one primary and one standby). Make sure system backups are not in the same data center – or even better, not even with the same provider

For example, one of our recently deployed configurations involved heavy redundancy and two providers. This configuration runs active/active web application servers and primary and hot standby PostgreSQL servers running at Rackspace. The entire infrastructure is backed up to Amazon S3. This way, if there were a complete failure at Rackspace, recovery can happen quickly from backups stored at Amazon. Note that Rackspace is unique among the three for not allowing the selection of where individual nodes are placed, so this is more critical at Rackspace than at Linode or Amazon.

Hopefully this article has not lead to information overload. If so, we’ve come up with a simple ranking to assist in selecting between one of the three cloud providers we recommend:

We hope this has been helpful – let us know if we can assist with a cloud deployment. In the spirit of full disclosure, we have become affiliates for both Linode and Rackspace since authoring the original article. We do make a small amount if you sign up for either one through our links or banners. If this article was useful, please help us out by signing up through one of our links.