Comparing cloud services for Startups

Every Startup that has services online needs a cloud provider. Startups do not have time to build their own physical server infrastructure. They need to focus on their product or service development. But what cloud to use? There are so many different options, and CTOs do not have time to test each one of them. Maybe this post will help new Startups  to choose between all cloud providers available.

The experience that I had with Playax was not typical, for two reasons: the first was that I have a lot of experience working with cloud. After working at Locaweb for 5 years, and developing software for internal cloud team, I spent one year in my PhD studying cloud services. The second reason is that Playax product is highly dependent from cloud. We are a BigData company. We needed a big infrastructure from day one. Our MVP needed a lot of cloud resources to be useful to our customers. Most of Startups do not need that much infrastructure, at least not before it starts growing fast.

In our case, we did not have funding (neither self-funding) to pay for those many CPUs we needed. We had the chicken-egg problem: to validate the business, we needed to build the expensive MVP (because of cloud costs). To pay for the cloud, we needed to have funding or paying customers.

We were lucky to find many cloud providers giving free infrastructure for us to build and test our product. The first one we used was Rackspace. They have a startup program that gives you US$2000 per month for one years. It is a lot of dollars! But for us, this was not enough. Our application has to process thousands of audio streamings simultaneously and identify what is being played using audio fingerprinting algorithms. So, I had to find out a way to have more free cloud. I discovered that almost all big cloud provider has its own Startup Program. I applied to each one of them and ended up building a multi-cloud solution. Today, I have part of my infrastructure divided into four different providers.

I had the opportunity to test every service and analyze the similarities and differences between them.

The result is the following table, which is my personal opinion about many different attributes. This is not intended to be an exaustive or complete evaluation of all services provided by each provider. It is a high level evaluation based on my experience using them. This evaluation was done along the year of 2014. This means that a lot of things can be changed after that. Services evolve over time (for better or for worse). I did my best to don’t make mistakes or be unfair in the evaluation. However, it is possible that I missed something. So, for example, if I considered that a service doesn’t have an existing feature, I welcome people to correct me commenting this post. Independently from this evaluation, I am thankful to all providers that helped my Startup. Here it is the evaluation table, more stars means better service (from 1 to 5).   

In my comparison, I evaluated these attributes:

• API - how easy is to use the API?
  Almost all services have an open documented API to interact with resources automatically. This is very useful if your system needs to create servers or storage files on demand. Between all cloud providers, the only one that does not have a good API is SoftLayer. The easiest API to use in my opinion were Google and AWS. Microsoft and Rackspace have APIs, but its usage were a little bit more complicated for me.
• Usability - how easy if the control panel and web interfaces?
  The cloud control panel is very important, specially for Startups, which does not have time to go through all documentation and screens. Everything must be intuitive and easy to use from day one. In this requisite, Google and AWS have the easiest to use web interface. Rackspace also has a very easy to use control panel. On the other hand, Microsofts interface is very confusing. The login process can be tricky, sometimes you have to go through many screens before you get to the feature you need. SoftLayer‘s panel is even worse: server creation is very confusing and pricing is a complete mess. There are a lot of pop ups windows and the panel is extremmely slow.
• MySQL and MariaDB - does the service provide out-of-the-box MySQL or MariaDB?
  MySQL is one of the most popular relational free databases, and many Startups need it. Configuring and optimizing a MySQL server from scratch can be a difficult and time consuming task. It is always preferable to have an out-of-the-box solution. All providers except SoftLayer have built-in MySQL compatible database. MariaDB is the open source MySQL fork after Oracle acquired Sun. I was using MySQL version 5.5 and noticed that MariaDB has a lot of performance improvements on version 10. I decided to migrate to MariaDB and my application became a lot better after that. I was lucky that Rackspace had this option. All other providers still don’t have MariaDB.
• $ for startups - How much US$ can you get from the provider in its startup program? How easy it is to get this money?
  Each cloud provider has different programs for Startups.
  • Google initially has a US$20.000 / year program, but recently they updated to a US$100.000 / year (you have only one year to spend all the money). We were indicated by a Googler to the program, so I don’t know how easy it is to get the money if you are not indicated. But the process for us was very simple. After some days you submit the your Startup  in their website, they send you a cupom with credits.
  • Microsoft has two programs called BizSpark and BizSpark Plus. BizSpark gives you US$150 / month for 3 years (US$5400). BizSpark Plus gives you US$60.000 for 1 year. It is not easy to get the Plus. I talked to a lot of people at Microsoft for that, but I could only get the BizSpark approval. Maybe if you are inside an accelerator, will be easier for you. The application for the program is in Microsoft’s  website.
  • SoftLayer has the Catalyst program that gives you from US$1000 to US$2000 per month (US$12.000 or US$24.000 / year). The usage is also limited to 1 year. We had an indication also, so it was easy to get into the program. You will have to justify if you need more than US$1000 / month.
  • Rackspace was one of the easiest to get the resources. They give you US$2000 / month for 1 year. We applied only in their website and after a couple of weeks the credit was available. You will need to put their logo into you home page.
  • AWS was the only one that we could not get credits. In the activate website there are two options: the (1) Self-Starter Package, where the only option is to use their Free Tier usage; and the (2) Portfolio Package, which gives you from US$1000 to US$15000 in credits. To access the Portfolio Package, you need to be in an accelerator.
• Support - how is the cloud service support?
  Related to customer support, Cloud providers are basically divided in two categories:
  1. Higher product prices with better free customer support: these providers focus on customers that need more assistance on basic infrastructure. The support level usually is great for beginners and terrible for advanced users. They have free support for all customers. Rackspace and SoftLayer are in this category. Everything you need, you can open a chat and start talking to an attendant any time. But if your issue is complicated, it will usually takes days, sometimes weeks to be solved (and sometimes they will not solve the issue at all).
  2. Lower product prices and self-service features with payed support: these providers have a better user interface and public API for advanced customers that have knowledge about how cloud computing. These kind of customers usually do not need basic support. They will need support only in cases of specific issues for advanced users. These providers charge for special support like this. Google, Microsoft and AWS are in this category.
• VMs diversity - how flexible are the Virtual Machine configurations?
  The Playax software has more than 20 different components, divided in 7 different server configurations. Every server has different requirements regarding to CPU, RAM and IO usage. The more flexible the cloud server offering, the easier it is to adapt your application with the lowest cost. In my opinion, Amazon and SoftLayer has the more flexible and wide range of Virtual Machines offering (and also Bare Metal servers in SoftLayer). Google has good options, and Rackspace is ok, but with fewer choices. Microsoft is the more limited in terms of Virtual Machine diversity.
• $ storage and Reduced Redundancy- how much costs file storage? Does file store provides a low cost option?
  Our application monitors thousand of radio stations, identifying more than 500.000 music plays per day. Every time we identify a play, we store that piece of music, so customers can listen to it later to audit the play. This means we need A LOT of file storage. Today we have more then 80 million of those 40 seconds files stored. So price is crucial for us. Google, AWS and Microsoft have the cheapest options for file storage. SoftLayer has an offering a little bit more expensive and Rackspace is very expensive.
  Besides pricing, have a low redundancy option for us is relevant. If one or two of those files get lost, this is not a critical problem, so it is worth to pay less for a lower redundancy option. Only  Google, AWS and Microsoft have this, and Google is the cheapest one (US$0,02 / GB)
• Documentation - how good is the service documentation?
  Startups need to be very productive when deploying their products. This means the time you spend on managing your infrastructure must be as minimum as possible, so you can focus on your product development. One of the most time consuming tasks is searching and reading documents. Documentation must be will organized and easy-to-find, as well as brief and objetive. Ideally, services must so easy to use that you don’t need to read any document at all.
  But when you use cloud APIs or don’t want to wait for customers support response, good documentation is necessary. From all cloud providers, only SoftLayer offers a poor documentation. All other services have well organized documentation pages. In my opinion, Google has the best between them.
• Stability / bugs - does the service has bugs? Is it stable?
  This topic is not easy to measure, since you need to use more features to find more bugs. But from the experience I had, Rackspace and Microsoft had some issues. I found two bugs in Rackspace, one regarding to public network provisioning and other related to cloud files deletion. I also found With Microsoft I had a lot of difficult to login the control panel and some bugs when trying to create and manage cloud servers. Regarding to downtime, I had only a brief programmed downtime with Rackspace. I also had a bad experience in the past with AWS, when my servers went offline for a whole day.
• Prices - how expensive is the service, considering what I use?
  Today I have more then 25 servers in the cloud. It is not too much for big companies, but for startups I consider a big number. My servers range from micro 512MB RAM servers to large 16 CPU instances. I also use cloud’s MySQL, Load Balancing, File Storage and have a relevant network traffic (inbound traffic is usually free, but most of my cost is because of the multi-cloud solution, I need to spend extra money to move data between clouds). In pricing comparison, while Google is the cheapest choice, Rackspace is the more expensive (costing 47% more than Google). See table below (numbers in USD):

  server type	qtd	Microsoft		Rackspace		SoftLayer		AWS		Google
  queue	1	13.4	13.4	15.9	15.9	27.6	27.6	9.36	9.36	6.57	6.57
  search	1	37.2	37.2	28.8	28.8	27.6	27.6	18.72	18.72	17.69	17.69
  web	2	37.2	74.4	28.8	57.6	27.6	55.2	18.72	37.44	17.69	35.38
  streamings	6	74.4	446.4	57.6	345.6	55.2	331.2	50.4	302.4	44.48	266.88
  api	8	74.4	595.2	115	920	110.4	883.2	100.8	806.4	88.95	711.6
  web apis	1	143	143	230	230	213.6	213.6	100.8	100.8	70	70
  web apis 2	1		0	57.6	57.6	55.2	55.2	50.4	50.4	44.48	44.48
  worker	1	37.2	37.2	28.8	28.8	27.6	27.6	18.72	18.72	17.69	17.69
  storage	4000	0.0935	374	0.1	400	0.04	160	0.024	96	0.02	80
  requests (M)	3	0	0	0	0	0	0	5	15	10	30
  transfer	520	0.278	144.56	0.12	62.4	0.1	52	0.12	62.4	0.12	62.4
  mysql	1	71	71	86.4	86.4	55	55	64.8	64.8	107	107
  mysql storage	1		0	51.9	51.9	0	0	6.25	6.25	100	100
  			1936.36		2285		1888.2		1588.69		1549.69

• Choice with $
  Considering that the cloud provider gave me free credit, the ranking for the best cloud providers would be:

1. Google
2. Rackspace
3. SoftLayer
4. Microsoft
5. AWS

• Choice no $
  If I evaluate only the cloud service quality, considering no money for startups, the ranking would be:

1. Google
2. AWS
3. Rackspace
4. Microsoft
5. SoftLayer

Pros and Cons

Here I present an abstract of PROS and CONS for each cloud provider

Conclusions

Evaluating cloud providers is a very difficult task, since every provider has its own service package, characteristics and prices. Moreover, there are many attributes that should be considered in an evaluation: security, scalability, reliance, compatibility and other things. As I said I didn’t meant to cover all aspects of all services. This article is based on my experience and I did not used any scientific approach to make my tests. Everyone is more than welcome to send comments to make corrections if I made any mistakes on this post. I will be happy to keep it updated.

The objective of this article is to help Startups to have more paremeters when choosing between cloud providers, remembering that the most important thing is not wasting too much of your time on this task. The most important thing is to focus on your product development and your customer.