GigaOm Pro Blog provides a list of likely trends, mostly in the infrastructure that powers cloud computing. They don’t have a lot to say about software, which I believe will also be a critical turning point over the next few years for both public and private clouds.
Clouds computing is about buying just the amount of data center resources that you need, and having the ability to change your mind about that quickly. Any IaaS cloud provider worthy of the name will let you spin up a new virtual machine any time you want to add capacity to your data center. The hard part is getting your application to take advantage of that extra capacity. (Despite what I wrote here, this applies to private clouds, too.) Most off-the-shelf applications don’t go twice as fast if you have twice as many servers – in fact, most off-the-shelf applications are designed to run on just one server (or virtual machine). Even the typical n-tier application is constructed from a handful of special purpose system – e.g. web servers, database servers, application servers, file servers. Although the web layer in this architecture can often benefit from just adding more servers, the database layer typically doesn’t. If you are building something more specialized than the typical “web server that displays stuff from a database”, you probably have to invent a way to distribute your work across many machines, and then you have to build admin or automation tools to support adding and removing resources. Now you are hiring developers that are good at building distributed applications, and spending lots of time (and perhaps a limited supply of start-up capital or project budget) building out a robust platform that your real project will run on.
In Rework, the founders of 37signals suggest that you should not worry about the scalability of your application, because once you start making money, you can always buy a more powerful machine. I believe that their point was that instead of worrying about a hypothetical scaling problem, you should get some customers and generate some revenue, after which you will have some money to throw at the problem. I agree wholeheartedly with that point. But more and more of us are in environments where we know that if we cannot support a large number of users, or a large data set, or provide fast response times, the project will fail. And we sometimes realize that even if we buy the biggest server that Dell or HP makes, it isn’t going to be enough.
So what we need is a good cloud platform for developing distributed applications that can go faster when we add more hardware, but runs fine with a small amount of hardware. It should be something that doesn’t require super-human distributed computing development skills, that lets the developer focus on his application rather than the plumbing, and that an administrator can configure for whatever scale the occasion demands (e.g. seasonal load spikes).
Got a solution like this? I’d love to hear about it. If not, I might have to go build it – feature requests welcome
This story suggests that a senator was able to coerce Amazon Web Services into dropping a paying customer (Wikileaks), without any legal due process.
Amazon refutes this, and claims that the decision was their own, because the customer violated the AWS terms of service.
From Amazon’s message, it seems like Wikileaks is in violation of the TOS:
It’s clear that WikiLeaks doesn’t own or otherwise control all the rights to this classified content. Further, it is not credible that the extraordinary volume of 250,000 classified documents that WikiLeaks is publishing could have been carefully redacted in such a way as to ensure that they weren’t putting innocent people in jeopardy. Human rights organizations have in fact written to WikiLeaks asking them to exercise caution and not release the names or identities of human rights defenders who might be persecuted by their governments.
The trouble is, Wikileaks has not actually been charged with or convicted of a crime, and the potential long-term effects of the leak are debatable, so these terms of service are fairly subjective. The whole idea of Wikileaks is, of course, controversial. The problem is that Amazon’s action feels like an activist action – like the Wikileaks service was suspended because Amazon doesn’t like them.
For those of us building a business based on the AWS Infrastructure as a Service model, this is very scary. The one thing that we can count on with our own servers is that they don’t judge our content. We don’t have to convince our disk drives that our cause is just, and our moral compass is intact. Whether AWS responded to pressure from Joe Lieberman or acted on their own doesn’t really matter. The message is that if AWS thinks you are up to something fishy, their policy is to drop your service first, and ask questions later.
When I am looking at a business plan, I worry about technology risk, market risk, and execution risk. Normally, I would think that building a business on top of a cloud platform like AWS would dramatically reduce the execution risk, but now I have to worry about the risk that Amazon won’t like what I am doing. Amazon has been leading the cloud computing charge, and has been saying every step of the way “don’t worry, you can trust us”. I think banning Wikileaks was a giant step backwards in their credibility as a utility partner.