24 July 2008

The Cloud and Cloud Computing consensus definition?


"Cloud Computing is the realisation of Internet ('Cloud') based development and use of computer technology ('Computing') delivered by an ecosystem of providers."
It's amazing that such a simple concept has caused so much confusion, but having spent the last few days reviewing the recent discussions it seems many are falling into the trap of trying to align Cloud Computing with (or contrast it against) existing terminology like SaaS and Utility Computing. It is in fact far more suitable as an umbrella term encompassing all of these related components.

'The Cloud'

While there can be multiple definitions for Cloud Computing, for The Cloud itself 'there can be only one' as it's a metaphor for the Internet; people talking about clouds (plural) are probably confusing it with grids. Yes you can replicate some of this in a 'private cloud', but it will always be exactly that: a replica, and it will likely be somehow connected to (and therefore part of) the real cloud anyway. Remember, much of the value of Cloud Computing comes from leveraging other services in The Cloud for a result greater than the sum of its parts.

Why 'The Cloud'?

Remember all those network diagrams with a fluffy cloud in the middle? Why a cloud and not a black box or some other device? Because we simply don't know, and better yet we don't need to know, what goes on in there - we just pass a packet down our pipe and (most of the time) it arrives at its destination. This is an abstraction (in reality the Internet is an incredibly complex beast) but an important one; it significantly reduces the complexity of our systems; a good example is relatively simple VPNs having quickly displaced many complex WANs.

Definition

Let's break down my definition (which I came to by collating the assertions that were in line with my view and then boiling the result down to the basic common elements):

"Cloud Computing...
  • ...is the realisation of...
    While many of the requisite components have been available in various forms for some time (eg Software as a Service, Utility Computing, Web Services, Web 2.0, etc.) it is only now they are reaching critical mass that the Cloud Computing concept is working its way into the mainstream. As more of a collection of trends (a 'metatrend') we still have some way to go yet, but Cloud Computing solutions are a reality today and will rapidly mature and expand into virtually every corner of our lives and enterprises.
  • ...Internet ('Cloud') based...
    Although some have [ab]used the 'Cloud Computing' term in reference to infrastructure (particularly grid computing, like Amazon's pioneering Elastic Compute Cloud), much of its value is derived from the universal connectivity of the Internet; between businesses (B2B e.g. Web Services like Amazon Web Services), businesses and consumers (B2C e.g. Web 2.0 like Google Apps) and between consumers themselves (C2C e.g. peer to peer like BitTorrent). Many of us are now connected to 'The Cloud' where we work (office), rest (home) and play (mobile) and there are solutions (eg Gears) for when we are not.
  • ...development and use of computer technology'...
    an accepted, all-encompassing definition of computing - there are very few areas which will not be affected in some way by Cloud Computing so I've gone for the broadest possible definition.
  • ...delivered by an ecosystem of providers."
    While it is possible to enjoy some of the advantages using a single provider (eg Google), it is hard to imagine a functionally complete solution which does not draw on multiple providers (in much the same way as we install task-specific applications onto our legacy computers). Your electricity is almost certainly generated by wholesale providers who pump it into the grid and similarly Cloud Computing will typically be delivered by layered (eg Smugmug on Amazon S3) and/or interconnected (eg Facebook<->Twitter) systems.
Cloud Computing Architecture

Cloud Computing is typically universally accessible, massively scalable (with vast pools of multi-tenant 'on-demand' resources), highly reliable (see my TrustSaaS site for proof that the main services are up over 99% of the time), cost effective and utility priced with low barriers to entry (eg capital expenditure, professional services), but none of these attributes are absolute requirements (no, not even massive scalability - even an esoteric web service may still be an absolute requirement for a small handful of users and thus an important part of the ecosystem).

Cloud Computing architecture looks something like this, with layers similar to the OSI networking stack:
ServicesClientwhich consumes these applications via a browser and/or programmatically
Composite(Composite Applications or Mashups) which are linked together using APIs like REST (eg TrustSaaS), in much the same way as 'pipes' are used in Unix to create arbitrarily complex systems from simple tools
SoftwareApplicationwhich ideally follow proven Unix philosophy of 'do one thing and do it well', but which may grow quite complex
Platformon which applications are built, including the language itself (eg Java, Python) as well as supporting systems like storage
HardwareInfrastructureconsisting of the physical computing resources (and virtualisation layer(s) at the hardware and/or operating system layers)
Networkingcourtesy the existing Internet (eg TCP/IP)

Cloud Computing Components

Although many of these are solutions to the same problems, most of them are actually components of Cloud Computing, rather than Cloud Computing itself (working from the ground up):
  • Grid computing, any network of loosely-coupled computers acting in concert, is mostly concerned with tackling complexity and improving managability of computing resources (for example, production servers not being taken down by server failures or routine maintenance). You'll find grids outside of Cloud Computing architectures, though there is a [vendor driven] tendency to confuse the two (particularly where some intelligent/autonomic management aspects are involved). Don't make this mistake yourself; although many Cloud Computing systems are based on grids because their scalability needs can only be satisfied by horizontal scaling (usually involving thousands of commodity grade PCs), these are very different animals.
  • Virtualisation (in the Cloud Computing context), which allows you to deploy a virtual server where you might otherwise have provisioned physical hardware, is an enabler for Infrastructure as a Service (IaaS). Increased automation of operating system and application deployment is pushing the interface further and further up towards the application layer itself (eg Desktone's Desktop as a Service).
  • Infrastructure as a Service (IaaS) (Amazon EC2, GoGrid, AppNexus) While Internet ('cloud') connected grids are particularly useful (and a natural progression for virtualisation and SOA solutions being rolled out en-masse in enterprises today), implying that this is somehow equivalent to cloud computing is too narrow a view. Integrate a SaaS/Utility style billing system to a traditional grid and you've got Infrastructure as a Service (IaaS). These are more cost effective, reliable, scalable and user friendly than their disconnected counterparts and are one big step closer to the panacea of autonomic computing. Expect to see existing 'virtual infrastrucutre' providers like VMware and Citrix seamlessly complementing on-premises solutions with cloud based services.
  • Platform as a Service (PaaS) (Google's AppEngine, Salesforce's force.com, Heroku, Joyent, Rackspace's Mosso): takes grid computing to the next level of abstraction by pushing the interface up to the platform or 'stack' on which applications themselves are built (eg Django, Ruby on Rails, Apex Code). This is primarily interesting for developers and power users and is an increasingly important component of the cloud computing ecosystem. It allows them to focus on development without the overhead of hardware and operating system maintenance, database tuning, load balancing, network connectivity etc. while exposing technology like BigTable (and massive scalability) which might not otherwise be available to them. More importantly, it eliminates capital expenditure requirements, allowing boutique Independent Software Vendors like us to 'stay in the game'.
  • Utility Computing (Amazon S3) is more about a 'utility' (gas, water, electricity) pricing model, yet one can derive the benefits of cloud computing with a more traditional pricing model, or indeed without having to pay for it at all (consider Google's AppEngine for example, where it's utility-style pricing only applies to the more demanding users).
  • Web Services (Amazon Web Services): 'the 'glue' that holds cloud computing components together', are finally maturing and being adopted 'en-masse' thanks in no small part to simplification by way of protocols like REST, commercilisation by providers like Amazon (Jeff Bezos' Risky Bet) and the abundance of web toolkits (e.g. Ruby on Rails) which lower the barrier to entry by providing native support. You can do everything from payments to 'human intelligence tasks' with Web Services now and mashups rely on them heavily to make products that are greater than the sum of their parts. Companies like Ariba and Rearden Commerce are taking this to the extreme.
  • Web 2.0 (Wikipedia, Facebook, WebEx) which while a force in itself, deals more with making the web 'read/write', shifting power towards the consumer and leveraging their collective energy. While AJaX does a lot to make this environment more user friendly, the underlying theme is turning the 'reader' into a 'contributor'. Most of the players in cloud computing exhibit Web 2.0 attributes.
  • Software as a Service (SaaS): (Google Apps, Salesforce CRM) falls under the cloud computing umbrella and is a primary component, but to align the two definitions is too narrow a view. SaaS is typically sold per user as pizza is per slice, but what is more important is that it is implemented and maintained by a provider who handles much of the complexity of running software on your behalf (eg scaling, backups, updates, etc.).
  • 'Cloud' System Integrators (Australian Online Solutions) and consultancies deploy the various components, make them work in concert together (using services like RightScale), integrate them to each other and with legacy systems using the exposed APIs as well as migrating data (eg email, calendars, contacts, documents, etc.) so that users can 'hit the ground running' and continue to collaborate efficiently with those who have not yet migrated 'to the cloud'. Seamless migration is a reality today, and a critical component for cloud adoption.
Cloud Computing Today
The Cloud Computing revolution is upon us. Expect it to rapidly proliferate your enterprise, with much of the drive coming from individual grassroots users (who are almost certainly already improving operational efficiency with Web 2.0 tools like Google, Salesforce and WebEx) so plan accordingly. It must be embraced for competitiveness rather than resisted (in much the same way as the PC was embraced decades ago) but it also requires careful governance and change management by experts. Low risk, high return offerings like messaging and web security are available for those who want to 'test the water' without opting for a complete Enterprise 2.0 deployment.

The draw of loosely coupled, massively scalable services will eventually result in most enterprises being swallowed by the cloud (or by more agile, possibly 'digital native' competitors who already were), or at least becoming nodes on it; indeed many already have. Barriers to adoption (eg offline support, security and compliance services) are being torn down every day and practical solutions exist for those that remain (eg encryption) so there are less and less reasons to sit on the sidelines.

Even the largest of enterprises are now starting to jump (typically having completed controlled pilots) and just as company officers would have difficulty explaining downtime losses caused by continuing to generate their own power after cheap, reliable utility electricity became available, shareholders will not accept companies wasting resources on commotitised infrastructure rather than focusing on their core competencies.

Thanks to Jeff Kaplan, Markus Klems, Reuven Cohen, Douglas Gourlay, Praising Gaw, Jimmy Pike, Damon Edwards, Brian de Haaf, Ben Kepes, Jack van Hoof, Kirill Sheynkman, Ken Ostreich, James Urquhart, Thorsten von Eicken, Omar Sultan, Nick Carr and others for their inadvertent contributions.


This article is also available as a Google Knol: Cloud Computing.

7 comments:

  1. Nice work - even nicer that I got an acknowledgement!

    Ben

    ReplyDelete
  2. Good work. This is a great definition. It covers lots of what we have talked about on the forum.

    Edward M. Goldberg
    [email protected]

    ReplyDelete
  3. Great article.

    Be good to update this in the light of Microsoft's PDC announcements and place their upcoming offerings in article.

    ReplyDelete
  4. Thank you ! By joining Cisco, you raised the stature of Indian women in Silicon Valley by ten fold. Look forward to your leadership at Cisco.

    ReplyDelete
  5. Great piece! The prediction in the end is dated, but you probably didn't see the crisis coming then ;-)

    I understand today's comments a lot better, and agree. Inspired me to yet a new post that covers IT in a broad sense and is about human (top, business, dynamic) vs machine (bottom, infra, static)

    ReplyDelete
  6. To complement this post, here's a nice little infographic about cloud computing that's much easier on the eyes and the brain than a long blog article: http://bit.ly/cZLjN3

    ReplyDelete

Note: Only a member of this blog may post a comment.