31 March 2009

A thinly veiled warning to the CCIF "committee"...

As I just said to the group in Summary: Next Steps for CCIF / Cloud Forum, the future of the Cloud Computing Interoperability Forum (CCIF) is hanging in the balance. Its members feel betrayed (with good reason) after being strapped to the ClosedOpen Cloud Manifesto[gate] and will certainly not tolerate another such indiscression.

Positive discussion towards formalising the structure of the CCIF was temporarily derailed (again) by Jason Meiers pushing his SEMP agenda but will hopefully recover now that he has been effectively (temporarily) banned. Time is of the essence however as the CCIF now has to live up to its self-assigned deadline: Thursday's CCIF Wall Street event; all eyes are on it to quickly get back on its feet after a stumble and deliver a suitable venue for productive collaboration between vendors and users.

Meanwhile the "committee" (Sam Charrington, Reuven Cohen, Dave Niesen and Jesse Silver) are all(?) running around at the Cloud Computing Expo in New York City doing what they do best: networking and evangelising. I get the feeling though that something is afoot and a recent post from Jesse Silver was particularly insightful:
Though, my thought is that the CCIF could stay more community oriented than
"trade organization" oriented, so the CCIF should probably count more on
volunteer work and small financial contributions.

This leaves room for another, larger trade organization with a larger budget
than CCIF should handle, that the CCIF partners with in some way, forming
the fully open community arm of the industry.
This looks like step 5 of my "On the Open Cloud Manifesto, Private Cloud Cabal and CCIF-NG" post:

  • Use the momentum to announce a Cloud Computing Alliance which you will own and operate, also developed in secret and inviting only the usual suspects (or an Open Cloud Alliance - I'm not sure which it is yet, though a contact at Salesforce who should know wasn't aware of any involvement).
For any of the CCIF committee members, having gained "a seat at the table" via their participation, to propose or even take a role in such an organisation (whether under the CCIF banner or as individuals) would be a farce of epic proportions and a[nother] even more serious breach of the community's trust; Open Cloud Manifestogate would pale in comparison.

Conversely the formation of a new "alliance" without including the existing CCIF community would constitute gross ignorance of the needs of the users and shunning of the greater community for the benefit of an elite group of vendors. The age of old-school top-down standards organisations like DMTF and IEEE is drawing to a close and it's time for the community to (quickly) pull together and get on with the job. Enough is enough.

I urge everyone to forget about your biases, conflicts and agendas and work together to create "a "melting pot" where both [users and vendors] come together to share ideas". I've had enough of waiting for something I can confidently throw my weight behind and I know I'm not alone on this.

Sam

Update: Interesting first response refers to this tweet: "#Cloud Expo. Cloud Manifesto did the job and got attention, now community getting its act together with a Cloud Trade Org. TBA Weds". I'm right more often than I'm wrong but I sincerely hope this is an exception. You can follow the discussion here.

Update: If you're a blogger/journo already thinking about spilling some digital ink on this topic then please don't write a CCIF is dead article just yet... if this doesn't come together by the weekend then you can write what you like but I'm going to give it another 24-48 hours to come good. We've waited a year already so what's another few days between friends.

Update: A Cloud Computing Alliance group has already been created:

The Cloud Computing Alliance (CCA) is an industry trade organization focused on advocating the adoption and use of Cloud Computing and related of technologies. The CCA focuses on building industry consensus in a formal setting.



Update: Google Group was deleted within minutes of posting about it. There's still a Twitter account though.

30 March 2009

Manifesto embargo lifted: Google dropped out, private cloud introduced

Now that the embargo has lifted at 12:01AM articles like IBM Twisting Arms In Support Of 'Open' Cloud Computing (IBM) (and a bunch of pre-prepared puffery) are starting to appear, revealing that Google was one of an undisclosed number of signatories to drop out along the way. It's also been revealed that half of the companies it was forwarded to rejected it outright.

More revealing though was publication of an old version 1.0.4 of the manifesto (1.0.9 was leaked and there may be more changes between it and opencloudmanifesto.org, also just revealed). Here's some of the highlights that reveal points of contention that had not already been resolved between 1.0.0 and 1.0.4, essentially proving that this is in fact a Private Cloud Manifesto:
  • Audience: Changed from "CIOs and other business leaders" to "CIOs, governments, IT users and business leaders" and "vendors" became "providers"
  • Private cloud: Pronouncement injected into definition: "The cloud architecture itself can be private (hosted within an organization’s firewall) or public (hosted on the Internet)."
  • Scalability on Demand: Previously "the cloud can [scale]", now "cloud technologies (both private and public) can [scale]" and "artificially high level of resources to handle peak demands" became "multiple sets of artificially high levels of resources to handle peak demands" (!?!)
  • Streamlining the Data Center: "The vast majority of the hardware, software, personnel and energy costs are borne by the cloud vendor, freeing an organization’s resources for other projects" (redacted)
  • Minimizing Startup Costs: "[costs] borne by the cloud vendor" became "[costs]borne by the cloud provider, whether the cloud is private or public". (!?!)
  • Data and Application PortabilityInteroperability: Previously "bringing systems back in-house" would "require the organization to replicate the cloud vendor’s environment or change the system to use their own environment". Now it's just "difficult and expensive".
  • The Goals of an Open Cloud: Disclaimer about differentiating between public and private clouds redacted.
  • Principles of an Open Cloud: Private cloud pronouncement about "many clouds" being "different" added: "Of course, many clouds will continue to be different in a number of important ways, providing unique value for organizations. It is not our intention to form standards for every capability in the cloud and create a single homogeneous cloud environment."
  • Principle #1: "Challenges [must be] addressed through open standards" are now "addressed through open collaboration and the appropriate use of standards".
  • Principle #2: Lock-in to "particular platforms" became "particular platforms and limiting their choice of providers".
The first batch of news reports is uglier than expected too:
Anyway it's not like this hasn't happened before...

Big day for the Open Cloud: Next steps for Cloud Forum

An open letter in response to An Open Future for CCIF:

Mark this date. Today Sunday 29 March 2009 is an extremely important day for the Open Cloud (that's right, with capitals ala Open Source). The Cloud Computing Interoperability Forum (CCIF) "committee" (for want of a better term, consisting of Sam Charrington, Reuven Cohen, Dave Niesen and Jesse Silver) have walked away from the Open Cloud Manifesto that they helped create and in doing so may have jeopardised personal and professional relationships with IBM and other signatories.

They have also published a heartfelt apology for their "direct and private engagement, in the name of the CCIF, vis a vis the Open Cloud Manifesto [which] may be viewed as a breech of this community's norms". I have tried to call Reuven to personally congratulate him for this stunning about face and if this editor had a "standing ovation" function then I'd be using it.

I've been a vocal critic of the CCIF since its inception, first arguing that standards efforts were premature (they were), launching a competing cloud community, declaring the CCIF dead over lack of openness (moderation/censorship) which resulted in reconsideration of its role, developing a truly open Cloud Computing Manifesto and finally revealing another secrecy scandal earlier today relating to the behind-the-scenes formation of a Cloud Computing AlliancePrivate Cloud Cabal that was to be announced at the CCIF event on Thursday. I can't take all the credit for the about face though - the language in the cloudforum Google Group over the last few days has been "flamboyant" to say the least and ultimately it was their decision to back down from an untenable position.

There are some loose ends to tie off, most notably the famous Monday Microsoft Meeting and Cloud Computing AlliancePrivate Cloud Cabal:
  1. For the meeting Jesse assures me that they have decided to "say nothing" and provided they both "say nothing" and report back to us the proceeds then I don't have any problem whatsoever with them officially representing the CCIF even in the absence of a formal governance structure. If anyone does have a problem with that then say so now or forever hold your peace (it is, after all, only hours away now).
  2. For the "Alliance" it's apparently off the table. The coming soon page has already been taken down and I would, in the strongest possible terms, caution about raising the issue again until well after the dust settles (if at all). Iff the community wants an alliance then it can build one from inception in the open.
Moving right along as we have a lot to do now and we need to act quickly as all of these points should be completed within the coming 72 hours (assuming we agree to them). Here's some things that come to mind, in order of priority:
  • Forgive and move on as the breach of trust is so far as I am concerned a thing of the past - I for one accept their apology and from what little contact I've had with each of them I belive they will stick to their word. Translation: this point doesn't need any further discussion in the group, except in the context of moving forward.
  • Damage control as there will be fallout from this decision that may need to be handled over the coming days. I don't think I need to go into details here as those who need to handle it will.
  • Defer governance issues at least until the dust settles. We've got way too much to do right now to risk descending into a debate about the nuances of memorandums and articles and other committee time wasters. When and if we need to review this we can, and then it should likely be a simple non-profit structure with a relatively static board of stable members dealing with administrative issues such as sponsorships, trademarks, domains, copyright licenses, etc. For now I would suggest that our illustrious leaders simply be called the "committee" and that this committee be grown or shrunk as necessary. If Reuven wants a title then I would humbly suggest that something like "Evangelist" may be better than "Instigator". If I understand well Jesse serves a PR/Marketing function, Dave is the CloudCamp extraordinaire and Sam is a business liasion(?) - appropriate role based titles could be assigned if deemed necessary.
  • Adopt consensus decision-making for most decisions, such as which standards to support. Votes may be held for limited tasks such as approval of creation of a working group, ratification of a standard, changes to rules or appointment of individuals to the committee.
  • Rename CCIF to "Cloud Forum" as Manifestogate has done us some serious damage, and how better to demonstrate serious change than with a change in name. I was always disappointed that the monarchists rejected an Australian republic just in time for the Sydney 2000 Olympics as I think it would have been a great way to enter the new millennium and the same applies here. By adopting the new name (bearing in mind the group is already cloudforum and domain cloudforum.org) we move from an essentially meaningless acronym to an appropriate name for a group dealing with general cloud computing advocacy.
  • Ratify & Adopt the Open Cloud Principles (or some functional equivalent) as this will put to rest any question of being a closed/secretive organisation (remembering that this was the primary criticism both internally and externally - even today's missive was titled An Open future for CCIF). One function of the Cloud Forum can be similar to that of the Open Source Initiative. Adopting the principles means opening the group by disabling invitations and moderation - spammers and trolls can be dealt with on a case by case basis and if it proves a problem we can decide as a community what remedial action to take.
  • Deliver an Open Cloud Open Manifesto but call it a Cloud Computing Manifesto or similar if IBM et al press on without us (otherwise Open Cloud Manifesto is fine). If we can take over management of the document once it is released that would be even better yet. The result should be the bastard child of the infamous Open Cloud Manifesto, the Cloud Computing Manifesto (seeded from Microsoft's suggestions) and other feedback from within and outside the Cloud Forum.
  • Officially support existing standards efforts starting with DMTF OVF which already exists. Also the Open Cloud Computing Interface (OCCI) (mailing list) is currently the best chance we have of realising a cloud infrastructure (IaaS) API in the short term (implementable draft by May) and an easy way for Cloud Forum to get some runs on the board as momentum is gathering quickly behind the project and I am confident it will be a success. Final call for the charter is 5pm CET Monday (today) for rubber stamping by the OGF steering committee by Wednesday.
By taking these steps (and possibly others I have failed to consider) I believe the Cloud Forum formerly known as CCIF will be in a great position to evangelise cloud computing and the Open Cloud. Thanks again to Sam Charrington, Reuven Cohen, Dave Niesen and Jesse Silver for "activating" the community.

29 March 2009

On the Open Cloud Manifesto, Private Cloud Cabal and CCIF-NG

I've mentioned on various occasions that the infamous Open Cloud Manifesto is just the tip of the iceberg but until now I haven't had a smoking gun (at least none that weren't shared with me secretly under modus operandi for this "shadowy cabal"). Now I do and (at the time of writing) it's posted at http://www.cloudalliance.org/ (whois). Obviously there's more in the pipeline than we've seen thus far and the next phase will be fed to us this week, probably on Thursday at the CCIF Wall Street event (after revealing the remaining shreds of the manifesto on Monday... assuming that goes ahead).

The process appears to go something like this:
  1. Hijack IBM's drive into the cloud computing space under the auspice of "Open Cloud", claiming credit as co-author, instigator, etc. and allowing (if not encouraging) press to report same.
  2. Deliver a "Private Cloud Manifesto" that focuses on the "private cloud" fringe theory and alienates all of the existing "real" cloud computing providers including Amazon, Google, Microsoft and Salesforce (you're a private cloud vendor yourself remember).
  3. Announce the offending document at the Cloud Computing Expo in NY on Monday morning anyway.
  4. Accuse dissenters of not being "open" in the same way that Internet censorship opponents are branded child pornographers and privacy advocates supporters of terrorism.
  5. Use the momentum to announce a Cloud Computing Alliance which you will own and operate, also developed in secret and inviting only the usual suspects (or an Open Cloud Alliance - I'm not sure which it is yet, though a contact at Salesforce who should know wasn't aware of any involvement).
  6. ???
  7. Profit!
To be candid, I think Reuven Cohen has done his run as self-appointed dictator and his attempts to pwn the community have constituted an epic fail; in comparison to the wildly successful CloudCamp franchise the CCIF itself has turned into a laughing stock (which is why the two should not be combined under one umbrella as is/was to be proposed). That's my personal opinion and while others may share it, many will not. It's not set in stone and is subject to change depending how this situation is handled from here - it wouldn't have got to this point had they have "gone open" on Thursday when called to by me, Microsoft and others.

What matters now is getting out of this rut, putting away the swinging handbags (I know it's a weekend so our work hats are off but still...), and most importantly, working out how to minimise fallout in what is shaping up to be an ugly week for cloud computing. Whatever happens in New York will be under close scrutiny and unfortunately has the potential to tar the entire community with the same brush.

Here's what I propose:
  • Someone not too close to the situation and without any (perceivable) conflicts/biases chair a working group to define what "ccif-ng" might look like - even a thread on the cloudforum list would probably suffice. In addition to Reuven/Jesse and their A-list (you know who you are) that rules me out given my [at times too] sharp criticism of the CCIF - even if I have only pure intentions.
  • A new group would likely need new leadership (or no leadership). I'm happy to contribute but it won't be as leader and it may be better there were none anyway.
  • Everything about the group from its inception be defined in public, allowing contribution from any party regardless of who they know, who they are and how big a membership fee they can stomach ($12,000 a vote for DMTF for example).
  • Avoid the many problems associated with voting by adopting a consensus based approach which relies on the strength of the arguments rather than the size of the posse (something of a meritocracy). Cloud computing is about the end users remember, not the people who deliver them service.
  • Consider taking over, ratifying and subscribing to [something like] the Open Cloud Principles I drafted yesterday (as well as the entire Cloud Computing Community wiki and cloudcommunity.org domain for that matter).
  • Operate an open community, publicly removing only those who cause problems (spammers, trolls, etc.), in the cloudforum group or somewhere else TBD.
  • Take heed of calls (including my own) to follow an IETF-style process (perhaps involving the IETF themselves for applicable standards).
  • Limit sponsorship (if any) to that which is absolutely required for the functioning of the organisation - for example "eating our own dogfood" by having a provider like Mosso volunteer to host the community/wiki (which is a work in progress). Even then "sponsorship" and "bias" are often intrinsically linked and I know of a number of initiatives here which have already been adffected by the whims of behind-the-scenes "investors".
  • Get on with evangelising cloud computing and related efforts such as relevant standards groups (without trying to boil the ocean ourselves), building and blessing definitions and models, creating whitepapers, preparing presentations, devising business cases, promoting the cause and generally raising awareness.
I'm quite sure this can been achieved in conjunction with the existing team - despite comments above Reuven is clearly a great evangelist and networker and Jesse is doing a lot of great work behind the scenes. I know they are both busy reviewing their strategy in light of all that has happened in the past days and I sincerely hope what results turns out to be largely compatible with this proposal.

In the mean time I encourage everyone to focus on what the future holds rather than what was done in the past - whether they meant well and were misunderstood or didn't and were caught out doesn't matter provided we work out how to move forward.

Enjoy your Sunday - I'm off to find a nice thermal spa in Germany having absorbed enough existing specifications for a lifetime this week.

Cheers,

Sam

Update: Jesse Silver aka silverguru has posted about "an olive branch to the CCIF community" that will be offered tomorrow. He confirmed plans for an alliance separate from CCIF but said that the site was out of date and would be taken down pending more information. He also denied any connection with opencloudalliance.com and related domains and on further inspection these are registered to Mark Trang (Salesforce's Director, ISV Marketing & Programs) at home using a personal email and may not be related to any official Salesforce.com initiative. Both domains were discovered with a domain search and could well be coincidental.

References:

28 March 2009

Introducing the Open Cloud Principles (OCP)

In light of the rapidly increasing (and at times questionable) use of the term "Open Cloud" I hereby propose the following (draft) set of principles, inspired by the Open Source Initiative (OSI) with their Open Source Definition (OSD).

I would be interested to hear any feedback people have with a view to reaching a community consensus for what constitutes "Open Cloud" (in the same way that we have had clear guidelines for what constitutes "Open Source" for many years). You can do so in reply to this post, on the document's talk page or by being bold and editing directly - if I don't hear from you I'll assume you're satisfied.

Examples of uses today include:
For the latest version of the document please refer to http://wiki.cloudcommunity.org/wiki/Open_Cloud_Principles

Open Cloud Principles (OCP)

From Cloud Computing Community Wiki

Overview

In order to stem the abuse of the term "Open Cloud" the community is forming a set of principles which should be met by any entity that wishes to use it, similar in spirit to the OSI's Open Source Definition for free software licenses.

Principles

Rationale: Open Cloud offerings should be available to the maximum number and diversity of persons and groups. Competition must not be restricted.
  • No Barriers to Exit: There must be no obstacles in the path of an entity that make it difficult to leave. For example, a user must be able to obtain their data in a utile machine-readable form on a self-service basis.
Rationale: Obstacles that prevent entites from abandoning one offering for another reduce competition, which must not be restricted. If the barriers of exit are significant; a firm may be forced to continue competing in a market, as the costs of leaving may be higher than those incurred if they continue competing in the market.
  • No Discrimination: There must be no discrimination, including against any person or group of persons or specific field of endeavor. For example, it may not restrict the program from being used in certain countries, select certain people, by a commercial endeavour, or from being used for genetic research.
Rationale: All users should be allowed to participate without arbitrary screening.
Note: Some countries, including the United States, have export restrictions for certain types of products. An OCP-conformant product may warn users of applicable restrictions and remind them that they are obliged to obey the law; however, it may not incorporate such restrictions itself.
  • Interoperability: Where an appropriate standard exists for a given function it must be used rather than a proprietary alternative. Standards themselves must be clean and minimalist so as to be easily implemented and consumed. For example, if there is a suitable existing standard for single sign on than it must be used by default, although including support for alternative interfaces is permissible.
Rationale: Standards foster interoperability and competition giving rise to a fairer marketplace. The absence of standards and to a lesser extent, complex standards, have the opposite effect.
Rationale: Free licenses impose no significant legal restriction relative to people's freedom to use, redistribute, and produce modified versions of and works derived from the content.
  • Technological Neutrality: No provision of any license or agreement may be predicated on any individual technology or style of interface. For example, it may not require that network clients run a certain operating system or be written in a certain programming language.
Rationale: Such restrictions limit the utility of the solution and freedom of the user by preventing them from using their preferred solution.
Rationale: Transparency implies openness, communication, and accountability and prevents unfairly advantaging or disadvantaging certain parties.

Acknowledgements


See also

Cloud Computing: A call for open community consensus...

On Sat, Mar 28, 2009 at 4:24 AM, ewindisch <eric@grokthis.net> wrote:

While I'm not happy with
the way it was presented or produced, and I am still waiting to see
what happens Monday, but if this is taken to a vote on Thursday and
ratified by the members of the CCIF
, I won't be terribly disturbed by
it, and think that it will be a good *step* towards making real
progress.

As sombody who has just lived through the last 8 years in the US and is now faced with the prospect of living through the next 8 I am surprised that you would be so quick to suggest that a vote is a useful mechanism here.

Voting on standards has always been problematic and always will be - voting on pre-cooked documents is even worse. Given the sordid history of this version of the manifesto I would encourage anyone given the opportunity to vote on it (that being an elite subset of this group able to travel to NY) should reject it on principle, and anyone thinking to propose it for a vote should reconsider given it just drives home how hypocritical this whole farce has been and makes us look like a pack of lame sheep. This is particularly pertinent given it will be an open ballot and those expressing their discomfort with the process are being branded "unhelpful" naysayers by those on the A-list. Many of those voting won't even be from this group anyway.

It's a shame that our illustrious leader(s) was dragged into this, and dragged the rest of us into it with them through various actions and inactions, but what's done is done. It's also a shame it wasn't handled differently, for example by opening the document up immediately on disclosure rather than trying to preempt a pre-announcement with a pre-pre-announcement (thus dragging us in further, as evidenced by Slashdot article above).

A few hours ago I threw open for discussion some rough design notes for the Open Cloud Computing Interface (OCCI). Those who don't grok wikis can discuss on the archives but those who do are welcome to roll up their sleeves and make changes, or propose them for discussion and building of community consensus on the talk page. This gives everyone a place at the table, promotes the best ideas to the surface and rapidly boils off puffery and other noise (which is the primary ingredient in the rambling monologue that we've just been force fed). Particularly stubborn stains are tagged with appropriate templates like {{peacock}} which states:

This article may contain wording that promotes the subject in a subjective manner without imparting real information. Please remove or replace such wording or find sources which back the claims.

By all means stick around and see who's still game to put their name on this garbage on Monday (I'd say that list has thinned out substantially, and I wonder if IBM's name is even still on it) but it wasn't without good reason that Microsoft specified that they "feel strongly that any 'manifesto' should be created, from its inception, through an open mechanism like a Wiki, for public debate and comment, all available through a Creative Commons license".

That's what we've done here with input from Microsoft, the Mozilla Manifesto, myself and some other random sources, and I personally think the result is much cleaner (both structurally and from an "untainted" point of view), more useful and has more teeth than the wishy washy pronouncement ironically branded the "Open Cloud Manifesto".

I personally think it's high time for a "reboot" of the CCIF; a revolution of sorts (and the cloud computing community in general for that matter). Having found my work summarily deleted and myself promptly banned from this group for responding to Microsoft's call in creating the rival document (albeit briefly given one of the moderators immediately threatened to quit his post over it) I wonder whether this particular group is beyond repair but I guess that will be seen in the coming days/weeks.

I am particularly disconcerted given that one of the insiders (to remain nameless) has asked me a number of times by phone and email to publish a retraction/correction regarding my role in the process, for my own benefit of course: "I think it's in your best interests to publicly state you are not the author of the manifesto. Can you do that on cloudforum and cloud-computing?".

Ironically I'm still banned from the latter group 6 months later (along with other bloggers including Geva Perry and James Urquhart) over my post announcing the first (open source no less) cloud computing shell (cush), though unsurprisingly I still have a problem with moderation/censorship and the associated power grab.

Accordingly I've given them their "correction" in the creation of Wikipedia's Cloud Computing Manifesto article, where the real story is both open (anyone can edit) and subject to Wikipedia's unrelenting BS filter.

Anyway here's what Brandon Watson (another Microsoft director) had to say in reference to the Cloud Computing Manifesto in his followup post "An Open Cloud Requires an Equally Open Manifesto":

Sam Johnston did what I would expect from the community. No bitching. No whining. He set up a wiki for an open cloud manifesto and said, let’s get this thing started. Awesome. Better yet, people are already writing to it. If you read this post and are even remotely interested in cloud computing, go check it out, and contribute. Please. We think that it’s great to have a discussion, in the open, where everyone, no matter what size you company, can speak and be listened to.

It will be interesting to see which direction the community takes from here - the open road or the well trodden path of politics, invite-only boys clubs, votestacking and "back room cigar-smoke-filled scam[s] of the good old days". I guess that will become clear this week, but I urge those of you still reading this diatribe not to take this assault lying down as I'm afraid it's just the tip of the iceberg.

Sam

Further reading:

26 March 2009

Announcing the Cloud Computing Manifesto

Earlier today Microsoft savaged an undisclosed effort to develop an "Open Cloud Manifesto" in secret, tweeting that "An open cloud deserves an equally open manifesto: http://is.gd/oZtA (and don't show up to a party making demands without bringing something)" while blogging:
"We were admittedly disappointed by the lack of openness in the development of the Cloud Manifesto. What we heard was that there was no desire to discuss, much less implement, enhancements to the document despite the fact that we have learned through direct experience. Very recently we were privately shown a copy of the document, warned that it was a secret, and told that it must be signed "as is," without modifications or additional input. It appears to us that one company, or just a few companies, would prefer to control the evolution of cloud computing, as opposed to reaching a consensus across key stakeholders (including cloud users) through an “open” process. An open Manifesto emerging from a closed process is at least mildly ironic."
Although there's no prizes for guessing who's behind this let's skip the speculation (the consortium will reveal itself "in the coming days or weeks" anyway) and get on with the job:
To ensure that the work on such a project is open, transparent and complete, we feel strongly that any "manifesto" should be created, from its inception, through an open mechanism like a Wiki, for public debate and comment, all available through a Creative Commons license. After all, what we are really seeking are ideas that have been broadly developed, meet a test of open, logical review and reflect principles on which the broad community agrees. This would help avoid biases toward one technology over another, and expand the opportunities for innovation.
It so happens that the Cloud Computing Community already has a MediaWiki (ala Wikipedia) installation (http://wiki.cloudcommunity.org/) for exactly this purpose and had previously developed a complementary document for users, the Cloud Computing Bill of Rights.

As such I am pleased to announce the Cloud Computing Manifesto which has been seeded with Microsoft's three initial suggestions:
  1. Open Process: Interoperability principles and any needed standards for cloud computing need to be defined through a process that is open to public collaboration and scrutiny.
  2. Balanced Representation: Creation of interoperability principles and any standards effort that may result should not be a vendor-dominated process. To be fair as well as relevant, they should have support from multiple providers as well as strong support from customers and other stakeholders.
  3. Continuous Development: Due recognition should be given to the fact that the cloud market is immature, with a great deal of innovation yet to come. Therefore, while principles can be agreed upon relatively soon, the relevant standards may take some time to develop and coalesce as the cloud computing industry matures.
It's an open wiki driven by community consensus and anyone can join and contribute (there's no invitations, membership fees, etc.) so be BOLD and if you have something to add then go ahead.

Update: This was picked up by TechFlash in Seattle who claim "Microsoft has a reputation for attempting to work standardization processes to its own competitive advantage". Whether true or not I'm inclined to assume good faith at least in this instance.

Update: eWeek are on the case now and have written a long, insightful article on the topic (Microsoft Calls for Open Cloud Standards) in which they refer to Bruce Perens' comment that "Microsoft employees that Open Source projects directly deal with are as sincere as you'd like, but they aren't top management and can't influence top management". They've fingered Amazon, Google and IBM as likely suspects (given they're competitors with Microsoft in the cloud computing space) which reconciles with "several of the largest tech names". If anyone knows for sure who the culprits are or has a copy of the offending "offer" then drop me a line (anonymously) or leave a comment.

Update: True to form and confirming our suspicions the Coud Computing Interoperability Forum (CCIF) has deleted the Cloud Computing Manifesto announcement leaving the error: "Topic not found: We're sorry, but we were unable to find the topic you were looking for. Perhaps the URL you clicked on is out of date or broken?". Amusingly though they left the broken link in a tweet on their Twitter page (for now):

24 March 2009

Cloud Standards Roadmap

Almost a year ago in "Cloud Standards: not so fast..." I explained why standardisation efforts were premature. A lot has happened in the interim and it is now time to start intensively developing standards, ideally by deriving the "consensus" of existing implementations.

To get the ball rolling I've written a Cloud Standards Roadmap which can be seen as an authorative source for information spanning the various standardisation efforts (including identification of areas where effort is required).

Currently it looks like this:
Cloud Standards Roadmap
The cloud standards roadmap tracks the status of relevant standards efforts underway by established multi-vendor standards bodies.
Layer Description Group Project Status Due
Client ? ? ? ? ?
Software (SaaS) Operating environment W3C HTML 5 Draft 2008
Event-driven scripting language ECMA ECMAScript Mature 1997
Data-interchange format IETF JSON (RFC4627) Mature 2006
Platform (PaaS) Management API ? ? ? ?
Infrastructure (IaaS) Management API OGF Cloud Infrastructure API (CIA) Formation 2009
Container format for virtual machines DMTF Open Virtualisation Format (OVF) Complete 2009
Descriptive language for resources DMTF CIM Mature 1999
Fabric ? ? ? ? ?

Other standards efforts
Vendor-owned standards
Other resources
Please check the Cloud Computing Community Wiki for the latest version as this information will be quickly dated. If you have any updates please feel free to contribute them.

13 March 2009

Approaching cloud standards with *vendor* focus only is full of fail

So I was taking stock of the cloud standards situation and found an insightful article (Cloudy clouds and standards) over at ComputerWorld via a colourful counterpoint over at f5 (Approaching cloud standards with end-user focus only is full of fail), hence the title. I made a comment which quickly turned into a blog post of its own (and was held for moderation anyway) so here goes:
I followed a link to this "short-sighted and selfish" view from Lori @ f5's Approaching cloud standards with end-user focus only is full of fail rant and have to say that as an independent consultant representing the needs of large enterprise clients it's not surprising that I should agree with you (representing the needs of end users in general) rather than a vendor.

Cloud computing is a paradigm shift (like mainframe to client server) and attempting to document it all in one rigid "ontology" is a futile exercise, as evidenced by the epic failure of attempts to do so thus far. A birds eye view of the landscape is possible, but only in the retrospective sense. One of the great things about cloud computing is that it is user-centric - for once the end-user has an opportunity to call the shots rather than being told what to do by vendors.

My various efforts (writing the Wikipedia article, setting up the Cloud Computing Community and more recently working on cloud standards starting with Platform as a Service) have all involved looking at what innovation is taking place in the industry and determining the consensus. Now is a very good time to do so as well because there are enough data points but no de facto proprietary standards (though the EC2 API is worryingly close to becoming one).

I tend to take advice from vendors on this topic with a grain of salt because most of their input tends to involve pulling the resulting "open standard" closer towards their particular offering - Enomaly's Unified Cloud Interface (UCI) for example not only focuses on VM provisioning but goes so far as to include them specifically alongside Amazon and Google.

The user doesn't [need to] care about this level of detail any more than they need to care about how a coal-fired power station works to turn on a light. The whole point of the cloud is that it conceals or "abstracts" details that ultimately become somebody else's problem. Using the power analogy again, our "interfaces" to the electricity grid are very well standardised (2-4 pins and a certain voltage cycling at a certain frequency) and "The Cloud" needs similar interfaces (for example for storing data and uploading and managing workloads).

Once we have that computing will be quickly commoditised, which is every users' best dream and vendors' worst nightmare (except for the few, like Amazon and Google, who still have a seat after the computer industry's next round of musical chairs).
In summary, cloud computing is finally an opportunity to shift the focus from the vendor to the user, where it arguably belongs. Vendors don't like this of course (and anything they say on the subject should be viewed accordingly) and are doing everything they can to stake a claim in what is something equivalent of a gold rush. Only this time (unlike the dotcom bust) it's real gold we're talking about (not fools' gold) and a large, sustainable (albeit heavily consolidated) industry of "computer power stations" and associated "megacomputer" supply chains will result.

11 March 2009

Crystal Ball: Apple's $599 "iPad Touch" Netbook (with pictures)



They say a picture is worth a thousand words so how better to illustrate what I think an Apple "netbook" might look like than to design what I'd build if I were them. I've affectionately called it the "Apple iPad Touch" - not because it's a particularly clever name but because it's close to "iPod" and thus avoids having to create a new "iNote", "iNet" or similar logo. There's been renewed discussion (e.g. here, here, here, here, here, here, here, here, here, here, here and here) about such a device of late thanks to rumours that Chinese manufacturers Wintek and Quanta will make the screens and manufacture the devices respectively, and confirmation by Dow Jones Newswires via "two people close to the situation".

The device would be bigger than an iPhone but smaller than a MacBook (actually it would be quite similar to the lid/screen of a MacBook on its own) and would be both light and durable. It may come with a protective case to protect its large, glass screen but these would definitely be available optionally and/or from third parties. It would stick with the aluminium/black glass theme, though given the appearance of the high density plastic backing for the iPhone 3G the temptation would be to make multi-coloured and/or special editions available, going back to the iBook clamshell roots.

The interface would be a breeze to use and the multitouch functions would be even more spectacular than when they first appeard on the iPhone. Thanks to advancements in the upcoming Safari 4, navigating to your faviourite sites would be as simple as clicking on a thumbnail in the new coverflow style speed dialer and the browsing experience will be similar to, but thanks to the extra power and pixels, much better than that of the iPhone. Multimedia (including video) will also be a pleasure to watch on its large high definition screen (but you'll have to keep an eye on the battery) and finding what you want will be trivial thanks to CoverFlow:

Here's the suggested specs:
  • ARM processor (1Ghz+)
  • 256-512Mb RAM
  • 32-64 Gb flash drive (ala iPhone, primarily for audio & video media)
  • iPhone-style stripped back Mac OS X operating system
  • 9-10" Widescreen HD ~800x400 multi-touch screen (with LED backlighting)
  • Built-in microphone, speakers and webcam.
  • Battery replacement program (for a clean, catch free finish)
  • WiFi connectivity (802.11 b/g/n)
  • WiMax connectivity (maybe saved for 2nd gen release, 802.16e-2005?)
  • 3G service (potentially with carrier contract)
While it would have been nice to break the psychological barriers given the current economic climate by hitting the $499 price point, price at launch would be around $599.00/€499.00/£449.00, and while unlocked iPhone 3G's sell around that today (in Australia they're apparently around USD550), supply with or without a 3G carrier contract may affect the price.

Connectivity is key so it will support at least WiFi, but for "always connected" service 3G may be required. WiMax may also be an option in some regions to provide connectivity while side-stepping the mobile networks. Given good connectivity the iPhone's dialer functionality may make an appearance too, but rather than holding the thing up to your head you'd have to go for a bluetooth headset, and similarly if you were bored with the onscreen keyboard then a bluetooth Apple Wireless Keyboard would be your only option. The charging cradle would sport the traditional iPod connector and a slot with a photo-frame style appendage for holding the device at an adjustable angle, facilitating the use of the keyboard and making the device a pleasure to use on the road (e.g. on trains/planes).

Although full-blown Mac OS X is pictured above, in reality a significantly reduced feature set would be available via an iPhone, Front Row and/or Cover Flow style interface running on something akin to Snow Leopard. This would be the main thing preventing cannibalisation of MacBook sales - the included AppStore wouldn't include mainstream apps like iWork and Microsoft Office. Basic functionality would be provided via a suite of cloud computing tools like iWork.com but the absence of a keyboard would be another limiting factor. Similarly a lot of the media management would depend on a more traditional device (like the iPhone and Apple TV), but that doesn't really matter since most of the content would be accessed over the Internet (at least when a high speed connection is available). Make no mistake - being a "netbook" this will largely be a single-purpose device, for watching YouTube videos, interacting with friends on Facebook, reading news feeds and so on. Nobody said netbooks had to be small or cheap but instant-on, constant connectivity is a must.

I for one would line up for this device (well, I'd submit an online order anyway) and if I were Steve Jobs I'd make its release my "I'm Back!" message.

Update: There's evidence of new devices in the iPhone 3.0 OS images including the iProd and iFPGA. The iProd is likely a personal trainer device, though my initial reaction was a touch interface ala the tablet above. The iFPGA on the other hand is certainly referring to a different type of chip than the usual ASICs that can be programmed at the hardware level to function differently - this could well be a new architecture for the new devices.

2010 Update: With the announcement scheduled for tomorrow I thought I'd update this post to refine my predictions based on what we've discovered in the interim (it's getting harder and harder for Apple to run a tight ship with secret product development - thanks to patent and trademark leaks, suppliers, and just plain old guesswork).

  • The device will be like a blown up iPhone and most importantly will run the iPhone OS (a derivative of OS X) rather than some skinned version of Snow Leopard.
  • It may be smaller than anticipated, but it will have a high quality screen like that of the Nexus One - possibly 7" @ 1280x720 (for HD video content).
  • It will cost ~€499 in Europe with a mobile contract like T-Mobile's Complete L (€899 without), and that contract may well include voice, text, etc. in addition to data. Contrast to the Amazon Kindle which provides free albeit limited service for store purchases, subscriptions and soon, 3rd party applications.
  • It won't be immediately available, rather it will ship February/March (possibly March 1st).
  • Not much will be said about the specifications (as is the case for iPhone) but they will be quickly discovered and likely in line with what I suggested above - except perhaps for a bump in solid state storage for video.
  • Apple tend not to mix messages so there will be a lot of focus on specific use cases. Figuring this to be young/middle-aged professionals with discretionary spending even in a tough economic environment I'm thinking education and entertainment. Books. Magazines. Newspapers. Games. Movies. Music.
  • Expect a lot of the content to be subscription based, though given 3G is the weak link for delivery of HD video the focus will likely more be on newspapers et al (which are relatively cheap to deliver while still offering significant value)
  • The interface will want to be sexy - think photorealistic page turns and [enhanced?] coverflow based browsing.

Company and product names are trademarks of the respective owners, some images courtesy Wikipedia.

This work by Sam Johnston is licensed under a Creative Commons Attribution 3.0 Unported License.
Creative Commons License

Cloud Computing Interoperability Forum (CCIF) reconsiders its role

Following my broadside on Sunday and a flurry of posts from an IP troll there's been a renewed discussion about cloud standards and particularly the role of the Cloud Computing Interoperability Forum (CCIF). There was a fair bit of chatter on twitter about it and eventually Reuven issued a statement "Reaffirming the CCIF Goals and Mission" before popping off to Sweden.

The main message was:
The goal of the CCIF is to be a thought forum and advocacy group for cloud interoperability and related standards, it is NOT to be a standards body.
And for that the mission is three fold:
  1. Free, in person events like Cloud Camp and the Wall Street / Mountain View & Washington Cloud Interoperability Forums.
  2. A community site for professional, open and unmoderated discussion, relating to cloud computing interoperability
  3. A place to incubate actual working groups doing real technical proposals for specific areas of the cloud computing interoperability. We should encourage these proposals to target submission for consideration to an existing standards or industry organization if one exists.
Ok events are cool. There's one in London in a few hours that a bunch of people are apparently going to and I'd rather like to be there myself were it not for time.

A community site for professional, open and unmoderated discussion. Check. In fact that's what I've been asking for all along. Only CCIF is still none of those things:
  • Professional? Maybe, except for here, here, here, here, here, here, and my personal favourite, here. And that's just this week.
  • Open? Perhaps, if you call "invitation only" open. One can request an invitation but that's not anything like my understanding of open, especially when requests tend to get rejected, ignored, missed, etc. (like my first 2 attempts to join the unifiedcloud group)
  • Unmoderated? Only it's not according to Google Groups (above), despite a unanimous consensus to reject moderation. I sent the following post many hours ago seeking feedback on the Cloud Platform Reference Architecture and it still hasn't hit the list - another little white lie perhaps?
MIME-Version: 1.0
Received: by 10.210.78.3 with HTTP; Tue, 10 Mar 2009 13:58:29 -0700 (PDT)
Date: Tue, 10 Mar 2009 21:58:29 +0100
Delivered-To: xxxx@samj.net
Message-ID: <21606dcf0903101358m5a54e0aeq9fbfe980830c37f4@mail.gmail.com>
Subject: Re: Cloud Platform Reference Architecture
From: Sam Johnston 
To: Jayson Vantuyl 
Cc: Cloud Standards Group , CCIF 
Content-Type: multipart/mixed; boundary=0015174be85a5822b20464ca03c4
The last point's the best though: "A place to incubate actual working groups doing real technical proposals for specific areas of the cloud computing interoperability". But wait, CCIF "it is NOT to be a standards body" so what's all this about "actual working groups" doing "real technical proposals"? Your guess is as good as mine. Apparently the idea is to tell the various standards bodies what to do, which is not a mode of operation I believe they're accustomed to. Fortunately there's already some initaitives underway with traditional standards bodies that I'll talk about later (and more in the pipeline) so that won't be necessary.

Reuven goes on to acknowledge that there's a number of different splinter groups trying to "get things done" (with some early successes I might add), offering up his "extensive global network of corporate sponsors & industry organizations" to "help you and your technical proposal get in front of the right people when the time is right"(?). He finishes up with a reference to the oh-so-standard-like unifiedcloud group who are having a meeting soon.

It all sounds good and it probably pacified a lot of people, but in reality nothing has changed. Reuven's claimed the group is open (it's not) and unmodereated (it is) and yet this is all I've been asking for the whole time - an unfettered communication channel so we can get on with getting some work done. When one of the only people actually doing something has to wait hours for each message while a troll can spam freely something is drastically wrong.

Basically until Enomaly and CCIF either deliver on cloud computing open source and open standards respectively or get out of the way and let the rest of us I'm going to be here to draw attention to their shenanigans and tomfoolery as and when it happens. Stating that the forum is "professional, open and unmoderated" when it is so obviously, provably not is particularly disingenuous. That said it doesn't hold a candle to downplaying critical vulnerabilities or concealing them altogether (2.2.1 contained a second attempt at fixing CVE-2008-4990 and CVE-2009-0390) and then moderating the vulnerability annoucement!

Update: Since this post the spammer has given up (or has he?), Reuven's sent a postcard from Sweden (resulting in more noise in Swedish and a pair of posts about sick children) as well as pushing his mate's new offering (which incidentally looks quite good) and the Unified Cloud Interface (UCI) bunch led by Pat Wendorf (one of Enomaly's minions) has held a meeting with all of three attendees (two vendors pushing their private cloud offerings and another vendor pushing an unfortunately named "Cloud API" OGF group that might actually get something done) talking about a cut-back demo that might be semi-functional in 3 weeks to "attract some good minds" aka free labour.

Conspicuously absent from all of this noise are the various posts about the new Cloud Standards Group work on cloud platforms which have had ample time to be approved by the various moderators of this "professional, open and unmoderated" group. Yet more proof that the CCIF is a complete and utter vendor-driven farce and why RSA just jumped on this sinking ship is beyond me.

10 March 2009

On the Google Docs sharing security incident

I was just trying to respond to ZDnet's hot-off-the-press article (The cloud bites back: Google bug shared private Google Docs data) about the recent Google Docs sharing vulnerability but ZDnet's servers are throwing errors. Anyway now that Google have announced that they "believe the issue affected less than 0.05% of all documents" (rather than just emailing the affected users) I was considering writing a post anyway so killing two birds with one stone:
It's convenient that they should prefer to use a percentage of an unknown number rather than a meaningful statistic, but given that sharing even a single document inappropriately could destroy a business or someone's life it is still very serious. Fortunately I've not heard of any such incidences resulting from this breach (then again often you won't).

Putting it in perspective though, for the same sample of documents over the same period how many do you think would have suffered security breaches under the "old way" of storing them locally and emailing them? And by security breaches I include availability (loss) and integrity (corruption) as well as confidentiality (disclosure).

People still lose/steal latops and leave data laden USB keys all over the place so I don't see that this is much different from before and may well be better. Security researchers need statistics though so it would be useful if vendors were more transparent with information about breaches.
It would be great to see some more objective analysis and reporting comparing cloud computing with legacy systems - I'd say the fear mongerers would be surprised by the results.

Here's some tips that cloud vendors should ideally try to follow:
  • Work with researchers to resolve reported issues
  • Always be transparent about security issues (even if you think nobody noticed)
  • Limited liability is not an excuse to be negligent - always write secure code and test thoroughly
  • Remember that at least until cloud computing is widely accepted (and even thereafter) you are in the business of trust, which is hard to gain and easy to lose.
That's all for today - back to cloud standards...

Cloud Computing Standards: Cloud Platform Reference Architecture

Almost a year ago I said that cloud standards were premature, noting that traditional standards bodies are not known for being "fertile grounds for agile innovation" (which is not necessarily a bad thing by the way). Back then I said "what we need are simple, rugged, market tested interfaces defined by the innovators in each area (virtualisation, storage, services, etc.)" and sure enough some of these APIs are now starting to gain some traction (e.g. AppEngine, EC2 and S3).

You may also have just read that over the weekend I got sick of waiting for the Cloud Computing Interoperability Forum (CCIF) to produce something (it has after all been ~6 months already) and declared it dead and doghoused. Rather than being one to complain without offering a solution, earlier today I kicked off a new cloud standards workgroup. No invites. No resumes. No dedicated websites. No moderation/censorship. No fancy graphics. No sponsors. No added salt. Just standards. For more see the:
This was all done under the Cloud Computing Community umbrella, with a view to getting some runs on the board fairly quickly. I'm pleased to annouce that after the first day of researching existing implementations and talking to platform vendors we already have a useful deliverable.

The majority of the attention around cloud standards has thus far been focused at the bottom of the cloud computing stack, in the physical and virtual infrastructure itself. That's understandable because that's where most of the pain is today, and with it most of the opportunity. Ultimately though the real value of cloud computing is further up the stack in the platform and application layers. Accordingly this is where I started: a generic reference architecture for cloud platforms like Google App Engine and Salesforce's Force.com.

Work is already underway on a Debian GNU/Linux based reference implementation but not being one to talk about unreleased software I'll tell you more about that later, for a start take a look at the diagram and the various implementations on which the consensus was based (copied and pasted from the Cloud Standards Wiki):

Platform


Reference Architecture


This cloud platform reference architecture is compatible with all of the existing platform deployments, allowing for developers to submit application code, data and metadata via Git and/or Subversion, WebDAV, or in the simplest case (per Google App Engine) native HTTP.
YAML is recommended for metadata as it is both human and machine readable, less fragile and complex than XML and supports complex data structures. There are also stable implementations in most common programming languages.
Runtime management should be available via REST over HTTP with an optional web interface.

Source Code Management (SCM)

Requirements
Uploading of application code (in source and/or object form) and data, as well as metadata required for the operation of the application (which can, for the sake of simplicity, be stored in file(s)).
Actors
  • Developers need to upload code, data and metadata for execution on the platform. Version control support (e.g. for collaborative development, rollbacks, compliance, etc.) is optional but desirable.
Existing Implementations
Compatibility is sought with the following existing implementations:
The Google App Engine appcfg.py tool uses an RPC over HTTP interface (appengine_rpc.py) to upload code, data and metadata to the cloud.
The Eclipse based Force IDE retrieves and saves code in the cloud using the Metadata API (as opposed to the usual Web Services API. There are options for CRUD and file (zip) based transactions and SOAP, WSDL and WS-I are supported. See the FAQ for more information.
Although Heroku has a web based source code editor, they also have a Git interface:
sudo gem install heroku heroku create cloud-standards git push heroku
Joyent's Smart platform uses Git exclusively, but the repositories are independent of the management console (which "checks out" the latest version of the code pushed up from developer machines).
Subversion is a mature server-based version control system with obvious roots in traditional CVS. In addition to working with local file systems and with a dedicated 'svn' protocol (optionally over ssh for security), it also works over HTTP with WebDAV/Delta-V.
Git is a distributed version control system whereby every Git working directory is a full-fledged repository with complete history and full revision tracking capabilities, not dependent on network access or a central server. Although native Git relies on SSH for security (which requires additional key management tasks) there is also an option of using Git over HTTP/WebDAV (git-http-push) which would likely be preferable in a cloud environment.
WebDAV is a set of extensions to HTTP that allows users to collaboratively edit and manage files on remote World Wide Web servers, essentially making the web read/write. Web servers including Apache has good WebDAV support built in (mod_dav).
  • Others?

Management

Requirements
Runtime monitoring and management needs to be catered for by cloud platforms, allowing actors to access performance data, modify application configuration, access billing information, etc.
Actors
  • Developers need to manage non-production environments (e.g. development, test, staging).
  • System Administrators need to be able to monitor and modify the configuration of running applications as well as change control.
  • Managers need to be able to monitor the performance of applications and manage billing and payment.
  • Users may have an interest in monitoring the performance of the application over time, learning about outages, upgrades, etc.
Existing Implementations
Runtime management of applications deployed on cloud platforms is currently done via proprietary web interfaces with little automation. There are no known platform management APIs at this time.
  • Any?
Update: Preempting any questions about intellectual property, there's no patents that I'm aware of for any of this and with a descriptive/generic name there should be no trademark problems too, so need to concern yourself with a re-run of today's CCIF "XMPP is too stuffy" debacle.