Why the Enterprise Cloud Needs Shadow IT to Succeed

I stumbled across a ZDNet article by Larry Dignan linking to Rightscale’s 2015 State of the Cloud Report referencing how Azure’s use is growing and it’s now become an emerging No. 2 competitor to AWS, a la the famous Avis commercials of long ago, where Avis said “we’re number two but we try harder.” This aligns with the general perception that Microsoft seems to have gotten its feet under it and is charging hard into the public cloud.

After downloading the report, subtitled “Central IT is Taking the Lead to Broker Cloud Services to the Enterprise,” I believe the more interesting and important material is, so to speak, buried in the back pages — where the respondent’s adoption patterns and technology choices are surveyed.

Is IT Really in Charge of the Company Cloud?

And here, despite the subtitle, which implies that central IT is now taking the primary responsibility for cloud computing within enterprises, a different picture emerges — one in which central IT and business units hold very different perceptions of who is responsible for cloud computing decisions.

For example, in the chart (Figure 1) describing “Enterprise Views of Role of IT in Cloud,” there is an identical perspective from both organizations that IT should take the role to “Broker cloud services,” although it’s only 40 percent in both parties.

 

In other areas, however, the comity and alignment vanishes. Taking the question of who gets to select public clouds, 59 percent of central IT believes it’s their responsibility, while only 35 percent of business units agree. Perhaps more surprisingly, on the question of private clouds, 57 percent of central IT believe it’s their responsibility, but only 35 percent of business unit respondents agree.

These are vast disparities in how these two groups view the role of central IT, with Central IT viewing itself significantly more important in terms of making choices and executing decisions. One could go so far as to say this portion of the survey indicates very large differences between these two organizations as to the future of IT decision-making within the overall company.

The question is why do these two groups view the ground rules of cloud computing so differently?

One clue goes to the very heart of the matter, in understanding what cloud computing is, and where central IT stands in terms of implementation.

In one question, Rightscale queried respondents about how large the cloud computing environments they operate are. In terms of private cloud, respondents said (Figure 2) that 22 percent of them are running cloud environments with over 1000 virtual machines. Frankly, I find that astonishing.

A clue to understanding this comes later in the report, when Rightscale asked respondents what technology they were using to implement their private cloud. Even though Rightscale recognizes that “although virtualized environments from VMware or Microsoft don’t meet all of the requirements of cloud computing, many survey respondents currently view these environments as a private cloud, and so we gave respondents an option to identify them as private cloud technologies.”

As one can see in the chart describing the private cloud technology options IT organizations are using (Figure 3), a full 43 percent of the 74 percent of IT organizations claiming to operate private clouds are using vSphere or System Center. In other words, they are using virtualization products and calling them clouds. My comment to a colleague was this is like me calling my minivan a Ferrari because both have an engine inside of them. It’s a near certainty that nearly all of those 1000+ virtual machine clouds discussed above are really standard virtualization farms, not real cloud environments.

 

Further evidence of the tardy pace of the movement to private cloud is evinced in a recent report by the 451 Group called “Cloud precursor projects dominate enterprise agendas.” 451 Group has kindly allowed ActiveState to host and distribute a copy of this private report, which you can find here.

This report documents (Figure 4) how slowly the movement toward private cloud computing is progressing.

 

A full 55 percent of the 191 respondents are focused on standardization, consolidation, etc. — in other words, basic virtualization. Only 6 percent are focused on automation, the hallmark of cloud computing, although 14 percent are creating a private cloud. This latter figure probably reflects that these respondents are using a similar perspective of what constitutes a private cloud as in the Rightscale survey, i.e., they assess their vSphere or System Center implementation as being a private cloud.

Given these survey results, it’s not surprising that, when analyst Tom Bittman surveyed 140 attendees at his session at the Gartner data center conference, a full 95 percent said their private cloud initiative was failing, as seen in Figure 5.

 

Why Private Cloud Initiatives Are Failing

As you can see, there are a litany of reasons that private cloud initiatives are failing (although I thought the funniest reason was advanced in the blog post’s comments, where one person said private cloud failure was down to “Using technology incorrectly.” Gee, do you think?

Frankly, the data from these three surveys is appalling. A decade — a full decade — after Amazon launched cloud computing with its Amazon Web Services offering, central IT is still puttering around with virtualization, and claiming cloud computing victory because they’ve got it working.

Even 84 percent of the Rightscale respondents who claim to have or be building a self-service portal for cloud services (Figure 6) are, in my experience, putting up a Web page that users can launch pre-configured virtual machines; useful, but by no means capable of creating, deploying and managing complex, multi-VM applications streamlined by DevOps practices.

 

To be honest, this makes business unit self-sufficiency in cloud adoption completely comprehensible. Whether you call it “rogue IT” (the highly pejorative term frequently used by central IT groups to refer to business unit self-adoption), “shadow IT” (the slightly more palatable pejorative term that still depicts self-adoption as somewhat disreputable) or “Type 2 IT” (Gartner’s formulation of new IT practices adopted outside the bounds of traditional Type 1 IT, which I wrote about recently), companies can’t wait for IT to get its act together.

Why Shadow IT is Alive and Well

These surveys illustrate why shadow IT is alive and well. Business units are under tremendous pressure to infuse what they sell with IT goodness. Ray Kurzweil, in his decade-ago best seller “The Singularity is Near” stated that we are at an inflection point in our society and, going forward, everything is going to be an information product or service.

Facing that kind of pressure, and served by a supporting organization that, at best, is stumbling toward computing environments that will support what’s needed, and, at worst, is dissembling about what it’s actually delivering, business units are going to turn to whatever solutions they can find to solve their today problems today.

Traditional IT may have a role to play in solving those today problems, but it sure doesn’t seem like it’s prepared to do so if these surveys are taken at face value.