Understanding the market for elastic cloud services

There is much excitement about the emergence of elastic cloud services to provide hosted infrastructure capacity on demand, with some pundits predicting a huge associated market shift. But will elastic cloud players such as Amazon with its EC2 offering really inherit the earth? Key points:

• The market for traditional fixed capacity contract based hosted services is well established and will continue to grow as the appetite for infrastructure outsourcing increases.
• Emerging elastic cloud offerings will complement traditional hosting by providing more flexibility for handling variable and high performance workloads in particular.
• Activity is relatively limited at the moment, but early indications are that elastic cloud will start out as an extension of the established hosting market.
• Short to medium term activity in the mainstream will primarily be concerned with incumbent providers selling to customers already comfortable with hosted infrastructure services.

The idea of utility computing has been around for a long time, but until relatively recently the ‘pay as you go’ approach to consuming IT infrastructure services never caught on beyond a few niche areas such as high performance computing (HPC) and a handful of bleeding edge early adopters. As little as three or four years ago, for example, players like Sun under the then trendy ‘grid computing’ banner were struggling to drive demand for their offer of computing capacity on demand. Then seemingly overnight, Amazon hit the market with its EC2 offering and immediately caught the imagination of journalists and analysts. Talk of a huge market transformation around the notion of flexible cloud based delivery of infrastructure capacity escalated, and played a big part in fuelling the IT industry obsession with all things cloud that is with us today. Some pundits have even gone to the extreme of predicting the death of the enterprise data centre as a result of all this. Coming back down to earth, Freeform Dynamics recently conducted a survey of IT professionals in the UK, USA and a range of other geographies to investigate the role of elastic infrastructure services against the broader backdrop of the existing hosted services marketplace. The hypothesis we were testing was that elastic services would complement traditional fixed capacity contract based services (that dominate the hosting market today) rather than replacing them, and would be particularly relevant for dealing with certain types of workload.

The survey was executed online, and to avoid the sample being too heavily skewed towards those with an interest in or knowledge of cloud, the study was billed as an investigation of server computing in general, and respondents were only presented with hosted services related questions once they had got some way into the questionnaire. The picture we see in terms of market penetration is therefore as accurate as you are likely to get by using this kind of methodology, and what it confirms is a healthy existing market for traditional hosed services (Figure 1). This chart was derived by aggregating lower level responses to questions relating to individual workloads, and summarises the overall use of traditional hosted services at the moment across organisations of all sizes. If you are interested in the detail behind it, check out the report entitled ‘Evolution of Hosted Server Computing’, which is available for download from the Freeform Dynamics website. In the meantime, our survey also confirmed an emerging market for elastic services (Figure 2). As we can see, about one in twenty from our sample currently have a significant ongoing commitment to this kind of service, with a further 10 per cent claiming more limited or occasional use. While this might seem low compared to the amount of noise we hear from press and analysts, who sometimes give the impression that on demand services are already steamrollering across the market, it’s actually not a bad result given the short period of time that elastic services have been properly available with the right delivery model in both technical and commercial terms. For the avoidance of doubt, however, significant use does not mean running the whole of your IT on outsourced infrastructure, or even the majority of it. It simply says that in one or more areas, you are making use of the services concerned on a regular or committed basis. So what are the areas in which elastic cloud services are beginning to find their place?

Well as we anticipated, the ‘hard core’ use at the moment seems to revolve around more variable or high performance workloads (Figure 3). The big caveat here of course (if you look at the scale on this chart) is the relatively small size of the sub-sample of respondents behind the picture we see, and we must be wary of the associated statistical error. If we home in on the more significant and sustained activity, however, and aggregate the responses across the various types of variable workload activity, the picture is quite striking and unlikely to be accounted for by pure chance (Figure 4). Looking at this, it is noteworthy that those with experience are putting less emphasis on the use of elastic services for workloads with more static demand profiles. There should be no surprise here as despite some experimentation in this area at the moment, it isn’t where the long term opportunity lies. Relatively static and predictable core workloads associated with everyday business applications gain little benefit from elasticity, and the commercial reality is that fixed capacity contract based services will probably always be more cost effective for dealing with these. Given that flexibility usually comes at a price premium (that’s just the way of the world), there is no point paying for it if you don’t really need it. And thus we have our first reason why pure-play elastic service providers such as Amazon are unlikely to take over the whole market, as what they are offering, while representing an important new outsourcing option, only deals with part of the problem in a mainstream context. Beyond this, we then have to consider that there is already a community of established players in the hosting market, and many of these are now also offering elastic hosting as an extension of their service portfolios. Furthermore, a couple of important factors exist that are likely to stack things to their advantage. Firstly, established providers branching out into elastic cloud provision represent a single source of delivery for the full spectrum of hosted services customers will require, under the same support and customer management framework. This will clearly be attractive as it means simplicity and convenience for those who like to minimise the number of suppliers they deal with, and avoids the kind of disjoints and conflicts that can arise when multiple providers are in the mix. Secondly, incumbent providers obviously have existing customer bases to which they can cross-sell and up-sell elastic services. This gives them a head start in the drive towards critical mass that is important to achieving economies of scale that in turn are the foundation for good and sustainable business. With these factors in mind, it is interesting to note that early adoption of elastic services is mostly associated with customers that are already committed to traditional hosting (Figure 5). Whether this dependency between the use of static and dynamic hosting models will continue is debatable. The big question is whether those organisations that have hitherto not been interested in traditional hosted services for more static requirements will recognise the arguably more acute benefits of elastic hosting in relation to highly variable workload requirements that are often associated with a high cost of ownership due to poor system utilisation rates and high management overheads. Many, however, are tackling such problems through the use of virtualisation technology within their own on premise infrastructures, so elastic hosting is not the only option for dealing with such challenges. Either way, when looking to understand the future of the market, it is probably not the glamorous and exciting pure-plays we should be focusing on, but the established players who already have an intimate knowledge of their customers’ requirements and a foundation of incumbency and trust upon which to build. And let’s not forget that there is a lot to think about with any outsourcing arrangement in areas such as security, data protection, service levels, and so on, so the importance of trust and track record should not be underestimated. All we then need is for finance and IT management to become more comfortable with the concept of elastic budgets and the market will be cooking …. but that’s another discussion.