While it’s well known that the cloud helps organizations “do more with less,” the follow up question of “how much?” is far less known. IT rarely asks that question, yet the business does ask, and the business wants answers. If IT is going to better align themselves with business operations, they must be able to objectively quantify the value of the cloud.
Looking to the wisdom of the crowd, we reached out to experts and asked, “How do you measure the value the cloud brings to your business?” As the responses came in we discovered there’s far more than one or even 20 different ways to measure the success of the cloud. A myriad of technical and business dynamics can be a definition of success. More importantly, what may work for one organization may not be appropriate for another. As you look through the following advice, pick and choose what’s right for your organization. Note: Please don’t skip tip number one. It’s appropriate for all businesses.
1: Define what ROI means for your business
“You need to define what ROI means for your organization first, before you identify your KPIs [key performance indicators],” advises Jeff Frankel (@docstarsoftware), executive vice president and principal at docSTAR.
“The KPIs that are used to measure the success of a cloud initiative should similarly be the ones used to measure impact on the overall success of the company itself,” adds Matt Podowitz (@mpodowitz), senior director at Pine Hill Group.
“Before implementing the cloud projects, have business/IT alignment up front to define what the business values are,” says Terence Ngai (@TerenceCNgai), head of cloud go-to-market and solution delivery at HP.
Podowitz agrees: “Measuring cloud success using technology rather than business KPIs diminishes the cloud effort to nothing more than a technology project, not a technology-enabled transformation of the business capable of creating real value.”
“The only KPIs for the success of moving to the cloud are how well it achieved your reason for moving to the cloud in the first place,” concludes Nicko van Someren (@good_technology), CTO, at Good Technology.
2: Obviously, Compare costs
For best cost analysis, M.J. Johnson (@threeeyedtoad), director, product marketing, at Akamai Technologies, suggests measuring “reduced hardware and software costs, reduced resources required to maintain infrastructure, plus eliminated on-premise and co-located data centers including leased lines for connectivity.”
“The KPI that measures success in the cloud is more related to cost avoidance. If we add 100 users to an on premise application, what will that cost us in terms of hardware, infrastructure, licenses and help desk costs over a defined timeframe,” adds Dean Wiech (@dwiech), managing director for Tools4ever.
3: Don’t just focus on cost savings
“Many companies make the mistake of only measuring hard cost savings associated with cloud, which frequently leads to misaligned expectations during the implementation phase,” says Charles Moore (@delphix), product marketing, at Delphix.
Keep that in mind as you read the rest of the advice, which addresses mostly non-cost-based ways to measure the success of your cloud initiative.
4: Measure revenue impact
“Cloud technologies allow you to more granularly align tech operations KPIs to your current revenue and business KPIs,” adds Scott Maurice (@scottjmaurice), managing partner at vail Partners. “You no longer have to take yearly budget and estimate costs; you can directly assign a technology cost metric to a specific revenue-generating application and by extension, specific revenue.”
5: Focus on customer KPIs
“Deployment of cloud, or any technology for that matter, should involve tight assessment of what it means for end customers of the business,” says Greg Johnsen (@gregjohnsen), CMO of GT Nexus, who suggests measuring the on-time delivery of goods to customers and the number of days you’re holding inventory.
Dave Davis (@DaveDavis), managing director at Redfly Marketing, adds “We've found that customer churn rate, average revenue per customer, free to premium up-sell rate, and what we call ‘time to slip away rate’ are what's worth focusing on from a customer perspective and a financial perspective.”
Additional customer focused KPIs “would include an increase in transaction completion, reduction in cart abandonment, and an increase of spinoff sales for the B2C world,” suggests Steve Prentice (@stevenprentice), senior writer, at CloudTweaks.
6: Calculate ‘time to daily use’
“We like to focus particularly on a KPI we call ‘time to daily use’ which measures how long a cloud product takes to become indispensable to a customer. The shorter this time, the less chance a user will slip away,” says Redfly Marketing’s Davis.
7: Survey user satisfaction
“One of the easiest and probably most underutilized ways [to measure the impact of the cloud] is to conduct employee satisfaction surveys across line-of-business (LOB) apps,” says Todd Schwartz (@GetSkyKick), co-CEO and co-founder of SkyKick.
“KPIs based on surveys not only demonstrate that a process is shorter and cheaper, but that the users are also happier with the new cloud-based set up,” says Max Dufour (@maxdufour), partner at Harmeda.
Schwartz’s experience confirms Dufour’s theory: “What we hear from our partners about their customers is that SaaS-based apps are easier to use, make employees happier, and provide more value to end users.”
8: Gauge market responsiveness
“The KPIs for the cloud are the same they should have always been for IT in the past,”says Jeff Kaplan (@thinkstrategies), managing director of THINKstrategies. “They are greater agility and responsiveness to market demands, and ability to identify and pursue new market opportunities.”
Akamai’s Johnson concurs: “The benefit resulting from instant-on cloud-based capabilities can be measured in time to market, newfound agility to deploy new applications and enter new markets, and increased customer acquisition/decreased customer attrition.”
“Cloud deployed in a global supply chain should enable a manufacturer to know the true cost to serve a market and foster better decisions based on these insights,” says GT Nexus’ Johnsen.
For example, Alastair Mitchell (@alimitchell), co-founder and president, Huddle, has “worked with pharmaceutical companies who’ve managed to dramatically reduce the time to market of new drugs through improved cloud-based collaboration during clinical trials and with universities who’ve been able to connect research teams who are dispersed across the globe.”
“Cloud also enables organizations to move fast and deploy quickly without having to involve finance or purchasing or to think about deployment factors (space, networking, setup, etc.),” adds Gerardo Dada (@gerardodada), VP, product marketing and strategy at SolarWinds.
9: Evaluate speed and control
While speed was a common theme we saw across many KPIs, it had to be balanced with the ability to actually conduct business.
“Success for IT in the ‘cloud’ is a complex mix between allowing speed (pace of new applications, operate at speed of the business, new APIs, new microservices) while still providing IT control (governance around data access and quality, security, identity, uptime SLAs, etc.),” says Chris Purpura (@ChrisPurpura), VP of digital enterprise strategy at MuleSoft.
10: Determine the opportunity cost
For example, Steve Herrod (@herrod), managing director, General Catalyst Partners, suggests you ask yourself, “How many employee-hours go toward the non-differentiated application and infrastructure work can be replaced by IaaS or SaaS offerings?”
In addition, the cloud affords companies to rapidly experiment. Without the cloud, the experiment opportunity is lost.
“Leveraging cloud-oriented assets can enable a business to quickly design, implement and test several choices in a timely, cost-effective manner,” continued Bordogna.
11: Don’t overlook continuity of business operations
The ability of the cloud to be always on and operational, without the excessive need for in-house technical staff, is a value add that should not be ignored.
“If a company suffers a server failure, security breach or problem due to simple human error, companies that deploy cloud-empowered data protection solutions are assured that their business remains operating smoothly. It is the continuance of operations KPI that showcases the success of the cloud,” explains Dave LeClair (@LeClairTech), VP, product marketing, at Unitrends.
This continuance extends beyond not only just keeping the lights on, but also to ongoing product deployment.
As MJ DiBerardino (@cloudnexa), CTO of Cloudnexa, notes, “When you are developing a new product, service or platform, the cloud allows you to maintain a continuous deployment model for your development operations.”
12: How does the cloud foster innovation?
“The biggest distraction for cloud adoption is the continuous focus on reduced costs on IT infrastructure. True value is gained from business gain agility with the new found ability to rapidly innovate,” says Larry Carvalho (@robustcloud), research manager and lead analyst – PaaS for IDC.
Shortening the TTI requires improving the ease and speed of collaboration.
“Measure connected worker engagement, the resources required to maintain engagement, and retention as it relates to delivering the business goals,” suggests Eric Hanson (@fuze), VP of strategic initiatives at Fuze.
13: Can you identify and solve a specific problem?
“The cloud allows IT professionals to focus on the actual business problem rather than focus on the intermediate technical disciplines,” says Dan Carney (@llnw), VP, operations at Limelight. “This is very healthy as it drives the IT staff [who understands the technology] to engage in business discussions about how the user [who does not necessarily understand or care about the technology] perceives the service.”
In one example, Honeywell, a high-tech company, outfitted its sales staff with a low-tech solution: phonebook-sized catalogs tied together with shower curtain rings. It wasn’t a pretty sight, and it definitely didn’t instill confidence in potential buyers. With the help of ArcTouch, they solved the problem by producing an easy to access catalog application that was connected to Salesforce data.
“With the new solution, Honeywell staffers could instantly capture new business leads, share meeting outcomes, define next steps, and set reminders,” says ArcTouch’s Shapiro. “At a single two-day trade event, the app helped Honeywell capture more than $1 million in new business leads, which more than paid for the solution we developed.”
14: Calculate mean time to solving problems
“The big KPI for cloud infrastructure for internal development is average time between bug report and solution deployment,” says Jeffrey Bolden, managing partner at Blue Lotus SIDC. “Most companies that move toward a cloud culture see reductions of 30 percent to 80 percent in this key metric of IT responsiveness.”
“Time spent on a key process or cycle is another good KPI,” adds Harmeda’s Dufour.
This could be the time spent on back-office tasks such as the time handling and processing invoices, suggests GT Nexus’ Johnsen.
15: Compute the time to develop apps
Neal Bradbury (@IntronisInc), co-founder and VP of channel development at Intronis recommends measuring “provisioning speed, or how quickly a business is able to set up a working application in the cloud.”
“This KPI measurement includes the time to prepare the infrastructure/VMs for dev, QA, staging, and production, which is often a major delay in enterprise shops,” adds Bernard Sanders (@BernardCBolt), CTO of CloudBolt Software.
You need to also include your ability to not have to seek third party support, and instead use the development resources and tools of your choice.
“Sure, you need to ask yourself if you’re creating something that will enrich the experience for your customers, but more importantly, are you doing this quickly enough to realize the benefit?” asks Michael Henry (@executionmgmt), CIO at Digital Realty.
16: Measure velocity, not agility
While tip #8, gauge market responsiveness, is valuable, it is still difficult to measure. Other types of speed, which many experts recommend, are far easier.