While it\u2019s well known that the cloud helps organizations \u201cdo more with less,\u201d the follow up question of \u201chow much?\u201d is far less known. IT rarely asks that question, yet the business does ask, and the business wants answers. If IT is going to better align themselves with business operations, they must be able to objectively quantify the value of the cloud.\nLooking to the wisdom of the crowd, we reached out to experts and asked, \u201cHow do you measure the value the cloud brings to your business?\u201d As the responses came in we discovered there\u2019s far more than one or even 20 different ways to measure the success of the cloud. A myriad of technical and business dynamics can be a definition of success. More importantly, what may work for one organization may not be appropriate for another. As you look through the following advice, pick and choose what\u2019s right for your organization. Note: Please don\u2019t skip tip number one. It\u2019s appropriate for all businesses.\n1: Define what ROI means for your business\n\u201cYou need to define what ROI means for your organization first, before you identify your KPIs [key performance indicators],\u201d advises Jeff Frankel (@docstarsoftware), executive vice president and principal at docSTAR.\n\u201cThe KPIs that are used to measure the success of a cloud initiative should similarly be the ones used to measure impact on the overall success of the company itself,\u201d adds Matt Podowitz (@mpodowitz), senior director at Pine Hill Group.\n\u201cBefore implementing the cloud projects, have business\/IT alignment up front to define what the business values are,\u201d says Terence Ngai (@TerenceCNgai), head of cloud go-to-market and solution delivery at HP.\n\u201cCompany strategy and employee goals should inform which tools and technology you choose \u2013 not the other way around,\u201d says Clara Liang (@claraliang), Chief Product Officer of Jive.\nPodowitz agrees: \u201cMeasuring cloud success using technology rather than business KPIs diminishes the cloud effort to nothing more than a technology project, not a technology-enabled transformation of the business capable of creating real value.\u201d\n\u201cThe only KPIs for the success of moving to the cloud are how well it achieved your reason for moving to the cloud in the first place,\u201d concludes Nicko van Someren (@good_technology), CTO, at Good Technology.\n2: Obviously, Compare costs\n\u201cThe value of the cloud is around overall cost cutting,\u201d says Michael Weiss (@Oildex), VP, software engineering at Oildex.\n\u201cCompare previous expenses around on-premise datacenter management to the expenses of cloud hosting,\u201d suggests Kevin Fishner (@KFishner), director of customer success, at HashiCorp.\nFor best cost analysis, M.J. Johnson (@threeeyedtoad), director, product marketing, at Akamai Technologies, suggests measuring \u201creduced hardware and software costs, reduced resources required to maintain infrastructure, plus eliminated on-premise and co-located data centers including leased lines for connectivity.\u201d\n\u201cThe KPI that measures success in the cloud is more related to cost avoidance. If we add 100 users to an on premise application, what will that cost us in terms of hardware, infrastructure, licenses and help desk costs over a defined timeframe,\u201d adds Dean Wiech (@dwiech), managing director for Tools4ever.\n3: Don\u2019t just focus on cost savings\n\u201cMany companies make the mistake of only measuring hard cost savings associated with cloud, which frequently leads to misaligned expectations during the implementation phase,\u201d says Charles Moore (@delphix), product marketing, at Delphix.\nKeep that in mind as you read the rest of the advice, which addresses mostly non-cost-based ways to measure the success of your cloud initiative.\n4: Measure revenue impact\n\u201cThe closer an organization can tie a cloud project to revenue, the better,\u201d says Eric Shapiro (@ArcTouch), CEO and co-founder of ArcTouch.\n\u201cCloud technologies allow you to more granularly align tech operations KPIs to your current revenue and business KPIs,\u201d adds Scott Maurice (@scottjmaurice), managing partner at vail Partners. \u201cYou no longer have to take yearly budget and estimate costs; you can directly assign a technology cost metric to a specific revenue-generating application and by extension, specific revenue.\u201d\n5: Focus on customer KPIs\n\u201cDeployment of cloud, or any technology for that matter, should involve tight assessment of what it means for end customers of the business,\u201d says Greg Johnsen (@gregjohnsen), CMO of GT Nexus, who suggests measuring the on-time delivery of goods to customers and the number of days you\u2019re holding inventory.\nDave Davis (@DaveDavis), managing director at Redfly Marketing, adds \u201cWe've found that customer churn rate, average revenue per customer, free to premium up-sell rate, and what we call \u2018time to slip away rate\u2019 are what's worth focusing on from a customer perspective and a financial perspective.\u201d\nAdditional customer focused KPIs \u201cwould include an increase in transaction completion, reduction in cart abandonment, and an increase of spinoff sales for the B2C world,\u201d suggests Steve Prentice (@stevenprentice), senior writer, at CloudTweaks.\n6: Calculate \u2018time to daily use\u2019\n\u201cWe like to focus particularly on a KPI we call \u2018time to daily use\u2019 which measures how long a cloud product takes to become indispensable to a customer. The shorter this time, the less chance a user will slip away,\u201d says Redfly Marketing\u2019s Davis.\n7: Survey user satisfaction\n\u201cOne of the easiest and probably most underutilized ways [to measure the impact of the cloud] is to conduct employee satisfaction surveys across line-of-business (LOB) apps,\u201d says Todd Schwartz (@GetSkyKick), co-CEO and co-founder of SkyKick.\n\u201cKPIs based on surveys not only demonstrate that a process is shorter and cheaper, but that the users are also happier with the new cloud-based set up,\u201d says Max Dufour (@maxdufour), partner at Harmeda.\nSchwartz\u2019s experience confirms Dufour\u2019s theory: \u201cWhat we hear from our partners about their customers is that SaaS-based apps are easier to use, make employees happier, and provide more value to end users.\u201d\n8: Gauge market responsiveness\n\u201cThe KPIs for the cloud are the same they should have always been for IT in the past,\u201dsays Jeff Kaplan (@thinkstrategies), managing director of THINKstrategies. \u201cThey are greater agility and responsiveness to market demands, and ability to identify and pursue new market opportunities.\u201d\nAkamai\u2019s Johnson concurs: \u201cThe benefit resulting from instant-on cloud-based capabilities can be measured in time to market, newfound agility to deploy new applications and enter new markets, and increased customer acquisition\/decreased customer attrition.\u201d\n\u201cCloud deployed in a global supply chain should enable a manufacturer to know the true cost to serve a market and foster better decisions based on these insights,\u201d says GT Nexus\u2019 Johnsen.\nFor example, Alastair Mitchell (@alimitchell), co-founder and president, Huddle, has \u201cworked with pharmaceutical companies who\u2019ve managed to dramatically reduce the time to market of new drugs through improved cloud-based collaboration during clinical trials and with universities who\u2019ve been able to connect research teams who are dispersed across the globe.\u201d\n\u201cCloud also enables organizations to move fast and deploy quickly without having to involve finance or purchasing or to think about deployment factors (space, networking, setup, etc.),\u201d adds Gerardo Dada (@gerardodada), VP, product marketing and strategy at SolarWinds.\n9: Evaluate speed and control\nWhile speed was a common theme we saw across many KPIs, it had to be balanced with the ability to actually conduct business.\n\u201cSuccess for IT in the \u2018cloud\u2019 is a complex mix between allowing speed (pace of new applications, operate at speed of the business, new APIs, new microservices) while still providing IT control (governance around data access and quality, security, identity, uptime SLAs, etc.),\u201d says Chris Purpura (@ChrisPurpura), VP of digital enterprise strategy at MuleSoft.\n10: Determine the opportunity cost\n\u201cA less obvious KPI for measuring cloud success is opportunity cost, or the cost of the best alternative foregone,\u201d says Ray Bordogna (@RayBordogna), partner and chief strategy officer at LiquidHub.\nFor example, Steve Herrod (@herrod), managing director, General Catalyst Partners, suggests you ask yourself, \u201cHow many employee-hours go toward the non-differentiated application and infrastructure work can be replaced by IaaS or SaaS offerings?\u201d\nIn addition, the cloud affords companies to rapidly experiment. Without the cloud, the experiment opportunity is lost.\n\u201cLeveraging cloud-oriented assets can enable a business to quickly design, implement and test several choices in a timely, cost-effective manner,\u201d continued Bordogna.\n11: Don\u2019t overlook continuity of business operations\nThe ability of the cloud to be always on and operational, without the excessive need for in-house technical staff, is a value add that should not be ignored.\n\u201cIf a company suffers a server failure, security breach or problem due to simple human error, companies that deploy cloud-empowered data protection solutions are assured that their business remains operating smoothly. It is the continuance of operations KPI that showcases the success of the cloud,\u201d explains Dave LeClair (@LeClairTech), VP, product marketing, at Unitrends.\nThis continuance extends beyond not only just keeping the lights on, but also to ongoing product deployment.\nAs MJ DiBerardino (@cloudnexa), CTO of Cloudnexa, notes, \u201cWhen you are developing a new product, service or platform, the cloud allows you to maintain a continuous deployment model for your development operations.\u201d\n12: How does the cloud foster innovation?\n\u201cThe biggest distraction for cloud adoption is the continuous focus on reduced costs on IT infrastructure. True value is gained from business gain agility with the new found ability to rapidly innovate,\u201d says Larry Carvalho (@robustcloud), research manager and lead analyst \u2013 PaaS for IDC.\nJason Dover (@jaysdover), director of product line management, KEMP Technologies, refers to this KPI as \u201ctime to innovation (TTI).\u201d\nShortening the TTI requires improving the ease and speed of collaboration.\n\u201cMeasure connected worker engagement, the resources required to maintain engagement, and retention as it relates to delivering the business goals,\u201d suggests Eric Hanson (@fuze), VP of strategic initiatives at Fuze.\n13: Can you identify and solve a specific problem?\n\u201cThe cloud allows IT professionals to focus on the actual business problem rather than focus on the intermediate technical disciplines,\u201d says Dan Carney (@llnw), VP, operations at Limelight. \u201cThis is very healthy as it drives the IT staff [who understands the technology] to engage in business discussions about how the user [who does not necessarily understand or care about the technology] perceives the service.\u201d\nIn one example, Honeywell, a high-tech company, outfitted its sales staff with a low-tech solution: phonebook-sized catalogs tied together with shower curtain rings. It wasn\u2019t a pretty sight, and it definitely didn\u2019t instill confidence in potential buyers. With the help of ArcTouch, they solved the problem by producing an easy to access catalog application that was connected to Salesforce data.\n\u201cWith the new solution, Honeywell staffers could instantly capture new business leads, share meeting outcomes, define next steps, and set reminders,\u201d says ArcTouch\u2019s Shapiro. \u201cAt a single two-day trade event, the app helped Honeywell capture more than $1 million in new business leads, which more than paid for the solution we developed.\u201d\n14: Calculate mean time to solving problems\n\u201cThe big KPI for cloud infrastructure for internal development is average time between bug report and solution deployment,\u201d says Jeffrey Bolden, managing partner at Blue Lotus SIDC. \u201cMost companies that move toward a cloud culture see reductions of 30 percent to 80 percent in this key metric of IT responsiveness.\u201d\n\u201cTime spent on a key process or cycle is another good KPI,\u201d adds Harmeda\u2019s Dufour.\nThis could be the time spent on back-office tasks such as the time handling and processing invoices, suggests GT Nexus\u2019 Johnsen.\n15: Compute the time to develop apps\nNeal Bradbury (@IntronisInc), co-founder and VP of channel development at Intronis recommends measuring \u201cprovisioning speed, or how quickly a business is able to set up a working application in the cloud.\u201d\n\u201cThis KPI measurement includes the time to prepare the infrastructure\/VMs for dev, QA, staging, and production, which is often a major delay in enterprise shops,\u201d adds Bernard Sanders (@BernardCBolt), CTO of CloudBolt Software.\nYou need to also include your ability to not have to seek third party support, and instead use the development resources and tools of your choice.\n\u201cThis is important because skill sets vary and most enterprises don\u2019t have the mobile skills they need,\u201d notes Sravish Sridhar (@sravish), founder and CEO of Kinvey.\n\u201cSure, you need to ask yourself if you\u2019re creating something that will enrich the experience for your customers, but more importantly, are you doing this quickly enough to realize the benefit?\u201d asks Michael Henry (@executionmgmt), CIO at Digital Realty.\n16: Measure velocity, not agility\n\u201cAgility is difficult to measure, but velocity is not,\u201d argues Ariel Tseitlin (@atseitlin), partner at Scale Venture Partners.\nWhile tip #8, gauge market responsiveness, is valuable, it is still difficult to measure. Other types of speed, which many experts recommend, are far easier.\n\u201cMeasure the time it takes to procure new servers. Measure the time from idea inception [product requirement] to production deployment. These will all give a good proxy for velocity,\u201d adds Tseitlin. \u201cMeasuring the number of new features deployed can give an indication of agility, but that is often a very coarse measure.\u201d\n17: Quantify the reputation of the IT group\nCloudBolt Software\u2019s Sanders jokingly suggests that \u201cthis all important KPI can be calculated by subtracting the number of curse words people use when talking about their IT department from the number of positive adjectives they use.\u201d\nOn a more serious note, \u201ccloud success is no different than all IT success,\u201d adds Heinan Landa (@OptimalNetworks), CEO of Optimal Networks. \u201cMeasure the number and severity of issues that your people have with the service.\u201d\n18: Determine the security metrics that measure exposure\n\u201cThe essential proactive security question is, how easily could someone break in? Building this as a security KPI involves understanding the assets and their networked relationships,\u201d says Dr. Mike Lloyd (@dr_mike_lloyd), CTO of RedSeal. \u201cThe ideal outcome and measurement benchmark is risk \u2013 the probability of a bad thing happening, times the downside of that event. Reducing risk is a complex business, but for purposes of cloud infrastructure, the essential point is that you must have enough visibility to answer a question like \u2018How easily could a bad guy break in and steal assets?\u2019 or \u2018How easily could a compromise of one cloud server [or service] spread to damage the rest of my organization?\u2019\u201d\n19: Calculate availability\n\u201cThe cloud should \u2018always be there\u2019 \u2013 even if part of it is broken, it should look there,\u201d says Sean Molloy (@TenableSecurity), VP, cloud services at Tenable Network Security.\n\u201cIf you have hardware or system failures in one environment, then you can move those users to another environment without impact to the customer,\u201d says Oildex\u2019s Weiss regarding the benefit of cloud availability.\nAvailability measurement works best \u201cfor cases where cloud consumers leverage the geographic diversity and redundancy options that the cloud offers,\u201d adds Craig McElroy (@contegix), co-founder and CTO at Contegix.\n20: Enforce KPIs to determine the success of your SLAs\n\u201cKPIs should naturally feed into the service level agreements (SLAs) you have with your cloud provider. KPIs are what will define success or failure in your SLA,\u201d says Setu Shah (@setushah2), business solutions manager at Orange Business Services.\nSteve Falkin (@HBR_Consulting), managing director at HBR Consulting, suggests that your performance-based and financially-oriented KPIs should be part of a contractual SLA with the service provider. Such KPIs to measure and back up with an SLA could include availability, recoverability from a disaster, turnaround time for applications and compute resources, plus actual costs and resource utilization versus what was projected.\nConclusion: What\u2019s your KPI for not moving to the cloud?\nEven though the cloud seems like such a logical move for so many, that doesn\u2019t mean one can completely ignore all the questions that need to be asked.\n\u201cToo often leaders fail to probe deeply enough to uncover the true motivations behind a move to cloud. Probe,\u201d advises Shlomo Swidler (@ShlomoSwidler), CEO of Orchestratus. \u201cAsk \u2018why\u2019 repeatedly, until the answers can be measured in terms of cash flow. Along the way, you will have uncovered the KPIs that indicate cloud success.\u201d\nDISCLOSURE: David Spark and Spark Media Solutions work with Tenable Network Security.