As IT departments walk the tightrope between boosting business outcomes and managing costs, it\u2019s become more crucial than ever for CIOs to gauge the productivity of their staff. How did Bob and Jill perform in the value stream while delivering product X to the business?\n\nThere\u2019s just one catch: CIOs have long lacked efficient mechanisms to do this. In a digital era drenched with tech that enables us to hail rides, control lights and file expense reports from a smartphone, most CIOs can\u2019t accurately measure the productivity and quality of the contributions of their staffs.\n\nWhile many IT leaders are addressing this data gap with do-it-yourself (DIY) approaches, or relying on old-school self-evaluations and performance reviews, startups are producing so-called \u201cworkforce analytics\u201d tools to help mitigate the issue.\n\nEquating employee workflows to server workloads\n\nThink of it this way: IT leaders can calculate the cost to run each server or storage array against the value it delivers. But some IT leaders are hungry for a way to analyze employee performance, as if they were a server to be tuned, reconfigured or redeployed based on business requirements.\n\nBenchmarking job performance versus the cost of each employee is something with which Tyson Foods CTO Scott Spradley has been wrestling because each role at the chicken production giant is different. For instance, the value ratio varies between a data scientist and a software engineer. \u201cIt\u2019s about taking a taxonomical value approach to each person in the workforce chain,\u201d says Spradley.\n\nTo date, approaches to quantifying labor have been old school. Companies pay McKinsey, Bain & Co. and other big-name consultants millions of dollars to assess workforce performance. They assign professional consultants to shadow employees for weeks or months, take notes, write up evaluations, make recommendations and move on to the next gig.\n\nBut the value of this approach has come into question as digital technologies have disrupted every business sector. It\u2019s no longer enough to simply clear Net Promoter Score targets or look inward to crunch employee satisfaction metrics.\n\nSimply, the nature of work has changed. A decade ago companies built software in \u201cwaterfall\u201d cycles, taking 18 to 24 months to ship products that were lucky to meet the businesses\u2019 needs. Today companies deploy \u201cagile\u201d teams that build software in one- to two-week sprints. IT and business siloes have yielded to cross-functional teams, with software programmers and UX designers co-located with product managers and business analysts. This confluence of speed and personnel hodgepodges can confound the savviest of consultants.\n\nCIOs need new tools that measures output down at the operational process and, yes, even task level. \u201cWe need a different style of management than how much work we can crank out,\u201d says Gartner analyst Bill Swanton. The phrase \u201cworkforce analytics\u201d will do.\n\nDIY analytics approaches\n\nMany CIOs are embracing the do-it-yourself approach to workforce analytics.\n\nWhile working as CIO of Aflac, Julia Davis created baseline metrics for measuring productivity, such as tasks done per full-time employee, per month. Tasks ranged from help desk requests, such as fixing printers, to code changes. When Davis progressed to analyzing application development, she ran into challenges measuring the individual performer in agile. However, consistent with bottoms-up approaches to management, Aflac\u2019s \u201cself-policing\u201d agile teams kept schedules and team members on track, says Davis, who retired from the company in 2018.\n\nFor the past two years, TD Ameritrade has used an internal methodology and tool it calls Go Live, which measures the business functionality generated by an agile team on a given sprint, says CIO Vijay Sankaran. Initially, Go Live measured the number of features generated for each app, but has since expanded to gauge the performance and productivity of each engineer to assess whether they are improving over time.\n\nEncouraged by the results, Sankaran\u2019s team is now broadening Go Live to gauge the various wait states for an application as it wends its way through an agile development cycle. Go Live scrutinizes an app from conception to story creation to code completion and so forth, all the way to its launch into production. Finding room for improvement, the team has since created an Objective and Key Results (OKR) metric to optimize the idea-to-production pipeline.\n\n\u201cThere are so many different operational levers to think through; what do you use, where do you place optimizable resources on agile teams, and at what investment levels,\u201d Sankaran says. \u201cThis art needs to be demystified, but it requires a big cultural change on the line-of-business side.\u201d\n\nTo analyze how TD Ameritrade uses software and hardware in total, Sankaran is also adopting technology business management (TBM), an increasingly popular methodology of aligning the cost to manage IT with business value.\n\nVendors set sites on workforce analytics\n\nIT departments have historically used spreadsheets to execute these analyses, but many are jettisoning this approach in favor of TBM analytics software from Apptio, which has also begun to tackle the thorny problem of scrutinizing labor costs.\n\nRecognizing that labor costs account for 50 percent of the cost structure of technology in an enterprise, Apptio last year launched Agile Insights, a SaaS solution that measures value delivery, cost of quality and labor utilization.\n\nCompanies can use the tool to, for instance, analyze code check-ins, cost-per-storypoints and other metrics generated through agile sprints in project management apps such as Atlassian\u2019s Jira and benchmark it against the financials in their ERP systems, Apptio CEO Sunny Gupta says. \u201cIt\u2019s about getting out of the anecdotal and getting into informed, quantitative decision-making,\u201d he says.\n\nOther startups are joining the fray. Fin Analytics is building software that seeks to help enterprise better understand how employees complete their daily tasks. The company, venture-backed by Kleiner Perkins, Accel, and CRV, is selling a Chrome browser plug-in that records employees\u2019 mouse scrolls, website clicks and browsing behavior, Fin co-founder Sam Lessin tells CIO.com. It also tracks application consumption, such as how much time each employee spends in Slack, or how much time they spent poring over a slide deck presentation. The tool records video and audio, creating a moment-by-moment process catalog.\n\nFin\u2019s software serves this information up in statistical dashboards, highlighting key metrics that managers can use to coach team members.\n\nLessin says that taking the process-level approach to improving productivity and performance is a differentiator from most analytics, which measure business outcome metrics, such as NPS and CSAT (customer satisfaction) scores. \u201cNo one has gone super deep on the process measurement to understand how the things we\u2019re doing are impacting our work,\u201d Lessin says, adding that dozens of customers use Fin today.\n\nHealthy skepticism around workforce analytics\n\nSuch a quantified approach to the workforce poses its challenges. Anna Frazzetto, chief digital technology officer of Harvey Nash, recalls efforts companies took a decade ago to determine the efficacy of software programmers. One programmer may program a widget writing 1,000 lines of code, while another may program the same tool writing 200 lines. But if the widget crafted in 200 lines has more defects, the efficiency factor goes out the window. \u201cIn my humble opinion, they haven\u2019t been able to solve the problem,\u201d Frazzetto says.\n\nIt\u2019s also hard to compare teams of people working on different products, says Swanton, the Gartner analyst. Given varying degrees of difficulty, a UX developer working on one product can\u2019t be evaluated the same as one working on another product. Ditto someone developing an AI algorithm, whom a CIO would hardly ask, \u201cHow many lines of code did you write today?\u201d\n\nMoreover, Swanton says CIOs who try to measure worker performance run the risk of seeming as if they are rewarding or punishing people in a way that discourages teamwork, a veritable death sentence for agile.\n\nSwanton\u2019s recommendations for measuring output harkens back to something most CIOs are laser-focused on today: Judge each product or agile team on the value that is created, or the business outcome achieved. Does the product deliver a measurable improvement in business capability? Does it generate revenue or curb costs? If it does any of these things, it\u2019s a win, Swanton says. \u201cThe goal of IT is to make the business work better,\u201d Swanton says. \u201cThat\u2019s the metric that really matters.\u201d\n\nWhile Lessin acknowledges some corporate leaders may be concerned about scaring employees with this virtual over-the-shoulder approach to process improvement, he says the heightened emphasis on providing superior customer service will win over skeptics and appeal to data-driven CIOs.