With the increase in the amount of data produced and analyzed in the enterprise, the increase in peer pressure to invest and grow big data, analytics and machine learning expertise and value, and the increase in the funding of big data and analytics initiatives, enterprises are getting drowned in a sea of data, analysis, dashboards and data portals.
This problem will get more acute as pressure to generate new insights increases. Phrases such as “dashboard hell” or “too much noise” will become more common. Inevitably, the number of dashboards and the density of information packed in a dashboard can become unmanageable leading to more information that can be consumed by employees.
With an increase in analytical content available for inspection and usage, employees face new problems stemming from an overabundance of analytics and are tasked with finding the relevant analytics for their needs and jobs. This search and discovery activity is in addition to employees having to correctly interpret the analytics and subsequently determine whether it meets their needs or has to be extended or modified.
Challenges in the path of being data-driven
The data-driven enterprise faces the following challenges:
- Employees have to search for and discover relevant analytical content.
- Employees have to understand the provenance and lineage of the analytical content to correctly understand underlying assumptions and biases.
- Employees have to interpret the analytical content correctly before using it to drive decisions and actions.
- Employees have to drive analytical agility into their existing job functions and workflows.
- Employees have diverse analytical acumen and analytical skills and interests.
The above list means that the challenge facing an enterprise that wants to be data-driven is only exacerbated once more data and analysis begins to show up around the enterprise. Another assumption that often enterprises can make is that every employee has the inclination and time to build and/or interpret analytics. This is often-overlooked assumption can turn out to be incorrect and end up being the reason why the adoption of analytics across an enterprise fails.
Among all these issues, it is important to keep in mind the reason why the enterprise has been on the analytics journey. A data-driven culture really means that all decisions and actions taken by employees are steeped in analysis and insight using the most factual, accurate and scientific analysis of comprehensive data that best represents the actual environment and context.
Achieving this strategic vision, with the understanding that not all employees want to or can be trained to be analytical experts, the requires a significant change in the thought process of how problems are tackled. I call it the “interrupt” strategy.
The interrupt strategy
The interrupt strategy stipulates that for every employee in an enterprise tasked with a certain set of tasks and goals, the enterprise provides notifications at the right times through the right channels that prompt employees to complete their jobs, providing the information they need and delivering the recommended or required action to improve the overall state.
Delivering value through interrupts
Designing, building and delivering interrupts that make employees more productive and efficient is essential to a successful interrupt strategy. The interrupt system needs to ensure that all subscribed employees are always reminded of what is expected of them, including the time and quality constraints. This ensures that employees are able to prioritize and plan accordingly to complete the task at hand.
The interrupt notifications should be encoded with the expected action and impact. Building prescriptiveness in interrupts, when possible, reduces the time required by the employee to strategize and plan. Prescriptiveness, especially when coupled with historical analysis of similar interrupts and subsequent interactions, can increase the quality and on-time completion rates of the required actions.
The interrupt system should also record key information about the action that was triggered by the interrupt, including the actions recommended, details about the actual actions that were taken, the time at which the action was taken and the impact of the action. This serves not only as an audit trail but can also be used to understand and improve the quality of the interrupts and the subsequent actions.
The impact of actions driven by interrupts should enable employees to free up their time to plan and organize around strategic activities as they rest assured that all required tactical actions will be preceded by timely, prescriptive interrupts.
Designing interrupts requires an innate understanding of the needs and expectations of each employee. Employees are targets for certain interrupts based on their roles, affiliations, job descriptions and the projects or programs that are currently their priorities. Thinking about spatial and temporal locality is very useful in designing the optimal interrupt design. Interrupts should be smart enough to recognize where and how each employee fits into the enterprise and what each employee would find relevant and contextual at a particular point in time.
Interrupt design requires the constant measurement and monitoring of relevance and recall. In addition to ensuring that all interrupts delivered to a particular employee are relevant to the employee, the system also needs to ensure that all relevant interrupts are being delivered to the employee. This means that if interrupts can help make employees better at any aspect of their jobs, they should receive those interrupts. And employees should find all of the interrupts they receive to be useful and necessary to perform their jobs.
Another aspect of interrupt design is the need to tune the frequency of delivery of the interrupts. Individual interrupts can be generated for each action that an employee needs to be concerned with, or several interrupts can be combined into one that notifies an employee of several actions. The frequency of the interrupts delivered needs to find the balance between noise leading to overexertion and underdelivery leading to missed opportunities.
Over time, the nature and context of interrupts can change, and therefore periodic, if not constant, monitoring of the interrupt generation mechanisms is required. Interrupts and their motivation can change at the macro level — i.e. the business context can change, thus removing the need for or changing the nature of the actions to be triggered by the interrupt. At the micro level, an employee’s responsibilities, job function, role, affiliations and associations with specific groups or projects can change, causing both the interrupt generation logic and targets to need to be updated.
A.I.- and ML-enabled interrupt generation
Generating and delivering interrupts can be achieved through several different techniques such as rules, heuristics or systems enabled by machine learning (ML) and artificial intelligence (A.I.).
A.I. and ML offer the ability to leverage and process more signals than humanly possible, leading to interrupts that otherwise could not have been generated or would not have been delivered to the optimal set of employees.
A.I. and ML can be used not only to generate optimal interrupts, but also to leverage the impact of the actions triggered by the interrupts to automatically adjust, update or improve the threshold generation process. In addition, A.I. and ML can be used to analyze the “actions” carried out by employees and their associated impact to improve the prescriptiveness quality of the interrupts.
When starting out with an interrupt strategy, enterprises should consider simple rules that generate interrupts for the most important use cases or scenarios in the enterprise. Starting out small not only makes the interrupt strategy more likely to be successful but also generates a lot of good information and data to enable the progression to more advanced interrupt generation techniques such as those enabled through A.I. and ML. Initially, interrupts can simply be notifications of past or present events, and over time, progressively, systems can grow to issue interrupts about impending events or required actions. As the interrupt strategy matures, interrupts can become automated, prescriptive and intelligent.