We routinely underestimate the effects that new technology has in the long run while also overestimating its impact in the short term. What has become known as Amara\u2019s Law, in honor of the late researcher and scientist Roy Amara, is playing out now in many organizations that adopt artificial intelligence and machine learning (AI\/ML) technologies. AI\/ML has capabilities that seemed science fiction not long ago, from autonomous driving to image recognition and advanced robotics, with surely many more on the way. At the same time, even advanced enterprises can struggle to achieve the anticipated return on their AI\/ML investments. Although expectations might have been set high, following several guidelines has helped our customers maximize the business value delivered by their AI\/ML initiatives. We\u2019ll share them in this AI\/ML blog post mini-series.\n\nThere Is No Magic: Acknowledge the Uncertainty\n\nThe pressure to deliver solutions faster and at lower cost while ensuring compliance, security, and reliability has caught many IT departments between a rock and a hard place. It\u2019s no surprise, then, that they\u2019re looking for new solutions to break through old barriers, and rightly so: modern technologies coupled with new ways of working can help transform IT from a cost center to an innovation driver. However, too often the latest technology is heralded as the panacea that\u2019ll cure all the previous technology\u2019s shortcomings. It\u2019s therefore important to remind ourselves that while modern IT solutions can overcome past limitations, there is no \u201cincrease my customer satisfaction\u201d algorithm, neither within AI\/ML nor in any other technology.\n\nAI\/ML solutions model human reasoning by allowing the computer to make judgement calls (\u201cinference\u201d) based on past positive or negative outcomes. Just as for human reasoning, the result remains an educated guess. Building more sophisticated models can increase accuracy, but the business value of that accuracy needs to be weighed against the cost of building and training the model.\n\nUpfront business cases that treat AI\/ML initiatives as if they were deterministic software programs run the risk of overinflating expectations. You can\u2019t expect an out-of-the-box AI\/ML model to already be perfectly calibrated to your business problem and immediately make good decisions. The value the solution delivers depends largely on how well you\u2019re able to balance investment and gain in the face of uncertainty. For example, if you\u2019re looking to use a new data source for ML (or any analytical process for that matter), you\u2019ll face a high degree of uncertainty, both for the required investment and the possible benefits. Let\u2019s say you want your ML product to use daily stock levels from your warehouses around the globe to decide which items to advertise. You can\u2019t be sure whether the data quality is sufficient for automatic processing, how much harmonizing data from the different warehouses will cost, whether daily stock levels are even relevant within your business processes, or how you can use that data to improve your top line or bottom line KPIs.\n\nMany enterprises got burned by taking an approach of \u201cput all the data into a data lake and find value later.\u201d Just adding \u201cartificial intelligence\u201d into the mix won\u2019t change that. Instead, reduce the uncertainty step by step while learning which investments impact your business value. Building on top of a cloud analytics and AI\/ML platform is a great way to limit the initial investments as your learning journey begins. Once on your way, use the following three mechanisms to continually improve and refine.\n\n1) Estimate the Value of Right Decisions and the Cost of Wrong Ones\n\nLet\u2019s assume you\u2019re looking to apply ML to improve sales forecasts for your consumer product, a common use case we assist many customers with. Such a system can achieve several positive business outcomes, including higher net sales by avoiding out-of-stock situations, increased customer satisfaction thanks to shorter delivery times, and cost savings due to operational efficiencies. A perfect ML model would work toward all three equally, but ML models aren\u2019t perfect.\n\nML models aren\u2019t clairvoyants. No matter how well trained, they make erroneous judgement calls, in both directions! The costs of those errors are determined by your business and are likely asymmetric. For example, overstocking seasonal goods might be as bad for your business as understocking them. Also, some customers might happily wait five days for a high-demand article as long as they have a firm delivery date. Given your cost of goods sold and storage cost, it might be more cost-effective for your organization to store items in a central warehouse and pay for premium delivery rather than maintain specialized or regional warehouses. As with any IT initiative, a clear understanding of your business economics is critical to improving your bottom line.\n\nIdentifying potential high-cost mistakes at the start of an AI\/ML project calibrates expectations and allows adjustments early on. It will also help you avoid building ML models that are penny-wise but pound-foolish. Because such mistakes come in many shapes, let\u2019s look at some possible scenarios for a retailer\u2019s sales forecasting:\n\nGoal: Increase net sales by forecasting demand and stocking high-demand articles beforehand.\n\nGoal: Decrease time to delivery and thus increase customer satisfaction. Use region-specific demand forecasts to replenish local warehouses.\n\nGoal: Increase operational efficiency by automating processes. Use demand forecast to automate replenishment and transportation orders.\n\nThe cost of these mistakes will depend on the dimension of the business value you\u2019re looking at. It\u2019s tempting to tweak an AI\/ML model to avoid each high-cost mistake, but doing so will increase costs, as completely new data sources might have to be integrated or spin-off models might have to be designed for certain article subgroups. So, look beyond the actual model to mitigate the high-cost mistakes. For example, you could couple the forecast more closely with your pricing strategy, campaign calendars, immediate customer feedback, or route optimization.\n\nSound complex? That\u2019s precisely the point. Estimating realistic cost and gain is not a one-dimensional, one-off calculation. That\u2019s true for conventional analytics and AI\/ML initiatives alike.\n\n2) Measure and Improve with Time\n\nBecause the route to business value is not a straightforward path, you\u2019ll want a continuous process that reduces uncertainty by gathering additional information from a working AI\/ML model. The likelihood of good decisions will increase over time if you keep honestly measuring, learning, and recalibrating the model along each value dimension. That\u2019s why Michelle Lee, VP of Amazon Machine Learning Solutions Labs, warns:\n\nThis doesn\u2019t mean recalculating business value after every sprint or small experiment. Instead, the AI\/ML development life cycle provides several natural milestones to recap leanings:\n\n3) Know When to Pivot\n\nIteratively calculating your AI\/ML solution\u2019s business value lets you adjust the solution according to what you\u2019re learning. Often, companies calculate a detailed upfront business case, which is prone to being based on unrealistic and fuzzy assumptions. When those assumptions meet reality later, companies then twist the project output to fit the original business case. The only thing this achieves, however, is cheat you out of business value.\n\nYou need to prepare to pivot and change course, whether that means adjusting the model type, acquiring additional (possibly external) data sources, or adapting the business process. It might mean switching to another use case altogether. Remember the above forecasting example: instead of trying to predict future demand ever more accurately, pivoting to a live view of all inventory (whether in production, in stock, or in transit) can enable a quick and lean reaction to customer demand, even if forecasted inaccurately. Similarly, one can pull levers to influence demand by dynamically adjusting pricing and campaigns, rather than seeing these as unconstrained inputs. This can not only decrease the forecasting error rate but also increase full-price sales. It might be worth experimenting whether search terms or customer comments on the website have predictive power for short-term demand.\n\nNo one can foresee what the best pivotal moves are for your ML model and your business. And that\u2019s okay. ML solutions aren\u2019t copy-paste exercises based on other organizations\u2019 successes. They are tools to maximize the value you derive from your assets or to overcome your specific constraints.\n\nUtilizing Emerging Tech Means Constant Recalibration\n\nA deeper understanding of AI\/ML methodologies can help you realize greater business value. An end-to-end understanding of how your business dynamics interact with the capabilities of emerging technologies, however, can achieve much more.\n\nThat\u2019s why AI\/ML isn\u2019t just another technology project. Remember these core points:\n\nThese steps are closely tied to making your organization data driven, whether it includes AI\/ML technology or traditional analytics. Any of the above calculations can and will change over time, along with your business and customer demand. Incorporating this evolution into a sustainable AI\/ML software life cycle will be the topic of our next post.\n\nAUTHOR BIOS:\n\nAnnina Neumann is AI\/ML Strategist at AWS Professional Services. In her role, she supports AWS customers to utilize emerging technologies for business value generation. Annina has been a lead advisory across diverse industries, such as retail, media or manufacturing. She has 15 years of experience with all things data, ranging from hands-on data science to leading cross-functional data technology teams.\n\nIn his role as Enterprise Strategist at Amazon Web Services, Gregor advises technology leaders in the transformation of both their technology platform and their organization. Drawing on his experience as Smart Nation Fellow to the Singapore government and as Chief Architect at Allianz SE, he connects the corporate strategy with technical decision making and vice versa. He enjoys sharing his thoughts on architecture and architects in his books, including The Software Architect Elevator and Cloud Strategy.