Speed and agility continue to drive how companies differentiate. In today\u2019s highly competitive environment, rapid application development and deployment is essential to helping businesses react and pivot. The ability to build apps quickly allows businesses to zig and zag with ease to respond to customer needs and capture revenue opportunities from major market changes \u2013 whether that\u2019s a global pandemic, new disruptive competitors, or supply chain interruption.\u00a0\u00a0\u00a0\nContainers to the rescue\nIf you\u2019ve been asked (or forced) to develop and deploy applications faster or to modernize your application estate, you\u2019ve probably already heard of containers or Docker or Kubernetes orchestration. Containers have taken the world by storm and are key to enabling application modernization for several reasons. They\u2019re portable from edge to cloud, they add agility to DevOps processes, and they bring simplicity and speed to application development and deployment.\u00a0\nThe good news is, if you\u2019re just getting around to modernizing your data and application estate \u2013 you\u2019re not too late.\u00a0 \u2018Application modernization\u2019 was once reserved for cloud native and microservices-based apps, but now there\u2019s a revolution underway to take those same techniques and apply them to your entire stateful, data-centric estate. Data scientists and IT admins have caught on to the modern trend to keep up with their rapidly changing toolkits and the fact that it often makes sense to develop in one location and deploy in another. In a recent study, 451 Research found that 94%1 of AI workloads delivered by 2022 will be deployed via self-service containers for these data- and analytic-centric workloads.\nToday\u2019s enterprise application modernization is multi-cloud for data analytics\n451 Research estimates that nearly two-thirds2 of today\u2019s current application estate still needs to be modernized.\u00a0 In general, that means the easy \u2018lift and shift\u2019 work has already been done \u2013 so now it\u2019s time to tackle the difficult clustered applications, data-centric applications, and analytic-heavy solutions.\u00a0\nAnother trend I\u2019m seeing is that single location deployments, be it on-premises or in a particular public cloud, limit application agility. The new reality is multi-cloud first architecture, and 451 Research findings show that to-be modernized workloads will be split 46% on-premises and 54%3 cloud deployments. This means you need to architect for multi-cloud. \u00a0\u00a0\u00a0\nISVs and open source are key to modernizing the remaining 63% of workloads\nSo what happens when the easy \u2018lift and shift\u2019 work is done and the piecemeal parts have been modernized? Unfortunately, unless enterprises want to completely re-architect their apps on their own, the majority of the remaining 63% of workloads are dependent on external modernization efforts. They require innovation by Independent Software Vendors (ISVs) and the open-source community, the two other key enablers of app modernization, to move them from monolithic to loosely-coupled deployments optimized for Kubernetes deployments.\nThis trend is in full swing. I\u2019ve witnessed ISVs like Dataiku, H2O.ai, and others spending the last few years optimizing solutions for the public cloud. They\u2019ve recently realized that a big chunk of their customer\u2019s workloads are actually still on-premises so they\u2019ve changed their focus \u2013 bringing this innovation to solution providers who can deliver on-premises performance and security and provide the bridge for consistent multi-cloud deployments.\nEven big names like Splunk and SAS have taken this path with their monolithic software to take advantage of modern, agile, and efficient K8s-based deployments. Splunk stands out to me because I was part of a major validation and benchmark with their beta Splunk operator for Kubernetes last year. We worked with a major US bank who couldn\u2019t physically scale the infrastructure to keep up with data growth rates. Running the traditional deployment of a single indexer or search head per server resulted in extremely low CPU utilization, requiring massive over deployment of infrastructure to keep pace with data growth. It took weeks to deploy new workloads, and as their data centers approached near maximum capacity, indexing fell behind \u2013 creating security blind spots. As a result, the bank was forced to find ways to optimize their delivery and consumption of Splunk.\nLeveraging Splunk\u2019s new operator for K8s, we were able to modernize the delivery of Splunk using containers to take full advantage of the optimized infrastructure to drive utilization and throughput. In minutes, we were able to independently scale from 1 to 6, and up to 12 indexers per host delivering an astounding 17X indexer throughput improvement (8.7 TB\/day per host) and driving CPU saturation up to 70%.\nI skipped over a lot of the details that required tight collaboration between HPE, Intel, Scality, and Splunk \u2013 but the point is that the customer needed the ISV to do the work to modernize their application. This in turn allowed them to fundamentally change the way this solution is deployed. As a result, we eliminated the security data blind spot with up to 17X higher data ingestion and indexing per host, flipped the TCO model by shrinking the infrastructure footprint by up to 10x, quickly addressed new use cases deploying new indexers and search heads in minutes, and balanced hot cache and S3 object storage for exabyte scale. And we even delivered the solution as a service!\nAs open-source software has become a key driver for modernizing existing workloads, enterprises have really warmed to open-source components over the past decade \u2013especially in the data science and data engineering communities. While most of the more recent tools are developed as cloud native out of the gates, many of the established and most widely deployed tools are still in the process of modernizing.\u00a0\u00a0\nApache Spark is the most obvious example of this. First released in 2009, it is now a key component of most open data analytics platforms for ETL, data science, and data engineering. As companies move away from Hadoop, Spark has evolved from deploying Spark on YARN to Spark on Kubernetes.\nFast forward to 2021 and Spark\u2019s modernization efforts have paid off. The\u00a0new Spark 3.x operator is now ready for primetime \u2014 delivering native GPU acceleration capabilities, S3 integration, and significant resiliency improvements. This type of open-source modernization will go a long way to modernizing that 63% of workloads that remain \u2013 and we\u2019re already seeing the container platforms and cloud offerings integrate in the new Spark operator to provide the surrounding components to make it enterprise grade.\u00a0 \u00a0\nTo learn more\nWith organizations still needing to modernize two-thirds of their current application estate, assembling the right key enablers such as containers, ISVs, and open-source software is essential and now within reach. Want to explore more about how to modernize your stateful, data-centric applications? Here are a few options to allow you to go deeper on this concept.\u00a0\u00a0\nRead the Pathfinder research paper from 451 Research, Application Modernization and the Age of Insight, and attend my session at HPE\u2019s Discover 2021 on how Containers Are Driving Digital Transformation. \u00a0If you prefer pictures, you\u2019ll enjoy this eBook, Fuel Edge-to-Cloud Digital Transformations, for real stories on how companies are successfully collaborating with HPE Ezmeral, ISV partners, and open source to modernize their data-centric applications. \u00a0\u00a0\n1,2,3 Pathfinder paper by 451 Research \u2013 Application Modernization and the Age of Insight, published June 2 2021\n____________________________________\nAbout Matthew Hausmann\n\nMatt\u2019s passion is figuring out how to leverage data, analytics, and technology to deliver transformative solutions that improve business outcomes. Over the past decades, he has worked for innovative start-ups and information technology giants with roles spanning business analytics consulting, product marketing, and application engineering. Matt has been privileged to collaborate with hundreds of companies and experts on ways to constantly improve how we turn data into insights.