Democratizing high performance computing

By Ed Turkel

In the course of my work as a high performance computing (HPC) strategist at Dell EMC, I encounter many misperceptions about HPC. One of the biggest, coming from people outside the industry, is that HPC is a niche field of computing that lives largely in the academic and government laboratories. While HPC does play a prominent role in academia and government, it has grown to include many other verticals and markets as a critical enabler for innovation and competitiveness.

In reality, HPC is now widely used in commercial, industrial and consumer applications. It’s at the heart of use cases as diverse as product design, financial analytics, personalized medicine, marketing automation, fraud detection and autonomous vehicles. Even though the backend systems may be mainly invisible to users, HPC touches our lives in countless ways throughout the day.

The rising importance of HPC is reflected in figures from industry analysts, who report that HPC systems now account for nearly a quarter of all server sales (ref. IDC). Looking ahead, analysts expect HPC to account for an even larger percentage of the server pie in the years to come, as more companies, including many small and midsize businesses, understand the importance and the opportunity as they catch the HPC wave.

At the highest level, this upward slope for HPC systems is driven by two overarching factors: need and accessibility. As for the need, to innovate and stay competitive, many companies now require HPC to do things like test and validate product designs in a fast and cost-effective manner. And as for the accessibility, HPC systems are now more available than ever before. We’re really talking about the democratization of HPC.

Let’s dive a little deeper and look at four trends that are fueling the widespread adoption of HPC systems.

Trend 1: The pervasive use of HPC in industrial applications.

In today’s world, you can’t design a car or an airplane without HPC systems. HPC is essential for pursuits as diverse as developing pharmaceuticals, probing the human genome, and searching for oil and gas deposits. Have you watched any movies dominated by Computer-Generated Imagery (CGI)? You’re experiencing the power of HPC systems.

Trend 2: The rise of high performance data analytics

Organizations are collecting enormous amounts of data from connected devices on the Internet of Things, social media, and their own corporate systems. However, data alone won’t do anything for you. To extract value from the data, you must couple big data platforms with data analytics and machine learning systems powered by lightning-fast HPC clusters. This convergence has given rise to a hybrid form of HPC — high performance data analytics (HPDA). Although it’s a fairly new field, HPDA has a great deal of momentum, and you’re going to be hearing much more about it in the years to come.

Trend 3: Cloud-based access to HPC clusters

Cloud service providers now make HPC systems available to the masses. For small and midsize organizations, cloud access to HPC levels the playing field. They can gain the benefits of HPC systems on demand without making large upfront capital expenditures. Larger companies, in turn, can extend the capabilities of their in-house HPC systems with a hybrid approach that allows them to shift certain workloads to the cloud to meet spikes in demand. The real value of HPC in the cloud is with a hybrid cloud model that provides an optimal mix of availability of HPC resources, flexibility to match changing HPC workloads, and cost economies.

Trend 4: Off-the-shelf HPC systems

Various technology companies now offer pre-configured HPC systems that are optimized for particular workloads. At Dell EMC, for example, we offer engineered HPC systems that are tailored to the needs of research, life sciences and manufacturing, along with fully configured systems for all-purpose HPC workloads. We even offer hybrid solutions that transparently offload workloads to cloud-based HPC systems. Compared to do-it-yourself approaches, these bundled and packaged offerings make it much easier to configure, buy, deploy and manage HPC systems.

The key takeaway here is relatively simple: HPC is now not only essential to our daily lives and the success of our commercial and industrial pursuits, it’s more accessible than ever before. Long gone are the days when HPC clusters lived mainly in the labs of academia and government. High performance computing is everywhere, and the future will bring much more of the same as the democratization of HPC continues.

Ready for a deeper dive? To explore the technologies behind HPC systems, and to find links to educational resources, visit Dell.com/HPC.

Ed Turkel is a high performance computing strategist at Dell EMC.