Intel flexes AI processing muscle

Cloud and datacenter architects searching for new ways to pack more artificial intelligence horsepower into already constrained spaces will want to take a close look at Intel’s new Nervana Neural Network Processors. Depending on the application, the processors may offer four times the performance or one-fifth the power draw as commercially available alternatives.

The new processors are Intel’s first ASIC offerings tailored specifically for deep learning workloads. The company announced last week the processors are shipping now.

In addition to the NNP-T1000 for training and the NNP-I1000 for inference, Intel also announced the coming generation of the Movidius Myriad Vision Processing Unit, which is designed for AI vision and inference processing at the edge. The ASIC, code-named Keem Bay, is scheduled to ship in the first half of next year.

ASICs are the inevitable response from the semiconductor industry to the young, fast-growing and intensely performance-hungry market for AI processing, which thus far has been served largely by general-purpose CPUs like Intel’s Xeon and GPUs like those built around Nvidia’s Tesla and Turing architectures.

In practice, cloud and datacenter deployments typically center on GPUs for training, due to their highly parallel nature, and CPUs for inference due to their superior programmability and flexible memory addressing. But the insatiable demand for performance per square inch is giving rise to new, more optimized solutions.

Hardware decisionmakers increasingly have been looking to field-programmable gate arrays, or FPGAs, from companies like Xilinx and Intel, which entered the market four years ago by acquiring Altera. FPGAs offer the potential for more efficient performance because the circuitry can be tailored for specific workloads. Despite the performance benefits, however, FPGAs have not taken off faster because they are comparatively more difficult to work with, less flexible and more expensive than either CPUs or GPUs.

Enter ASICs

ASICs represent the next turn of the screw in AI processing, because they are more efficient and cost-effective than the other alternatives. In fact, Google has deployed its own internally-designed ASIC – called the Tensor Processing Unit, or TPU – specifically to tackle deep-learning workloads. Intel’s new NNPs may be the first AI-specific ASICs from a major supplier to hit the open market for cloud and datacenter applications.

With the potential to pack far more AI performance into the same space – at significantly lower power – the NNPs could give Intel a big leg up in deep learning computing in datacenters and in the cloud. In fact, datacenter OEMs tell me they’re seeing a lot of early interest from customers wanting to test evaluation NNP boards and systems.

There’s no guarantee, of course, that all the tire-kicking will translate into a wholesale shift in the market. To be determined is how the software development ecosystem will respond in making it easy for developers to tap into the new processors’ deep-learning capabilities. Indeed, Nvidia’s CUDA development platform may be the biggest impediment to Intel expanding its footprint in AI because CUDA is so widely adopted.

Toward that end, Intel is investing heavily in its own ecosystem with the hope of luring developers. At its AI event last week, in fact, Intel unveiled the AI DevCloud for the Edge, a cloud-hosted platform for building and testing AI systems. In addition to the AI DevCloud, Intel also offers its OpenVINO toolkit for vision applications as well as its upcoming oneAPI initiative to develop across all of Intel’s hardware AI architectures.

That will come in handy. In addition to the new NNP and VPU ASICs, Intel disclosed its first GPU architecture designed for the datacenter on Sunday, just ahead of Supercomputing 2019, which begins today in Denver. Boards based on the upcoming GPU architecture, dubbed Ponte Vecchio, will be available in 2021.

With its existing CPU and FPGA offerings, the new additions will give Intel the most diverse set of AI processing options available.

Still, that’s no guarantee for success. Because the best hardware won’t win the day unless developers can use their favorite tools to build with them. Can Intel achieve that? Watch this space. I will, too.