Emerging Technology: The Truth Behind InfiniBand

It isn’t every day that a new technology arrives that promises to boost connections to dazzling speeds, shrink the size of servers and perhaps even ease California’s power crisis. And it certainly isn’t every day that hardware and software vendors work together on a specification. Perhaps that’s why InfiniBand?the result of a collaborative effort?isn’t just any technology. “It’s definitely something big,” says Tom Macdonald, general manager of Intel’s Hillsboro, Ore.-based advanced components division.

Something big is what the InfiniBand Trade Association?a 225-company consortium formed by Compaq Computer, Dell Computer, Hewlett-Packard, IBM, Intel, Microsoft and Sun Microsystems?had in mind last October when it ratified a specification that calls for removing I/O from individual machines and distributing it across a switched fabric. “The association wanted to create a network bus for the 21st century,” says Jim Bowers, a product manager at IBM’s microelectronics unit in Burlington, Vt.

InfiniBand is designed to replace the nearly decade-old PCI bus?a shared, general purpose interconnect?with a connection that can juggle several messages simultaneously and transmit each as if full network resources were devoted to it. InfiniBand’s supporters boast that the technology can eliminate the data flow bottleneck that’s inherent in the PCI bus and allow Internet data centers to take advantage of the new era of high-speed networks. Advocates also note that InfiniBand allows administrators to hook multiple servers together so that they can work as one?boosting performance and promoting efficiency. “Clearly the PCI bus is running out of steam,” remarks Dan Tanner, a senior analyst of storage and storage management for the Aberdeen Group, a Boston-based technology market research company.

But before InfiniBand can work any of its promised miracles, it must overcome several critical hurdles. These obstacles include the firmly entrenched technologies that would be replaced by Infini-Band, interoperability problems and a widespread?though largely unspoken? feeling among many CIOs that the status quo is good enough. “The killer application is not as clear-cut as one would hope,” says Vernon Turner, vice president of global enterprise server solutions at Framingham, Mass.-based IDC (a sister company to CIO’s publisher, CXO Media). “Yet the benefits and advantages are all there to be discovered.”

Banquet of Features

While most new technologies provide only a handful of benefits, InfiniBand promises an entire smorgasbord of new features and enhancements. Some of the tempting advantages include a fast and scalable transfer rate, intelligent channel adapters that offload much of the communications processing workload from the system’s processor, a modular design that can cut hardware costs, remote access capabilities for linking to distant customers and suppliers, and improved load sharing management. Other features that allow IT departments to easily work with an array of transport media include virtually unlimited network expansion as well as direct support for copper, optical and printed circuit wiring.

InfiniBand will become a standard part of server motherboards by 2003, says Peter Urban, a senior analyst at AMR Research, a technology research company in Boston. He notes that Intel’s recently released 64-bit Itanium chip will be the first processor to adopt the technology. Vendors benefiting from the technology include Vieo and Lane15, both in Austin, Texas, which create the software that makes InfiniBand work, as well as server makers that will be able to target more demanding applications. “Manufacturers of open databases, like Oracle, should also see performance improvements as the pipe between shared disks and nodes is fattened,” Urban says.

For CIOs, InfiniBand’s prime benefit will be greater computing capacity from comparable resources?or equivalent capacity from fewer resources?because of a reduction in data-traffic congestion among hardware devices. The technology would also simplify a data center’s infrastructure by presenting a common fabric to interconnect server, switch and storage components. “In doing so, the data center’s operations dramatically improve its total cost of ownership,” Turner says.

How much money will InfiniBand save its adopters? “This is hard to quantify today, since there are no benchmark or performance metrics to compare to,” Turner says. Still, the technology’s raw power?scalable transfer rates of 500MBps to 6GBps, compared to PCI’s 132MBps to 1GBps pace?leads experts to believe that for most adopters the overall savings will be substantial. Urban believes that InfiniBand will be a better value than the status quo. “You get faster performing servers, which will translate to faster performing applications and better e-commerce,” he says.

For all of its power, InfiniBand has a price that isn’t expected to reach to infinity. Although it’s too early to project system costs, InfiniBand technology isn’t expected to significantly drive up server hardware prices, especially considering the performance advantages. “The pricing can’t be excessive,” says Urban.

With Closed Arms

Despite the growing hoopla surrounding InfiniBand, not everyone is welcoming the technology with open arms. Organizations with substantial investments in what will soon become legacy servers and related hardware are viewing the technology with a combination of skepticism and concern. “InfiniBand has the capability of being classed as a disruptive technology, causing paradigm shifts in the way we look at the three primary components of servers, storage and networks,” says IDC’s Turner.

One way that InfiniBand would disrupt the status quo is by supplanting an entire range of established standards. Besides PCI, InfiniBand promises to displace such familiar technologies as PCIx (an enhanced PCI bus), iSCSI (a storage networking protocol) and fibre channel (a storage networking standard). Businesses have made substantial investments in those technologies and will be reluctant to ditch them. Although InfiniBand servers may be able to work with some legacy products as soon as middleware vendors develop the necessary solutions, the new technology still represents a quantum change for CIOs and their staffs.

Even when faced with InfiniBand’s numerous benefits, a number of CIOs will undoubtedly resist the technology, convinced that they simply don’t need the new capabilities. But Lauri Vickers, a senior analyst at Cahners In-Stat Group, a technology research company in Scottsdale, Ariz., thinks that most of those CIOs will find themselves being pushed over to InfiniBand before too long. She compares the technology’s introduction to the way PCI replaced ISA technology on desktop PCs several years ago. “Eventually, if you wanted the latest and greatest video card and you wanted the best sound card, you could only find them in the PCI format,” she says. “It’s something that just happens, whether you want it or not.”

Interoperability headaches loom as yet another potential InfiniBand roadblock. Ambiguities in the InfiniBand specification, which leaves certain communications functions to vendor discretion, could create conflicts between products. But Alisa Nessler, Lane15’s president and CEO, doesn’t believe that will develop into a major problem, since a wide range of InfiniBand vendors are already working with each other as InfiniBand Trade Association members. “I think you’ll see vendors reach a consensus on interoperability issues fairly rapidly,” she says.

Power Saver?

One InfiniBand attribute advocated by some supporters is the technology’s potential power-saving capability?an important consideration for energy-conscious organizations, especially those with operations in California. There’s no need for a network interface card between an InfiniBand server and an InfiniBand switch, Vickers says. “If you eliminate enough of them, you can start powering some homes.”

PCI cards can consume as much as 30 watts of power, claims Eyal Waldman, chairman and CEO of Mellanox Technologies, an InfiniBand silicon provider in Santa Clara, Calif. “InfiniBand replaces those PCI cards and could potentially reduce power consumption by 33 percent,” Waldman says. Vickers notes, however, that an organization would have to dump a lot of network cards to see any meaningful power savings.

Whether or not InfiniBand turns out to be a significant power saver, the technology’s numerous other benefits appear to guarantee it a place on the acceptance fast track. IDC’s Turner sees no reason why InfiniBand won’t ultimately succeed. “I believe that the risks are for those not adopting this technology,” he says. AMR’s Urban is urging CIOs to wait on major server purchases, “because if you buy something right now, two years down the road the thing’s going to be totally obsolete.” Aberdeen’s Tanner is even more succinct: “InfiniBand will become the industry standard.”