News Feature

Many Working Together: Massively Parallel Projects

By Fred Hapgood

February 01, 2002 — CIO — Product names are not always obvious?think of the anxiety drug Alprazolam, for instance?but sometimes marketers get to go home early. One such moment passed in the early ’90s when a computation technology called massively parallel processing (MPP) launched its bid for CIO mind-space. The name came ready-made from the scientific and technical domain, where it referred to the practice of throwing very large numbers of processors at resource-hungry problems such as fluid dynamics, seismic processing, molecular modeling and astrophysical simulations.

The term resonated with a developing issue in enterprise computing: By the early ’90s CIOs were well aware that resource demands were going to grow forever?or at least until well past retirement. That prospect underlined the need for an IT architecture that could expand smoothly, simply and indefinitely by merely plugging in identical computing units one after another. Conventional, sequential computers did not fit the picture because there were limitations inherent in depending on a single processor. Even if a system contained several chips, one would always need to act as a manager for all the subprocessors. Inevitably this top processor and its memory buffers would become overwhelmed. When that happened the IT department would need to rethink the whole system and rewrite lots of software. MPP, on the other hand, seemed to promise plug-and-play scalability over the long term.

We liked the logic, and in a July 1993 article called "Divide and Conquer," we argued that MPP looked like a viable possibility for companies facing exceptional scalability problems. Admittedly, we noted, there were reprogramming, support, training and maintenance issues that needed to be thrashed out, but as more companies ran into the limits of conventional architectures, the market for solutions to these problems could only grow. "The cycle has begun," we said.

There may have been some sense that this was true. Still, had we known that a couple of years later most of the product lines (and some of the companies) mentioned in the article could be dead, our tone would have been cooler. The programming issue turned out to be particularly lethal. For example, one processor requires about a hundred instructions to send a message to a second processor. There is no logic in stepping through a hundred instructions just to send one; the sending processor could save 99 cycles by simply executing that one instruction locally. As a result, messages sent between MPP processors need to carry at least a hundred instructions just to break even. Therefore, MPP programmers had to think about more than just the immediate function. Their programs had to accumulate packages of instructions for each of the processors?it’s like FedEx freight planes flying between two cities. If the packages were too large, some processors would be left waiting; too small, and they would fail to pay for the "stamp"?the processing cost of the transaction.