‘Clustering’ of PCs Gains Traction

When immigrants landed on New York’s Ellis Island in the 19th and early 20th centuries, record-keeping consisted primarily of scribbling a few notes into a big book. Today, The Statue of Liberty-Ellis Island Foundation requires powerful computer technology to store and present some 25 million individual records and 3.5 million related images.

Like a growing number of enterprises, the foundation has discovered that clustering?the practice of connecting two or more computers together in such a way that they behave as a single computer?is the answer to its massive data management requirements. Clustering technology, which is used for functions such as load balancing, fault tolerance, high availability and high-performance computing, has been available for several years. Now, the interrelated lures of enhanced performance and lower cost?made possible by new software and hardware technologies?are encouraging a growing number of enterprises of all sizes to look into clustering technology. The number of new clustered servers installed worldwide is projected to increase from nearly 772,000 in 2002 to around 1.8 million in 2007, according to Gartner.

Reasons to Be Popular

Clustering offers a variety of advantages over other system architectures for a variety of tasks. And no single issue is driving clustering’s rising popularity, says Nathaniel Palmer, vice president and chief analyst at Delphi Group, a Boston-based IT consulting and research company. “There are several different trends, ranging from how computers are built to the emergence of new software standards,” he says.

Other analysts agree. “Web services development, the shift toward scaling out at the edge server level and in business logic servers, and the emergence of new blade server systems will drive up the use of clustering,” says Jamie Gruener, a senior enterprise computing and networking analyst at the Yankee Group, a Boston-based technology research firm. Add the fact that major hardware vendors?including Dell, Hewlett-Packard and IBM?are aggressively promoting clustering, and it’s no surprise that the technology’s profile is rising rapidly.

It’s also quite easy to make a solid business case for a move to clustering. In an IT world where data consistency, availability and reliability are paramount, the redundancy of clustering can’t be matched by standalone systems?or even multiple machines.

The Cluster Migration

For Sam Daniel, The Statue of Liberty-Ellis Island Foundation’s IT director, clustering proved to be the most affordable answer for powering an extensive 100GB database and a website that can receive as many as 8 million requests per second (which can happen, for example, when the site gets television publicity). “It would have been more expensive to get one huge server to handle all these types of queries coming to our site,” says Daniel, who adds that clustering also allows for cheaper and easier scalability. “By putting in a cluster system, we could add in several less costly servers as the need arises.” Daniel estimates that the hardware, software and maintenance makes the cluster at least 30 percent less expensive than acquiring and operating a large, standalone server.

The foundation currently operates 25 Windows 2000-based HP ProLiant DL360 Web servers. Database operations run on a pair of DL580 servers, with four processors each, running Red Hat Linux 7.2 and Oracle 9i Real Application Clusters (9i RAC). The installation, completed last year, replaced a pair of nonclustered Compaq AlphaServer systems, which Daniel says suffered from being both underpowered and unreliable. “All the traffic and all the queries would go to one server,” he recalls. “In case that server went down, we had another server to boot up, but that would take about 15 minutes.”

Daniel decided that a cluster-based configuration would provide both the capacity and availability the foundation required. “We used to go down several times during the week,” he says. “Now we bring down our database servers only for maintenance?about every two months.” The more reliable technology has also given Daniel better control over personnel allocation. “We’re able to free up a lot of time for people to move over to other long-term projects,” he says.

A Software Thing

A new generation of cluster-savvy software is helping to push cluster systems further into the IT mainstream, say analysts. “New platforms such as Oracle 9i RAC, IBM DB2 Enterprise Edition and others are starting to make customers consider a more distributed view of achieving availability and performance,” says Yankee Group’s Gruener. Delphi Group’s Palmer concurs. “With new approaches?being able to partition application logic and to run applications on multiple servers?the big cost advantages and the performance advantages of clusters have increased significantly,” he says. Palmer points to less expensive and cluster-friendly operating systems?particularly Linux?and the increased use of Java and other development environments that are capable of supporting application logic partitioning as key elements in clustering’s rising popularity.

At automaker DaimlerChrysler, better cluster-oriented software and hardware have given the company the ability to run specialized applications, such as vehicle crash simulators, in a clustered environment. “Companies like IBM are beginning to offer these clusters that are specifically tailored for our environment, like high-performance computing,” says John Picklo, the high-performance computing manager for DaimlerChrysler in Auburn Hills, Mich.

A cluster installed last year by DaimlerChrysler, for example, intends to give the automaker’s engineers a better understanding of various vehicle design factors. “We try to avoid the word crash, but basically we simulate what happens to a vehicle in a crash or an impact so that we can determine if it’s safe enough before we actually begin to build them,” says Picklo. When Livermore, Calif.-based Livermore Software Technology, the developer of the simulation application used by DaimlerChrysler, ported its software to Linux, Picklo decided it was time to begin a move toward clustering.

The impact simulation software now runs on a 172-node IBM IntelliStation cluster. But while powerful, the cluster still only supplements DaimlerChrysler’s existing high-end computer inventory?at least for now. (The cluster replaced roughly 20 percent of DaimlerChrysler’s existing impact simulation capacity.) “We still do this work on traditional Unix supercomputers?things like HP Superdomes,” says Picklo, although he notes that clusters will likely play a dominant role in the company’s future high-performance systems planning.

The decision to add a cluster system was based purely on price-performance criteria, including the cost of the hardware and software, maintenance expenses and estimated downtime losses, says Picklo. “We estimate that this cluster is 20 percent faster than something like a large Unix server,” he says. “It’s also 40 percent less expensive.”

Fault tolerance was also a major performance consideration for DaimlerChrysler when it decided to adopt clustering. “A cluster is more resilient,” says Picklo. “When you have a big, single machine that’s running many jobs and you have a hardware failure, you lose the whole machine.” With its standalone systems, the company’s engineers often run as many as five simulations at one time. “If we had a problem, we would lose all five, and you might lose a couple of days’ worth of work,” says Picklo. “With clustering technology, if you lose a piece [of the system] you only lose one job.” DaimlerChrysler’s cluster system can support up to 28 concurrent jobs.

Speedier CPUs and the ability to tap the power of multiple processors located inside a single server are giving clusters power equal to?or in some cases in excess of?standalone supercomputers. Each of DaimlerChrysler’s workstations, for example, includes a pair of Intel Xeon CPUs, giving the system a total of 344 processors.

But cluster technology isn’t relegated to a single box or boxes sitting side by side. Down the road, grid computing could play an important role in clustering. Grid computing can harness the unused processing cycles of LAN- or Internet-connected computers to handle tasks that are too intensive for a standalone machine. The approach promises a fast, cheap and easy way to gain clustering power. Instead of installing new servers, a CIO could simply tap into a network of additional processors supplied by service providers. “It becomes a utility that’s used just like electricity or phone service,” says Delphi Group’s Palmer. Grid computing could emerge as clustering’s next phase, although security, bandwidth usage and availability problems currently tarnish its overall appeal.

No More Bags of Wires

With the cluster market continuing to expand, and competition remaining cutthroat, vendors are striving to make it easier for customers to get a clustered system up and running. The days when a CIO would buy some servers and a clustering kit?a “bag of wires,” as Picklo describes it?are quickly receding into history. In today’s tight technology market, vendors are always anxious to help their customers migrate to clustering, often at a cost that’s more than equalized by the added benefits of a pain-free, speedy implementation. “It’s well worth the extra investment,” says Picklo.

IBM delivered its IntelliStation cluster to DaimlerChrysler as a turnkey system. “They did all the work on their own, put it all together and even tested it before they shipped it to us,” says Picklo. As a result, the system went into full operation less than two months after delivery, far ahead of Picklo’s expectation.

To keep things running smoothly when installing a cluster, it’s imperative for an IT department to work closely with both hardware and software vendors. When Green Mountain Power was ready to implement its new clustering system?just prior to HP’s acquisition of Compaq?it turned to its vendors for installation support. “Compaq came in first and set up the machines for us, and then Oracle came in a few days later and got the database and 9i RAC set up for us,” says Todd Julius, chief technology manager for the Colchester, Vt.-based power utility. “We were up and running without any problems within 16 days, which was nice and fast for us.”

Clusters may now be less troublesome to install and can provide a greater level of reliability than standalone systems, but they are not a panacea?and CIOs shouldn’t cloak themselves in a false sense of security. “You must make sure that your IT team does its job by taking a staged approach,” warns Gruener. “Do all of the necessary work to make sure clustering will be 100 percent available.” Thorough testing, under a variety of real-world conditions, will help ensure that a cluster will perform as designed.

Given continuing budget constraints, CIOs must squeeze the maximum value out of a limited budget. Clustering can go a long way toward maximizing value. “We’re getting more performance for a lot less cost,” says Daniel. “When you’re hungry and you need to do something good, you actually find out what the best systems are.”