Performance Management Tools

When an alarm goes off, George Tumas doesn’t panic. The senior vice president of Wells Fargo Bank’s Internet Services Group faces potential catastrophe calmly, thanks to his investment in a performance management tool.

The latest such tools?also known as component and service-level agreement management tools?are designed to help IT departments keep a close eye on critical Web-enabled systems. As these systems grow larger and reach outside the enterprise to customers, partners and suppliers, the need to keep performance at the highest possible level becomes ever more pressing. That’s not always easy, however. “The new Internet applications have multiple tiers?they’re distributive and complex,” says Dennis Gaughan, a software industry analyst at AMR Research, a technology research company in Boston. “There’s just an increased demand for tools to measure performance and to make sure that the applications are performing to meet the requirements of the business.”

Performance management technology has been around for decades, helping users wring maximum value out of servers, storage, networks and other components. But the newest tools lift oversight from individual components and subsystems to a new holistic, business-focused level. Bill Gassman, a software analyst at Gartner, a Stamford, Conn.-based technology research company, offers an aviation metaphor: “It’s the difference between knowing that your airplanes are flying at 550 mph, burning so much fuel, and knowing whether your flights are making it on time.” In other words, the objective is to gain a bottom-line insight on system performance.

Multiple Paths

As the need for Web-based performance management practices grows, CIOs can pick from an array of products and services. McLean, Va.-based Managed Objects, for example, offers Formula?software that sits on top of the system and retrieves data from various underlying vendor management tools, such as Hewlett-Packard’s OpenView and Tivoli Enterprise Console, as well as a variety of other information reporting sources. The product continuously examines the relationship between the performance data and business concerns in real-time or over a period of time. Formula then customizes the information and delivers it to designated users.

Dirig Software, based in Nashua, N.H., features an agent-based technology that collects information without relying on underlying management software. The company’s Dirig Agent offers real-time and historical monitoring capabilities of server functions, such as CPU, memory, disk-space utilization, log monitoring and process monitoring (the observation of application executables). It also can take automatic corrective actions when systems exceed specified thresholds.

Mercury Interactive, located in Sunnyvale, Calif., also uses an agent-driven technology. The company’s Topaz software can continuously monitor the availability of more than 40 aspects of the Web infrastructure, including the performance of servers, load balancers and the Internet backbone, from inside and outside the firewall. An ASP version of Mercury’s software, ActiveWatch, is also available.

A variety of other startups, including San Jose, Calif.-based Euclid, Keynote Systems and NetReality, as well as old-line software providers such as BMC Software, Computer Associates, Hewlett-Packard and IBM, also offer various types of performance management tools. The products differ on the basis of price, scalability and component source monitoring features. Product prices can range from $100,000 to $500,000, depending on the size and scope of monitoring. Ideally, these multiple paths to performance management should let CIOs select the best tool for their needs. But the wide-open marketplace can also lead to confusion. “You have to understand what are the important processes and services that you’re delivering, and what needs monitoring,” says Gaughan, “before you can apply the technology.”

Tying It Together

Wells Fargo’s Tumas, who oversees a number of departments within the Internet services development department, selected Mercury Interactives’ Topaz technology to replace a variety of monitoring programs developed in-house. Topaz monitors four Web-based business lines that handle an array of consumer and commercial financial services as well as Wellsfargo.com. “The old tools simply weren’t keeping pace with the rapid growth of Wells Fargo’s IT and Internet services. They weren’t scalable or Web-enabled,” says Tumas. “You outgrow your tricycle to go to a bigger bike?that’s what happened to us.”

Tumas says he picked Topaz for its ability to provide holistic monitoring capabilities. “For a long time, we had individual alarms within each application, but nothing to tie everything together,” he says. Topaz boosts staff productivity by pinpointing the root cause of performance problems anywhere within the Web infrastructure, such as on an Internet backbone. By doing so, the software lets staffers detect problems that might have gone unnoticed, such as unacceptably slow response times. “If you’re not monitoring end to end, you may not even have an idea that you have a problem until your customers start flooding your call center,” says Tumas.

Like many enterprises, Wells Fargo relies on a Web environment that’s been built in piecemeal fashion over several years. Such an arrangement can make finding and fixing performance problems a complex and time-consuming task. The company’s consumer Internet service, for example, runs on a mainframe connected to an assortment of firewalls, proxy servers, load balancers and middleware. Tumas says Topaz lets his staff cut to the heart of a problem, such as a last-mile communications bottleneck, before it has the chance to affect large numbers of customers. “Saving time is critical,” he says. “That’s the key benefit the software provides.”

On the Edge

Just as performance management tools can help organizations cope with complex legacy systems, the products and services can also benefit organizations that rely on cutting-edge Web technologies such as Web services. Performance management tools are a requirement for systems that use advanced Web technologies, says Jim Struve, assistant manager of information support services at WEA Trust, a Madison, Wisc.-based nonprofit organization that provides insurance and retirement services to Wisconsin public school employees.

WEA Trust has built its Web-based customer service system on Lotus Notes, Lotus Domino and IBM WebSphere products. A key element within the environment is Java Virtual Machine (JVM) technology, which Sturve says is particularly difficult to monitor and troubleshoot. That’s because JVM software often presents no overt trouble signs until a problem becomes critical. WEA Trust’s Web system runs several JVM applications simultaneously. “A server could be running, and the other JVMs could be running, but one could be dead, and we wouldn’t know it,” he says. WEA Trust uses Dirig software to monitor its Web system and some of its JVM applications. “The Dirig tool is able to alert us when one of those JVMs goes under,” says Struve. “This proactively makes sure that our Web presence is up and available.”

Evaluating ROI

Measuring a performance management tool’s ROI is difficult. Unlike the previous generation performance monitors, which were developed to help enterprises tweak component performance, the new technology is primarily designed to boost one of the Web’s great intangibles?end user satisfaction. But that doesn’t mean performance management tools can’t exert a positive effect on an enterprise’s bottom line. A retailer, for example, may be able to attribute increased sales to improved website performance or decreased downtime. Likewise, banks, newspapers and other service-oriented organizations could credit higher user retention rates to the positive effects of a performance management tool, although other technologies and business practices can also affect user satisfaction and system reliability levels.

The technology, by pinpointing and, in some cases, fixing problems, also offers the potential to trim IT personnel expenses. “It’s possible to eliminate people hired to troubleshoot applications,” says WEA Trust’s Struve. (Though Struve notes that he hasn’t cut any positions.)

Trouble Ahead

Considering the complexity of monitoring an entire Web-based application infrastructure and the various ways the task can be approached, it’s not surprising that many CIOs have trouble implementing performance management tools. “There’s a fair amount of integration work that you need to do in order to tie together all of those different agents collecting event information,” says AMR’s Gaughan.

For Jeff Crawford, senior vice president for Internet operations at Boston-based Fidelity Investments’ e-business unit, the problem was tracking too much system data too closely. The monitor was constantly sending alerts because one threshold or another was always being crossed, he says. It took Crawford’s staff five months to configure Managed Objects’ Formula software so that only major performance failures generated alarms. “The problem was simply access to too much data,” he says. “It’s like taking everything good out of your refrigerator and throwing it into a pot?it’s not going to taste too good.”

Training and its related costs represent yet another potential pitfall facing performance management technology adopters. As Crawford’s experience proves, getting a performance management tool to work well requires both patience and practice. Given the technology’s complex nature, WEA Trust’s Struve feels that it’s important to let staffers “kick the tires” in test mode before sending them off to training classes.

Despite the various implementation headaches, the long-term gains are more than worth the short-term pains, says Wells Fargo’s Tumas. He notes that performance management technology is the price one has to pay for Web-based systems that are called on to perform an increasing array of sophisticated tasks, yet must also maintain continuous access and rock-solid reliability. The bottom line is end user satisfaction. “It’s truly customer-centric to make sure that customers have the best experience they possibly can,” he says. n