Proprietary to Open: Middleware Evolves

London institutional brokerage Prebon Yamane Group could be considered a throwback in at least one regard. In this age of e-business, its brokers take orders exclusively by telephone from institutions all over the world.

But in another sense, Prebon is on the cutting edge. Its technology group uses sophisticated middleware from Sonic Software to ensure that its customers and brokers get almost immediate electronic trade confirmation; previously, they would have waited hours or even days for fax notification that the deal had gone through.

Since Prebon is a voice brokerage, the capability to send online, near-real-time confirmation of clients’ trade details boosts customer service. “This is a revenue exercise. We’re aiming to retain our customers by giving them greater efficiency in dealing with us,” says Geoffrey Sanderson, CEO of Prebon Technology Group, the IT arm of the $434 million brokerage.

The software Prebon uses?SonicMQ?is an enterprise message server. And it’s merely one example of the variety of modern products that expand on middleware’s traditional role as “glue” to tie together disparate applications. With a generous push from XML and Web services, middleware has moved away from proprietary schemes toward flexible, open standards. And new vendors such as Ascential Software and Sonic Software have joined traditional players such as Tibco and Information Builders in offering cost-effective applications that target specific pieces of the integration puzzle.

Rather than going away, traditional middleware has branched out into subtypes including service-oriented architecture (SOA), enterprise service bus (ESB), business process management (BPM) and Web services. The diversification results from an increasing need for CIOs to find ways to integrate evermore disparate systems, from ancient mainframe applications to the latest Java servers. Integration, and therefore middleware, is today’s cost of doing business.

Early Days

Middleware first appeared in the mid-1980s (though the term itself didn’t take firm hold until the 1990s and has morphed in meaning over the years). Even at the dawn of the client/server revolution, large companies were grappling with how to extract data from mainframe systems and serve it to other platforms. Middleware provided that connection. Unfortunately, these products used proprietary frameworks for communication, and they often required hand-coding for each system. The result was costly software and long-term projects that focused on making the IT function more efficient rather than delivering and retaining customers or igniting sales.

“Traditional middleware was too expensive, too hard, took too long and required too much training. It was very IT-centric,” says Eric Austvold, research director for enterprise applications and technology strategies at AMR Research.

Because each connection among systems had to be coded manually, first-generation middleware was also very inflexible. If a mortgage company wanted to change the maximum age for a borrower from 60 to 70, for example, there was no way to make the change in one place and have it cascade through all affected data. As a result, maintenance of systems connected by middleware was expensive.

And the companies’ investments of money and time made it easy for their vendors to sell them multiple products and services. “They used middleware to entice you in,” recalls Sanderson, who has used various middleware products for many years. Once you committed to one vendor’s platform, it was tough to change and there was every reason to keep buying more from that one company.

But during the 1990s, the drumbeat of standards began to grow louder. Customers started to push software vendors to offer application programming interfaces (APIs) as backdoors into their closed environments, thus making integration easier.

Then the Internet took hold, with its plethora of data transmission and access standards, including HTTP, FTP, simple object access protocol (SOAP) and XML. The last of these set the stage for Web services, a relatively simple, standardized way of integrating applications. The goal, ultimately, is to go beyond simply linking applications loosely together, but instead to integrate pieces as needed, quickly and easily, thereby creating “composite applications.”

For their part, middleware vendors voice support for standards in the same breath they defend their proprietary value-add. “Standards reduce our development cost,” says Ram Menon, senior vice president of worldwide marketing at Tibco. “XML is a normalized format. It will get the data moving, but the applications still can’t talk to each other.”

It’s hardly surprising that vendors are trying carve out their own special place in this volatile, near $1 billion market. But they make some moves grudgingly. “If you’re BEA, you still don’t want your stuff to be used in an IBM project. If you want to connect BEA to IBM, you can do that thanks to standards, but you can’t expect BEA to be happy about it,” says Gerry Cohen, president and CEO of Information Builders.

“The large vendors are seeing the benefits of being open. They’re not 100 percent there, but they are much more open than they were,” adds Barney Sene, vice president and CTO for WellPoint, a $20 billion health insurance company. The nation’s second-largest health insurer, WellPoint uses IBM’s MQSeries Integrator middleware to form a bridge between the legacy systems that process member claims and the Web-based, IVR and EDI customer-facing systems so that its employees can better serve customer needs.

But despite all the talk of standards, Prebon’s Sanderson, a veteran user of all the big-name middleware packages, remains skeptical that proprietary middleware schemes are a thing of the past. “They can talk openness and standards, but I have to sign the checks. It’s a fairly obvious strategy [to push a single architecture], I don’t blame them for it,” he says. On the other hand, he doesn’t want anything to do with it. Sanderson and Patrick McGrath, head of technical strategy, are pursuing a strictly best-of-breed integration approach; a strategy predicated on open standards.

Sanderson has the luxury of spending the money necessary for best of breed since he is in the financial services industry, which competes on being able to serve its customers better information faster than its competitors. “Technology is a key differentiator in what they do,” says Austvold. “They are taking on quite a burden in terms of managing different technologies and vendor relationships, but it may well be worth it.”

What makes this complexity worthwhile is the ability to put information in the hands of those who did not have it before. Steve Elkins, manager of business intelligence for Capella University in Minneapolis, used Ascential’s MetaStage tool to let his users perform their own queries on a new data warehouse. Previously, if the school’s dean wanted to see how many students failed the advanced management course, for instance, he would have had to ask an IT person to query the legacy reporting database and then wait days or weeks for the result. Now, Ascential’s DataStage product pulls the data from Capella’s business systems and delivers it to the data warehouse. Ascential MetaStage provides the “data dictionary” that makes the data in the warehouse understandable for the school’s users. Brio’s business intelligence tool then acts as the user interface for both the data from the data warehouse and the explanatory meta-data in the MetaStage data dictionary.

Since MetaStage maps types of data from one application (such as the warehouse) to another (such as the business intelligence tool), MetaStage effectively integrates the two, making data accessible where it was not before. Previously, only three IT people at Capella were skilled enough to know how to query the legacy reporting database. With MetaStage doing the heavy lifting of translating between applications and serving the data in an easily comprehensible format, now more than 100 university personnel can do their own queries, greatly improving productivity.

Striving for Simplicity

Increased end-use efficiency isn’t the only goal of modern middleware; one of the biggest trends is to simplify maintenance tasks as much as possible, thereby reducing costs. Toward that end, WellPoint’s Sene recently began evaluating technology from a startup called Pantero. Pantero’s Shared Data Services (SDS) tool reconciles the inconsistencies in data between WellPoint’s many enterprise systems. Pantero SDS manages the rules that govern data usage and interpretation so that data is interoperable and accessible across the organization. “Pantero runs on top of our systems and makes it easier to build and manage the adapters and connectors that bridge the systems,” says Sene. The key to Pantero is reusability. “You can have hundreds of those adapters and connectors and it is a huge maintenance nightmare. Pantero helps us manage them. I don’t have to hire as many people to do the hard coding for those connectors. It’s a major cost reduction,” says Sene.

Like Ascential’s MetaStage, Pantero also does data mapping between systems so that there is a common representation of data for various entities in the enterprise. “We have a common data model and we map all our legacy systems to the model. It makes things much easier. We’re not doing it manually, so our costs are reduced,” says Sene. Sene has built an SOA in which nuggets of software functionality can be reused among systems.

To Guido Sacchi, speed and agility are the best payoffs of standardizing the data architecture. “Connecting our systems via standards shrinks our time-to-market,” says Sacchi, CIO at CompuCredit, an Atlanta-based credit card and financial services company. “We need fast integration of credit portfolios that we acquire and high efficiency in our collections.” For example, CompuCredit telephone agents no longer need to put customers on hold to consult other parts of the organization, since they can access the information they need directly, reducing call times and increasing customer satisfaction. Sacchi uses an SOA based on middleware from Software AG in conjunction with a variety of standards, including XML, SOAP and UDDI (Universal Description, Discovery and Integration). The new architecture builds interoperability between different systems and databases. It gives all users the “one version of the truth” (where everyone sees the same data) because of an XML meta-data repository. The business advantages include better customer service, more exact tailoring of product offerings and better information for the agents.

So many businesses have benefited from the integration through middleware that they have had a powerful collective effect?even on the world economy as a whole, according to Austvold. And while it’s certainly far from the only reason (outsourcing and consolidation also play a large role), middleware’s efficiency is even contributing to the so-called jobless recovery, he notes. Integration technology has helped industries from manufacturing to telecommunications cut masses of employees. Those jobs are not coming back any time soon?bittersweet testimony to the success of what was born as middleware two decades ago.