Early last year, Michael Dreiling faced a stomach-churning problem. The vice president of technology for Quadrem U.S., a Dallas-based global electronic marketplace serving the mining, minerals and metals industries, needed to find a way to seamlessly integrate data from more than 1,000 companies.

Traditional middleware products could take care of the nuts-and-bolts job of converting files spewed out in EDI, legacy data formats and various flavors of XML. What they couldn’t do was discern the meanings contained within the files. To cure his data integration indigestion, Dreiling looked into a new type of middleware: semantics-based integration tools.

Like Dreiling, an increasing number of CIOs and their staffs are being asked to integrate data from multiple, dissimilar sources into an electronic marketplace hub, EAI platform or database. The need may arise because of a merger, an acquisition, a CRM effort, the building of a data warehouse or a company’s participation in an e-marketplace. And getting all the various fields and rules to mesh seamlessly into a single location is no easy job.

To accurately map source fields to targets, it’s important for the middleware to completely understand the full semantic meaning of each data source element and how it behaves over the entire scope of source data. "Essentially, you’re looking for semantical equivalents," says Jess Thompson, a research director for the application integration and middleware strategies group with Gartner Research in Stamford, Conn.

Unfortunately, while conventional middleware and hard-wired solutions are generally good at connecting noncompatible systems?converting protocols and formats?they often fall flat when it comes to interpreting the meaning of specific information and then applying it to a new environment. While a standard middleware product may know, for example, that Nov. 14, 2002, 11/14/2002 and 11-14-02 are all the same shipping dates, it may not be able to understand whether the "shipping date" means "available for shipping," "on the dock" or "released to carrier." That’s where semantics-based integration tools step in. These products, while not perfect, aim to make sense out of ambiguous and potentially conflicting information, reducing the labor-intensive need to manually link and synchronize data. "Semantics-based middleware could eliminate the need for application-specific APIs, traditionally used in integration today," says Eric Austvold, a research director at AMR Research in Boston.

Putting It Together

Semantics-based integration technology is being pioneered by a variety of software companies, including Contivo, Modulant Solutions, Network Inference and Unicorn Solutions. Semantics-based integration tools are considered by many analysts and industry players to be a kind of middleware that thinks. That isn’t surprising, since many of the tools utilize a combination of natural language analysis, pattern recognition, artificial intelligence and other leading-edge cognitive technologies.