Integration's New Strategy

Old concepts and new technologies have recently converged to generate a new strategy to improve IT responsiveness while reducing integration costs. It's the integration layer, and it may put an end to all those complaints about IT's slowness and inflexibility.

Reader ROI

  • How messaging and services work together
  • How to get developers to write service components
  • How a SOA can create IT-business alignment
  • For years, whenever Wisconsin state CIO Matt Miszewski tried to discuss systems integration with agency heads, he sensed a fog settling over the room. "You could see it in their eyes," he says. "They tuned out."

    Worse, when the fog cleared, it was replaced with anger. Difficulty with integration, Miszewski could tell they were thinking, was just another IT excuse for slowness, inflexibility and inability to give them what they wanted.

    And Miszewski couldn't blame them. Technical limitations and time pressures often made integration a haphazard affair. Rushing to meet deadlines, developers cobbled together direct links (point-to-point integration) to share data and business logic among applications.

    While that kind of integration is quick and relatively simple, over time it has crippled the health and flexibility of most IT architectures by creating a cobweb of hundreds, even thousands, of brittle linkages that have to be torn apart and reassembled every time one of the applications changes.

    All those links, built up over decades, have created a crisis that goes far beyond IT. The rise of the Internet has made businesses completely dependent on IT to add new capabilities. The foundations of those new capabilities often lie buried inside old systems that were never designed to communicate with one another, let alone over a global network. Adapting those systems to communicate - that is, systems integration - can take so long that entire generations of business opportunities (new products, alliances, mergers and acquisitions) can grow old while IT fiddles with the wiring. Unable to explain the complexity of their problem, CIOs inevitably wind up on the defensive, with only the most patient, tech-savvy CEOs able to commiserate.

    But markets don't commiserate with anyone. Markets see only one speed: fast. Aesop didn't spend any time on Wall Street. On the Street, the tortoise almost always loses.

    The urgency and intractability of this problem is underlined in CIO's "State of the CIO" survey: Integration has been the number-one technology concern of CIOs since we began doing the survey in 2002. (CEOs also put it at the top of their list when we began surveying them in 2004.) CEOs' three primary complaints about IT, according to survey after survey - that it is too expensive, too slow and too inflexible - all lead back to integration.

    In short, the business is tired of waiting for IT to catch up with its demands.

    And so are CIOs.

    How the Integration Layer Can Untangle the Enterprise

    All this is hardly news. CIOs have been painfully aware of integration's costs for years. What they've lacked is a unified strategy for dealing with those costs. But recently some old concepts and some new technologies have converged to generate a new, winning strategy that promises to blow away the cobwebs and radically improve IT's responsiveness (while also reducing integration costs). It is the integration layer, a virtual stratum in the architecture that is composed of two major pieces: messaging and services.

    The foundational piece - known as the messaging infrastructure - is like a good executive assistant, translating, routing and monitoring information from different systems without these systems needing to connect directly. Adding, changing or removing a system becomes a matter of modifying a single link, rather than ripping apart connections to all the different systems it may need to communicate with.

    But while the messaging infrastructure makes connecting systems easier, it doesn't free business processes from their mainframe prisons, or eliminate redundancies in applications, or provide any impetus to create a useful architecture. Indeed, a good messaging infrastructure can perpetuate the chaos by making it easier to deal with.

    Of course, not every company has redundant systems and processes. For them, messaging may be enough. But for most companies - especially big ones with dozens of different ERP systems all doing more or less the same things - messaging alone won't create an effective integration layer.

    Messaging is an IT solution, not a business solution. It has long lacked a higher purpose, a strategy. Service objects (or just plain "services") is that strategy, and it is the second core piece of the integration layer. The idea behind services is simple: Technology should be expressed as a unit of business work - like "get credit", or "find customer record" - rather than as an arcane application such as ERP or CRM. This is an old concept, based on object-oriented programming from the 80s. Services extract pieces of data and business logic from systems and databases around the company and bundle them together into chunks that are expressed in business terms.

    Page Break

    Object Lessons in Services

    At telecom company Verizon, the service called "get CSR" (get customer service record) is a complex jumble of software actions and data extractions that uses Verizon's messaging infrastructure to access more than 25 systems in as many as four data centres across the US. Before building the "get CSR" service, Verizon developers wanting to get at that critical lump of data would have to build links to all 25 systems - adding their own links on top of the web of links already hanging off the popular systems.

    But with the "get CSR" service sitting in a central repository on Verizon's intranet, those developers can now build a single link to the carefully crafted interface that wraps around the service using the Web service standard simple object access protocol (SOAP). Those 25 systems immediately line up and march, sending customer information to the new application and saving developers months, even years, of development time each time the service is used.

    Though the productivity savings for IT are huge, the strategic implications for Verizon are just as important. Create enough services and you can start to build a map of the business expressed in technology - a service-oriented architecture (SOA). A SOA is the blueprint that guides the development of the integration layer and its two major components: messaging infrastructure and services. A SOA is the big picture of all the business processes and flows of a company. It means businesspeople can visualize, for the first time, how their businesses are constructed in terms of their technology.

    "When I said we have 18 slightly different versions of 'credit check' buried inside different applications in different agencies," says Miszewski, "the agency heads could understand why having all those different versions was a problem, and they could support creating a single version that everyone could use."

    Toby Redshaw, Motorola's corporate vice president for corporate IT strategy, architecture and e-business, had a similar experience when he told his 80-year-old mother about IT's central catalogue of business services. "When I was done explaining it, she said: 'Manufacturing broke its work up into pieces 200 years ago. What's taken you so long?'"

    When your business has an integration layer containing services, "a change in business policy can be made quickly rather than opening up an application project", explains Randy Heffner, vice president for Forrester Research. Integration becomes strategic, rather than an afterthought. "With business services, you are saying you are designing the business, and the design is too important to leave to ad hoc implementation teams working on a deadline," Heffner adds.

    Not only does integration become strategic, it becomes a lot cheaper (at least 30 percent cheaper, according to estimates by research company Gartner) and faster too, taking months off development cycles for new projects. Anecdotal evidence pegs the financial and productivity gains much higher. At Motorola, Redshaw says that in some cases, integration costs have been reduced by a factor of 10. Shadman Zafar, Verizon's senior vice president for architecture and e-services, says that his catalogue of services let him skip forming a project team for the development of a phone-line ordering process, because the services necessary to compose the process were already in place. "With point-to-point integration we would have had a central project team to write the overall integration, and local teams for each of the systems we needed to integrate with. With [the phone-line process], we had a single team that was focused almost entirely on end-to-end testing." That saves time and resources and improves the quality of new applications, because testing is no longer the last hurdle of an exhausting application development process; instead, it's the focus.

    If you can do all this, CIOs will be waiting for the business to catch up - not the other way around.

    Page Break

    Why Layering Requires Centralized Management

    Nothing in IT is ever simple, including the creation of an integration layer. The goal of the strategy is nearly always to minimize redundancy by having as few messaging infrastructures as possible and creating a single repository of services. Therefore, by definition, the integration layer demands centralized IT management.

    "Absolutely," says Rick Sweeney, chief architect for Blue Cross and Blue Shield of Massachusetts. "SOA isn't a technology; it's a framework, a blueprint, and if you don't have control over it and aren't guaranteeing that it's being applied across the organization, you'll never have a true SOA. There are probably a million ways to architect an integration layer, but you can't have a thousand different ways inside your company."

    For example, Verizon, though geographically dispersed, has a centralized business and IT model, so there was no conflict over the composition of the integration layer or the SOA strategy behind it. Indeed, the original impetus for Verizon's strategy was to eliminate the redundancy of systems that resulted from its mergers and acquisitions of other telecom companies such as GTE and Nynex.

    That wasn't easy. When Zafar's team began picking over the remains of those systems in 2001, it combed through every button and drop-down menu in every application it could find - roughly 2900 in all - looking for shards of software functionality that might be incorporated into a service component. From the thousands of function points they found, the team isolated between 200 and 500 functions that are needed for the more than 90,000 business transactions (such as setting up a new landline account) that Verizon performs. Then the team looked across the infrastructures and found five to 25 redundant versions of each function. Zafar says that gave him all the incentive he needed, and all the proof that Verizon CIO Shaygan Kheradpir and other company executives demanded, to approve the integration layer strategy.

    The Trouble with Developers

    Surprisingly, a services strategy, despite its obvious advantages, may not be all that popular among developers. "Developers want everything to be easy," says Clint Petty, a director in professional services for software and services vendor CommerceQuest.

    Services are not easy - at first. Extra work is required to create an interface, the part that describes what the service does in business and technical terms and how other systems access it. Good interfaces are like good friendships: easy on the surface with no hint of the relationship's history of ups and downs. "A good service knows who it is, can describe itself to others and show who wants to connect to it," says Jeff Gleason, director of IT strategies for the financial markets group of Transamerica Life Insurance and Annuity. "The essence of service-style integration is that the interface is intelligent and communicative."

    Developers writing links to a good interface need only write the basic communication code that accesses it - a programming version of a handshake and a hello - and the service does the rest. The developer doesn't need to know what the service is composed of, what computer language it is in or even how it works - only what it does in business terms.

    But developers need incentives to build those interfaces. "Everybody is focused on getting a new application up and running now," Verizon's Zafar says. "And their primary focus is not on how they can continue to add value to other systems. If they feel pushed to the wall, they won't opt for the best possible design - just the best design to get the job done."

    1 2 Page 1
    Page 1 of 2
    Security vs. innovation: IT's trickiest balancing act