The Integration Imperative

Integration is no longer a choice - it's an obligation. And that means far more than merely connecting systems. It means marrying IT strategy to business goals. Not an easy task, but once achieved, the payoff is huge.

Despite years of advancement in application connectors and data interchange technology, integration between databases remains a major problem within both government and commercial sectors. The average organisation spends nearly 40 per cent of its IT budget on integration, according to some estimates, and difficulties bringing this down over time confirm the extent of the difficulties CIOs face in wrestling with the complex machinations of inter-database communication. The challenge is particularly tough in the government sector, where decisions about just what is allowed to connect to what are dictated not only by natural commercial imperatives for efficiency, but by constant external pressure from ministers, legislators, budget makers and others in powerful places.

What may seem like a simple collaboration between departments, announced with the usual pomp and circumstance by seemingly well-meaning ministers, can easily become an extra nightmare that government CIOs simply do not need. And in places where integration might normally make sense, political mores and legal restrictions on intermingling data can make it difficult for commonsense projects to proceed.

Legally enshrined methods for maintaining the independence of the judiciary, for example, have long-hindered the technical integration of court and law enforcement interests. In South Australia, even though the police, Attorney-General and Corrections are being linked in a developing integrated justice program, special approval was needed to allow the South Australian Court Administration Authority (CAA) to progress with development of a service that allows lawyers to go online to lodge civil claims directly with the CAA.

According to project director Steven Taylor, the organisation's ageing ObjectStar-based case management system, which handles more than 70,000 cases a year, would have required "unnatural acts and cost us an absolute fortune" to be upgraded in order to allow real-time transaction processing. But by putting validation forms online and sourcing information from claimants in XML format, the CAA developed a "workaround" whereby new claims are received throughout the day, then entered into the system in a nightly batch process.

This integration alone has shaved turnaround time for lodging such claims - which comprise 85 per cent of all claims heard every year - from three or four days to less than 24 hours. Better yet, the CAA has extended the life of the current case management system by one or two years, at which point Taylor believes a new database (Oracle or DB2 are likely candidates), fed by standards such as the evolving LegalXML, will make data exchange far easier to manage.

Approval Woes

Despite the clear benefits of integration within and between departments, getting approval for such projects is not easy. "There are a lot of restrictions," Taylor says. "That's why integrated justice systems have never got off the ground. You've always got your technological issues with old databases, and doing real-time data integration from the Web has its own issues. But usually there's very sound and tried technology [to fix these]; it's not the technology, it's the bureaucracy and political climate. There's a lot of conservatism here."

Local councils have the benefit of being relatively free of the political performance pressure put on state and Commonwealth departments by optimistic ministers. And while the imperative to adopt e-government has forced many councils to look at ways of exposing their core systems to customers, integration with different levels of government has taken a back seat to simply consolidating and clarifying internal IT structures.

That process has forced IT decision makers first to address the often harrowingly complex heterogeneous infrastructure that has grown organically over the years. New South Wales' Sutherland Shire Council (SSC), the second-largest in the state, has been busy phasing out no fewer than seven different legacy databases to standardise on Microsoft SQL Server and Intel-based servers.

SSC's previous environment was, by all accounts, an integration nightmare. A decade ago it chose to base its core finance and administration system on a Fujitsu mainframe and IDB/2 database because, as IT manager Sid Curry explains, "other platforms we saw were developed for smaller councils, and we weren't feeling confident that other platforms could do what we wanted to".

Since then, SSC's IT panoply has grown to include applications based on UniVerse, Oracle and a number of proprietary Unix databases attached to systems handling land information, library, mapping and other functions. Those core systems that already offer native support for SQL Server are being configured to do so; others are being replaced with different applications or updated versions that are SQL Server-ready.

The new environment went live in time for the new financial year on July 1. While conceding that the change has introduced a major culture change and necessitated considerable retraining, Curry is optimistic that it will both save money and eliminate the complexities of the council's previous silo-based database architecture.

"We can't see any other way out of it; it's just too much of an expense trying to keep the level of skills and people required" to continue managing so many platforms, he explains. "We believe we need to get our systems in a satisfactory state. In the 10 years since we took the direction we had, there were more vendors out there. Now there's less variety, and councils will be coming together in terms of what they use."

Integration by consolidation is a luxury unique to local governments, however, since it is far easier to muster the top-down political will to force cooperation between business units. When inter-departmental issues come into play, the situation becomes far more complicated, and any IT strategist knows that complexity can mean delays and difficulties in driving the technical case for integration.

Planning, Planning and More Planning

Even when technical integration can be achieved, CIOs must plan and execute a massive data cleansing operation that can become a major sticking point for seamless interchange of data. That is because, database standards aside, each department will have its own way of representing data and may represent data about two different people in a different format. Even different naming or spelling conventions can make it difficult to cross-match data between departments, compromising the efficacy of whatever database integration is put in place.

Noting the increasing imperative for organisations in every sector to provide bridges between disparate applications, analyst firms envision strong and continuing demand for integration services of all types.

IDC, for one, anticipates the worldwide market for network consulting and integration services will grow from $US18.2 billion in 2002 to $US39.8 billion by 2005, with the Asia-Pacific region leading the world with annual growth of 31 per cent. At the same time, IDC has predicted, demand for enterprise integration software will lead the rest of the software industry with a compound annual growth rate of 43.9 per cent through to 2005.

While these figures include private enterprise as well as government bodies, the imperative to integrate is increasing equally across every sector. IDC identified a number of factors driving this push: the need to increase ROI on corporate information systems; the continuing requirement to link new systems with legacy environments; employee expectation of a consistent Web-based interface to departmental information systems; and structural changes brought about by politically driven forces such as departmental restructuring or IT outsourcing.

How, then, can CIOs reconcile the intrinsic complexity of integration with the service demands and ideological limitations of their political masters? One way out will be the widespread adoption of Web services - that evolving class of software standards that allows departments to hide the complexity of their internal systems while providing for the seamless exchange of data as necessary.

Web Services to the Rescue?

Web services standards are still very much works in progress - particularly those dealing with security, the first of which was only launched earlier this year by Web services architects IBM and Microsoft. Yet their core enablers - XML for data representation, SOAP (Simple Object Access Protocol) for moving that data between Web services, and UDDI (Universal Description, Discovery and Integration) and WSDL (Web Services Description Language) for directing services how to find and communicate with each other - are well enough advanced that workable Web services can now be built and integrated very effectively.

That means individual departments can continue to set their own technology agendas while collaborating with others as necessary or desirable. And that ability may ultimately be more appealing to departmental decision makers than the perceived sense that they are compromising their departmental strategies to meet the demands of others.

Analysts caution that early authentication and security standards mean departments should only trial Web services internally for now, but one organisation taking a more proactive approach is the Australian Taxation Office. Recognising the need to standardise representation and exchange of data across various departments, it was recently revealed that the ATO has spent nearly two years working with Microsoft and consulting firm Accenture to build the massive Australian Business Register (ABR).

Designed from the ground up around Microsoft's evolving .Net architecture, Microsoft spokespeople indicate the ABR is going to be positioned as a centralised repository of information about Australian businesses that will underscore new services from the ATO and other departments.

By keeping a single version of the truth and offering appropriate parties the opportunity to access that truth using a Web services interface, the ATO could single-handedly provide both the technical infrastructure and political assurances to propel government past its current integration stalemate.

ATO representatives declined to offer more detail on the ABR until it is formally launched, but as described so far, its potential for facilitating integration is clear. Other departments will eventually take the ATO's lead, abandoning expensive proprietary integration projects to rework their systems around XML. Economic downturn or no downturn, IDC anticipates that increasing takeup of XML-enabled databases will drive the feverishly-paced XML business-to-business market to rocket past $US3.7 billion by 2005.

Much of this growth will come at the expense of traditional enterprise application integration (EAI) middleware, which has long been used to mediate between disparate platforms but is becoming less critical as XML provides similar benefits through an open and universal standard.

Many departments are already making the shift towards XML in pilot projects or more ambitious deployments. Web services takeup among government bodies will no doubt mirror or even lead that in the private sector, where research firm Gartner has predicted the technologies will dominate deployment of new applications by 2004.

And, true to form, it appears that large Commonwealth departments will lead this integration push within the overall government sector. "I don't see a lot of [integration] at the state level," says Gartner research director Greta James, "and I rarely get an enquiry from local government outside Queensland."

A Time for Cooperation

In Queensland, which is now publicly billing itself as "the smart state", an accepted imperative to improve services is forcing some open discussions about the political and technical challenges of interdepartmental integration. Queensland Transport, for one, is weighing up the complexity of providing a smartcard-based driver's licence that also serves as a de facto identity card to support applications relevant to other government departments.

Data protection and consistency issues mean attempts to foist such services onto other departments are almost certain to fail without strong inter-departmental cooperation. But with the right approach, Gartner suggests, gradual introduction and expansion of the program would build public confidence in the system and therefore increase political will to support its expanded use.

The Queensland government's philosophical support would bolster back-end integration efforts, quickly opening up new service opportunities as other departments came on board. This process would be particularly well executed if Queensland Transport took the ATO's lead and eventually exposed its licence management system as a carefully controlled Web service securely accessible by other departments online.

1 2 Page 1
Page 1 of 2
6 digital transformation success stories