by Dan Rosanova

The SOA Knowledge Gap

Dec 30, 20087 mins
DeveloperEnterprise Architecture

The traditional corporate hierarchy--the organizational structure that serves the company's business needs--can get in the way of software developers who need to create a service-oriented architecture which serves the business as a whole. Here's how to address some of the thorny issues it raises.

The common business structures that enable large corporations to function effectively can actually inhibit the development of an effective service-oriented architecture (SOA). Most enterprise employees outside of IT work for a team, in a department, in a division or some similar hierarchical structure. This pattern of organization has effectively served large corporations, governments, and militaries for a long time. Understandably, people in such organizational structures see the world through the context of their position in this hierarchy. But the organizational structure can present challenges for SOA analysis when the IT solution requires input from representatives in all parts of the business.

Over the years, the software world has matured to the point where analysis and design are quite refined processes. Most people in the IT world are by now quite familiar with the main techniques used for gathering business requirements and developing system architectures. The role of Subject Matter Expert (SME, sometimes called a Domain Expert) is now commonplace on software development projects. This has served well for building line of business systems—software silos, if you will—that traditionally served the business unit for which they are built. Tapping directly into the knowledge of a business expert allows the development team to produce a solution that closely resembles what the business unit actually needs. This process is by no means simple, but it is at least common and well understood.

A unique SOA challenge is its need to bring together SMEs from across the enterprise. SOA builds a new collective knowledge base, representing how the business operates at a level above the individual business lines. This poses several distinct problems for the SOA analysis process. Representatives from every line of business need to be involved in analyzing the SOA’s needs and capabilities. If each business unit has its own IT staff, it may need to participate as well.

It’s not just an issue of getting more people to provide input, explaining what their department needs. As the number of people in this analysis process grows, so do the number of viewpoints. Business unit representatives may see the analysis skewed by their proximity to their unit of the business, neglecting the views and needs of other business units. This is really sort of to be expected, as each of these people function in an area they are familiar with and may not realize that other areas do not function in the same way. Usually, SMEs are leaders in their departments and may have an “It’s all about me” attitude; where me is really my department. This is rarely intentional, but represents the fairly common form of system bias in SOA and integration initiatives I wrote about earlier. I like to have meetings with many representatives together to encourage the participants to see the larger picture.

In my line of work I often work in meetings with representatives from Finance, Accounting and Operations (post-Sales), along with IT. These team members all have unique perspectives of a single flow of data throughout the enterprise, but each only sees one piece. The representation of this data is different for each department, but the underlying entities are the same: Orders and associated financial data.

Finance usually is concerned with how revenue is booked and earnings are calculated. Accounting really doesn’t care about those details, but wants to be sure that the General Ledger accurately reflects the intentions of Finance and meets GAAP and auditing requirements. Operations teams tend to be mostly concerned with moving orders through approval processes and between external systems, and often is unaware of how the financial aspects appear to either Accounting or Finance. A common conflict arises when Finance wants to recognize revenue that may be partially earned; Accounting says that will violate their GAAP; and Operations says you can’t split orders without breaking their processes. Discussing the issues with all these representatives concurrently allows them to understand how the other departments interact with different aspects of this same data. When leaders from each line of business are confronted with the realities of the other lines, it often softens their resistance to compromise. Ultimately, they are all on the same team.

By definition, SOA is about bringing people together to build systems that serve across the enterprise. That results in another issue: the effect of personality differences on the group dynamic. This is only made worse by all those meetings with many representatives, since the reason they’re all invited to participate is their differing needs. It is common for someone in the group (who generally is very persuasive or respected) to gain undue influence over the design decisions that come out of such sessions. To defuse this effect, I like to mix large and small group sessions, as well as one-on-one follow-ups (or pre-meetings, but be careful there). That helps sift through the plethora of information and divergent views that emerge throughout the process. This group dynamic effect is even more pronounced in a multinational enterprise.

The final issue is one that is common even in traditional software analysis: the communication gap. A question’s answer can be very different depending upon how the question is asked. This can be the result of both basic communication skills and the context in which the question is asked (to me, context is part of communication, so maybe I’m restating here). Often when discussing the details of an existing process asking an abstract question generates an answer that’s too specific. That is, the answer may be too implementation-specific, and does not help the enterprise architect to understand the scope of the problem as well as the specific variations of this use case.

For instance, the discussion might involve eliciting from the SME, “We need to receive orders in the formats A, B and C from partners X, Y and Z.” It is important to not get too lost in the specifics of the moment. This can lead you off the important trail of an abstract understanding, disconnected from implementation details. The subtle and damaging problem is that it may be difficult to get back to abstracts after being so specific. Yet, you still need the specifics. Oddly enough, the difficulty can be either with the SME or the business analyst; or it can apply to the group as a whole. Sometimes, you can’t help this. Just do your best to take notes; but be sure to be aware that the capability or service, is really receiving orders. There may be customer-specific implementation differences, but that is farther down the chain in the analysis process.

One trick I used a lot was to ask the SME the same question in three different ways, usually not in the same meeting. Often this exchange may follow a pattern like this:

  • “So what are the possible workflows (state changes) than an order can follow?”

  • “Are there any times an order can follow a path other than… (list from above)?”

  • “What do you do if there is an exception and an order needs to make an unspecified state change?”

Using parts of information from the answer, restated in another way, really can cause SMEs to think beyond their immediate or current implementation. They’ll reveal more of the true requirement rather than the current capability. This is even more important when discussing business capabilities needed by a SOA.

To make this all run smoothly, establish a vision early on. Communicate it effectively with the SMEs and other members taking part in the SOA analysis. At first, your vision will just be what a SOA can do for the enterprise and how the process will begin. Over time, the vision evolves from the generic initial statements to a specific vision for the particular enterprise under analysis. This vision acts as a tool that allows you to convey the introductory and abstract concepts of SOA to the SMEs (who increasingly are aware of SOA) in a clear and non-technical manner. You may start communicating this vision, perhaps with senior IT management supporting you, but the enterprise will come to own it quickly and drive its evolution. This is where the pieces come together.

Dan Rosanova is principal consultant at Nova Enterprise Systems, a consultancy that specializes in Enterprise Applications and Microsoft BizTalk Server solutions. He has ten years experience delivering enterprise solutions in financial, insurance, telecommunications, and medical industries on Windows, Solaris and Linux platforms.