What tools (if any) are critical for testing SOA?

RR: Randy speaks about tools that allow for test harnesses and mentions SoapUI and ITKO's Lisa product as two testing tools that are designed to give testers structural access to services. He says that some existing tools can be extended but not to the level required to effectively test SOA.

FC: Frank states that SOA testing tools should let you do repurposing. For example, the tools should allow the tester to take a developer's test harness and extend it through functional testing. Frank's company has built Testmaker Pro, an open source tool that provides testers a framework for repurposing without having to learn new languages like Groovy, Jython, etc.

JG: Jim stresses the importance of performance testing tools and recommends a tool like ITKO's Lisa to follow proxy services and dynamically detect the path of called services. In some SOA implementations, architects tend to leverage many proxies in front of their services.

What additional risks does SOA introduce to testing and deployments?

RR: With no hesitation, Randy warns of performance penalties and costs. Services are determined at runtime from various locations. This creates unknown integration points that testers do not know in advance. Another key risk is that the organization may not have a robust enough understanding of SOA and what it takes to implement it successfully. This could lead to underfunding, not addressing talent issues, not paying attention to organizational challenges, and many other common mistakes. Randy highlights that a company's first experience with SOA is a learning experience. I highly recommend that companies seek outside help to bring much needed experience to the projects.

FC: Frank states that "SOA is a methodology not a standard". SOA is still evolving and the lack of standards creates all kinds of issues. One example that Frank gave me was the fact that there are at least 33 different XML parsers and each has its own performance profile. An SOA architect may define one XML parser as the standard even though this parser may not scale for certain use cases. Frank reiterates that XML parsers have different performance characteristics and there is not a standards body that is providing the proper guidance that architects need to make better informed decisions.

JG: Jim points to rapid change as the big risk in an SOA. I have said in the past that SOA brings simplicity, flexibility, and agility to the business but creates complexity, multiple failure points, and manageability issues for IT. Underneath the promise of SOA is a dangerous world that needs to be managed accordingly. Jim mentions that there are also risks caused by the integration of layer specific components when software is deployed into QA environments. Developers tend to be layer specific and what looked good in unit test may not work well in QA. Jim says that it is likely that the standard smoke test for SOA is a full regression test.