Six Questions to Consider Before Building a SOA Testing Team
With all of the focus on the proper design of business services, business alignment, and SOA governance, sometimes the challenges that SOA creates for testers is overlooked early on in SOA initiatives.
CIO — We questioned three individuals to get viewpoints on SOA testing from three different perspectives: trainer, vendor, practitioner. Randy Rice is a leading author, speaker and consultant in the field of software testing and software quality. Randy's company provides training for SOA testing. Frank Cohen is an author and founder of open source testing software company, Push2Test. Jim Giddings is a practitioner and former SOA road warrior who has worked on several SOA initiatives over the past few years.
What is the biggest change that SOA brings to testing?
RR: Randy states that "accessibility to services" or the "headless nature of services" creates new challenges for testers. Testers will have to test many services without the aid of user interfaces and must perform rigorous testing to validate these services. Randy says that testing services reminds him of the challenges of testing drivers which require test harnesses and testing frameworks.
FC: Frank talks about how services are "always available" which is different from n-tier architectures where "you are in control". With SOA you have services calling other services and in the world of mashups, testers do not always have knowledge of how these services will be used before deployment. Scale and performance are also high on Frank's list of challenges.
JG: Jim says you should expect frequent testing cycles much like agile projects and a test driven development methodology should be employed. Jim states that testing governance becomes critical in an SOA. Like his peers, Jim discusses the challenges of the headless nature of services and performance testing.
Are you seeing challenges with testers having the required technical skills to understand SOA?
RR: Randy stresses the importance of strong technical knowledge. He states that testers come from different areas of the enterprise (business, development, etc.) and may not necessarily have the proper skills for SOA. He also stresses the importance of the need for more business orientation which he calls the "forgotten area of testing". Testers who focus solely on the technology, miss the business process side of the equation. Non-business savvy testers often do not have the right level of access to business people.
FC: Frank points out that testers are starting to code and coders are starting to test. Test driven development has created this shift in roles and responsibilities. If you have testers who cannot code, this may be a challenge. Another point Frank makes is that common record and playback methods do not work for SOA and frameworks are needed. Testers must learn how to work with frameworks which may require scripting and/or coding. Testers must also understand the concepts of statefullness.
JG: Jim mentions that many testers equate SOA to Web Services and do not fully understand the scope of the architecture. Not all testers are technical enough to "dig into the architecture as implemented". SOA requires more than just black box testing. Jim says SOA requires a combination of black, white, and gray box testing since there is "so much going on at each layer" of the architecture.
