SOAPSonar's fundamental testing unit is a test case. Normally, test cases are organized into suites; however, SOAPSonar's project tree view allows you to work with test cases in a sort of staging area. Test cases appear as nodes in the tree, attached to the parent node of WSDL-based Web service to which they apply. You can craft as many test cases against a given WSDL as you wish. Once you have a test case that has been verified and deemed ready, it can be moved into a test suite.

Click for larger view.

SOAPSonar operates in one of several modes. You choose the Current mode from a menu selection, and typically, you run SOAPSonar in QA mode, which provides functional testing. However, if you switch to Performance mode, SOAPSonar's load-testing features are enabled. You can define multiple virtual clients and configure them to execute tests against target Web services. SOAPSonar can also exercise a Web service's resilience to security attacks, using a patent-pending XSD Mutation technology. XSD Mutation modifies an outgoing SOAP request in ways that expose the Web service to known assaults such as SQL injection, XML bombs, and so on.

Although SOAPSonar is, in accordance with its name, primarily a SOAP-testing application, it can test REST (Representational State Transfer)-style Web service interfaces as well. For a given REST-style test case, you can enter comma-separated name-value pairs that the tool assembles into the request URL. You can also use SOAPSonar's entire range of input data creation capabilities to generate input values for the request, giving you the ability to craft as rich a set of REST-style tests as SOAP-style tests.

Once your army of tests is created, you can automate their execution, provided you purchase the additional APC License Component. This component adds features to the standard SOAPSonar package that include a command-line interface for integrating test scripts with the Windows Task Scheduler.

There is also a free edition of SOAPSonar: Personal Edition. It lacks features such as WS-Security validation, performance testing, and vulnerability testing. A comparison between the Personal Edition and the Enterprise Edition is available at the company's Web site.

SOAPsonar presents itself as the critical tool you need to fulfill Crosscheck Networks' vision of a Web service testing way of life: the “four pillars of SOA deployment diagnostics.” There's the functional pillar: verifying a given request produces the correct response and that a Web service fulfills its design requirements. Second is performance: measuring a Web service’s throughput and response times. Third, compliance: verification of adherence to recognized standards. Last is vulnerability: ensuring that the Web service is tolerant of and resistant to malformed requests. This is a fine collection of Web service testing principles, and SOAPSonar does an admirable job of upholding them.

iTKO LISA 3.6e

LISA's learning curve is smooth and easy. The tool imposes a cyclic test development process. Create a new test case, and LISA builds a test structure consisting of skeletal test steps that act like bookends – one is the start step, the other is the end step. Import a WSDL through the Web Service Step Wizard, and you're handed a list of that WSDL's Web methods. Select a method to work on, and the wizard opens an object editor to supply input data, simultaneously adding a new step between the bookends. Enter test data for the request, then submit the request to the Web service and see if what comes back looks right. If not, go back, tweak the step (or correct the Web service method), and try again. There are more details to this, of course – compliance testing, for one, but LISA supports that as well.

Apply the above process repeatedly for the different Web methods on the WSDL, and ultimately you'll have a complete test case for a specific WSDL. Test cases can, in turn, be gathered into a test suite, which is really just a kind of folder in the LISA environment.

Click for larger view.

LISA provides a healthy collection of test step types, though in most test cases, the majority of steps are of the "Web service execution step" type: Send a request, examine the response, and determine success or failure. Other step types can verify a Web service's compliance with various standards, execute external Java classes, or even call command-line scripts.

In addition, each test step can be adorned with a variety of filters and assertions. The former is provided to parse the content of response messages. For example, you might apply a filter to fetch a specific response value and store it in a property for use in later steps. The assertions manage verification of response data, WSDL, and message conformance. Also, the assertion section of a step specifies whether it has passed or failed, and it identifies whether execution control should proceed to the next step or to some other step.

The full LISA product is very Java-aware. It can generate JUnit tests, functional tests of Java classes, database tests via JDBC, and EJB tests. The free version, WS-Testing, is limited to generating only test cases and for Web services only. In addition, some test steps types are unavailable (it lacks any J2EE-related test step types, for example).

For all the initial ease of learning LISA, navigating the UI is sometimes bumpy. For example, when entering a new value for a field in a test step, there is no obvious way to save that value, nor to cancel the change. I found the only way to cancel input was to select a different node in the explorer, then dismiss the dialog that asked if I really wanted to do that.

The LISA documentation makes a big deal of no-code test development, as if that is the high road to simultaneously simplifying and accelerating test development. Perhaps, but while LISA's pure UI-approach does have the benefit of live interaction and is more accessible to QA engineers inexperienced at coding, it has limitations that a tool with easier access to scripting does not. Some testing nuts can only be cracked by a well-sharpened piece of code.

Mindreef SOAPscope Server 6.0

SOAPscope Server, like QEngine, is a thin-client-based tool. Behind SOAPscope's browser UI is a Tomcat server, girded by an RDBMS (relational database management system) that can be MySQL, Oracle, Microsoft SQL Server, or the embedded Apache Derby database. (Derby is supplied with SOAPscope but not recommended for even moderately large installations.)

SOAPscope Server's service spaces are the overarching containers of testing assets. An administrator will use service spaces to organize users into groups. Within a service space, member users can create one or more workspaces in which to store their, well, work.

Inside a workspace you'll find WSDL contracts, tests, notes, and other ancillary material needed to support actual testing. Typically, a workspace corresponds to a WSDL: When you create a new workspace, the first prompt you encounter is for a WSDL URL. You can, however, add more WSDL contracts to the workspace once it is created. Once you've imported a WSDL into a workspace, you can begin adding messages to that workspace. A message is really a SOAP request/response pair, created when you invoke a Web method on a WSDL. The invocation also optionally creates an "action" within the workspace.