Logging into QEngine ushers you into the suite manager screen. From there, you can either create or import a new test suite, or choose to work with an existing suite. If you choose the latter, a dialog materializes, giving you the option to work on Web functionality, Web performance, Web services functionality, or Web services performance. In other words, a single suite can host up to four categories of tests, and these categories are never really aware of one another. So, for example, if you're doing Web functionality test work, you don't see the Web services performance tests within that same suite. This takes some getting used to.

Click for larger view.

Scripts perform actual test execution. When you add a new Web service to a test suite, QEngine queries the Web service's WSDL and from that generates a set of basic test scripts – one for each Web method on the service. Select a script from a suite's explorer tree, and the generated source code appears in an editor window. QEngine's scripts are written in Jython, the implementation of the Python language that executes in a Java virtual machine.

The prebuilt test scripts are extremely spartan; you have to flesh them out for them to be useful. This is a two-step process. First, you supply the content of the requests using a pair of menu selections: DataSet Configuration and Parameterization. Dataset Configuration lets you set the sources of your input data as either a database (QEngine supports Oracle, SQL Server, or MySQL) or a CSV file. After you've configured your datasets, choose Parameterization and you can set specific input values to be supplied either by the dataset you just configured or by manually entered values.

Second, you add response analysis to the script. This requires coding, but QEngine helps out with a large selection of built-in response-processing functions. A Function Generator dialog simplifies choosing the right method to call. Select a function from the categorized list, and the dialog provides a description and an input parameter list. In effect, it fills out the function call for you and pastes it into the script.

Even with QEngine's hand-holding, validating a response is not particularly easy, unless you choose a simple processing function. To really get inside the response, you have to pretty much peel apart the XML, so you might want to keep an XPath manual on hand. When you execute a script, the results are gathered into a report summarization screen. It's loaded with links that you can click to drill down into the specifics of the success or failure.

QEngine strikes a good balance between reliance on the UI and raw coding. Through the Parameterization dialog, you can enter test data without having to wrestle with XML. But when XML-wrestling is unavoidable, you can drop into Jython code and wreak whatever havoc you deem necessary. However, QEngine's user interface is not easy to navigate; more than once, I found myself unable to backtrack to a known location. In addition, inactivity for a period of time (which I was unable to deduce) logs you out. You have to log back in and crawl back to where you left off.

Crosscheck Networks SOAPSonar 3.0.5

SOAPSonar offers numerous ways to feed data into SOAP requests. Using the tool’s Automation Data Source, you can input data from an SQL database via ODBC (Open Database Connectivity), an Excel spreadsheet, or a raw file. If you want to reuse data in a previous SOAP request, you can configure the request to use a recalled entry, which propagates data from a previous response into subsequent requests. For the ultimate in data-generation flexibility, you can call upon an automatic data function. Such functions are drawn from an extensive library that ranges from manually entered data series to arbitrarily complex user-defined data generation algorithms.