Crispin, Lisa - Agile Testing - A Practical Guide for Testers and Agile Teams

Testing Web Services.Web services is just another form of an API that enables other applications to access your application. Let’s talk about some of the tools you can use to test various inputs into your system.

CrossCheck. CrossCheck is one example of a tool for testing web services. You supply the WSDL (Web Services Description Language); CrossCheck compiles the page and then presents you with a tabbed menu that contains textboxes for you to fill in. It has a Run mode where you can add your tests to a suite and then run the suite. Neither Lisa or Janet have tried this tool, but it was noted on the Yahoo agile-testing group as a tool to use for testing web services if you were running the same data through each time.

Ruby Test::Unit. One project Janet was on used Ruby’s unit testing framework, Test::Unit, to test web services, with great success. In fact, the team was able to test early to give the programmers immediate feedback, which helped with the final design.

See the “System Test” example in Chapter 12, “Summary of Testing Quadrants,” to see how Janet’s team used Ruby Test::Unit to test web services.

soapUI. Another tool suggested for testing web services is soapUI. It has a steep learning curve but can be used for performance and load testing. Because it can loop though rows in an Excel spreadsheet or text file, it can be used for data-driven testing.

Tests that work at the layers below the presentation layer are well suited for writing and automating customer tests that guide coding. Some practitioners haven’t gotten the value they expected from story test-driven development. Brian Marick [2008] hypothesized that an application built with programmer test-driven development, example-heavy business-facing design that relies heavily on whiteboard discussions, a small set of automated sanity tests, and lots of exploratory testing could be a less expensive and equally effective approach. Whichever approach you take, if you’re testing an application with a user interface, you’ll need some automation at the GUI level.

Tools for Testing through the GUI

Wait a minute. How can we use GUI tests to drive development, because the GUI won’t be ready until the story is complete? It sound counterintuitive, but automated GUI tests are important to help us while we’re developing new functionality. Test frameworks can be used to specify test cases for a GUI tool before the code is written. In addition, you can automate GUI tests before coding is finished, either by using HTML mock-ups or by developing an end-to-end bare-bones slice through all of the screens that simply navigates but doesn’t provide all of the functionality yet. Even if you’re not using a lot of automated story tests to drive development, manual exploratory testing that helps us learn about the functionality and provides immediate feedback gets pretty tedious and slow without any assistance from automation. Let’s look at the types of GUI test tools that help drive development using business-facing tests.

A Tool Selection Rationale

David Reed, a test automation engineer, and his team went with soapUI Pro to automate testing for their web services. Here are some reasons he gave for choosing this particular tool.

• It has an open source version, so you can try it out it for free. You can learn it, kick the tires, expand stuff, and learn its strengths and weaknesses.

• It was easy to figure out what requests to make for what service.

• The assertions provided for verifying the results from requests are great and expandable. One really helpful one is verifying that the response comes back in an acceptable amount of time, raising an error if it doesn’t.

• The Pro version takes a lot of the hassle out of designing XPath queries to verify results. It also adds some nice touches for retrieving database data.

• It’s expandable with Groovy, a Java-based scripting language. (They’re working on a Java application, so it pays to have Java-friendly tools.)

• Developers can use it without sneering at it as a “test tool.”

• It’s easily integrated with our continuous integration environment.

• It has a feature to check code coverage.

• The price is right.

Record/Playback Tools

Record/playback tools are appealing because you can usually learn how to record a script and play it back quickly, and you can create lots of scripts in a short time. However, they have drawbacks. Early GUI test tools recorded mouse movements using X-Y screen coordinates. Scripts using those tools might also be sensitive to changes in screen resolution, color depth, and even where the window is placed on the screen.

Most modern GUI test tools use objects to recognize the controls in a graphical application, like buttons, menus, and text input widgets, so they can refer to them symbolically rather than with raw screen coordinates. This makes the application much more testable, because it’s more robust standing up to changes. A button might move to a different part of the screen, but the test can still find it based on its object name.

Even with improved object recognition, scripts created with record/playback are usually brittle and expensive to maintain. Recording can be a good way to start creating a script. Testers or programmers who know the tool’s scripting language can refactor the recorded script into an object-oriented model that’s easier to use and maintain. Historically, record/playback tools used proprietary scripting languages, which programmers aren’t interested in learning. It’s also more difficult to change the design patterns used in the tests.

Some script-based tools such as the ones we’ll talk about in the next few sections offer a record feature to help people get a quick start on writing the test script. However, with those tools, the recorded scripts aren’t intended for straight playback; they’re just a starting point to creating a well-designed and easily maintained suite of tests.

Many agile teams prefer tools and scripting languages that let them create their own domain-specific language (DSL). This makes tests much easier for business experts to understand and even write. Let’s look at some of these next.

Agile Open Source Test Tools

Each of the tools in this section was originally written by an agile development team that needed a GUI test tool and couldn’t find any third-party tools that worked for its situation. With these tools, you can write scripts that use web applications just like a human user. They fill in text fields, select from lists, and click checkboxes and buttons. They provide a variety of ways to verify correct navigation and contents of pages, such as tool-specific verify steps or XPath. Some of these tools have a higher learning curve than simple record/playback tools, but the extra investment of time usually pays off in scripts with a low total cost of ownership.

Ruby with Watir.Watir (Web Application Testing in Ruby) is a simple open source Ruby library for automating web browsers that works with Internet Explorer on Windows. There are different flavors for other browsers, including FireWatir for Firefox and SafariWatir for Safari.

Janet’s Story

I worked on a project that developed a three-layer test framework using Ruby and Watir. The first layer was a common set of libraries, and the second layer was to access the pages and provide navigation. The third and top layer created a domain language using fixture-type methods that mapped to the business needs. This allowed the manual testers to write high-level automated tests for workflows before coding was completed. If a fixture didn’t exist because of new functionality, the test could be created and the action word for the missing fixture could be “dummied” in. As soon as the fixture was coded, the test could be run as an acceptance test.