Crispin, Lisa - Agile Testing - A Practical Guide for Testers and Agile Teams

See the bibliography for links to more information on graphical tests and model-driven development.

A Picture Is Worth a Thousand Words

The saying “A picture says a thousand words” can also be applied to test cases and test validations.

Paul Rogers [2008] has been experimenting with some cool ideas around this and explains his team’s approach to its problem in the following sidebar. Figure 17-3 shows the UI model he describes.

The application I work on is very graphical in its nature. It allows a user to modify a web page by adding “photo enhancements” such as glasses, hats, or speech bubbles to images, or by highlighting the text in the web page with a highlighter pen effect.

There is a complex set of business rules as to what additions can be applied to images, how and where they are affixed, and how they can be rotated. To explain the tests for these rules, it was much simpler to draw a sketch of a typical web page with the different types of additions and add small notes to each picture.

Text highlighting also posed many challenges. Most problematic were the areas where text highlighting covered only part of an HTML tag. To describe what should be expected in many different situations, we created different web pages and printed them out.

Using real pen highlighters, we highlighted the areas we expected to show as highlighted after starting and ending in certain areas. This way, we had an easy-to-read regression test.

Figure 17-3 Sample of UI modeling technique

Low-tech tools can take the mystery out of complex application design. Find ways to express business rules as simply as possible, and share those with the entire team.

Mock-ups can convey requirements for a UI or report quickly and clearly. If an existing report needs modifying, take a screenshot of the report and use highlighters, pen, pencil, or whatever tools are handy. If you want to capture it electronically, try the Windows Paint program or other graphical tool to draw the changes and post it on the wiki page that describes the report’s requirements.

See the sample mock-up of UI changes in Chapter 16, “Hit the Ground Running.”

Distributed teams need high-level tests available electronically, while co-located teams might work well from drawings on a whiteboard, or even from having the customer sit with them and tell them the requirements as they code.

See Chapter 9, “Toolkit for Business-Facing Tests that Support the Team,” for some ideas on tools to gather and communicate requirements.

What’s important as you begin the iteration is that you quickly learn the basic requirements for each story and express them in context in a way that works for the whole team. Most agile teams we’ve talked to say their biggest problem is to understand each story well enough to deliver exactly what the customer wanted. They might produce code that’s technically bug-free but doesn’t quite match the customer’s desired functionality. Or they may end up doing a lot of rework on one story during the iteration as the customer clarifies requirements, and run out of time to complete another story as a result.

Put time and effort into experimenting with different ways to capture and express the high-level tests in a way that fits your domain and environment. Janet likes to say that a requirement is a combination of the story + conversation + a user scenario or supporting picture if needed + a coaching test or example.

See Chapter 8, “Business-Facing Tests that Support the Team,” for more about what makes up a requirement.

Reviewing with Customers

Earlier in this chapter we talked about the importance of constant customer collaboration. Reviewing high-level tests with customers is a good opportunity for enforced collaboration and enhanced communication, especially for a new agile team. After your team is in the habit of continually talking about stories, requirements, and test cases, you might not need to sit down and go over every test case.

If your team is contracting to develop software, requirements and test cases might be formal deliverables that you have to present. Even if they aren’t, it’s a good idea to provide the test cases in a format that the customers can easily read on their own and understand.

Reviewing with Programmers

You can have all of the diagrams and wiki pages in the world, but if nobody looks at them, they won’t help. Direct communication is always best. Sit down with the programmers and go over the high-level tests and requirements. Go over whiteboard diagrams or paper prototypes together. Figure 17-4 shows a tester and a programmer discussing a diagram of thin slices or threads through a user workflow. If you’re working with a team member in another location, find a way to schedule a phone conversation. If team members have trouble understanding the high-level tests and requirements, you’ll know to try a different approach next time.

Figure 17-4 A whiteboard discussion. Used with permission of Mike Thomas. Copyright 2008.

Programmers with good domain knowledge may understand a story right away and be able to start coding even before high-level tests are written. Even so, it’s always a good idea to review the stories from the customer and tester perspective with the programmers. Their understanding of the story might be different than yours, and it’s important to look at mismatches. Remember the “Power of Three” rule and grab a customer if there are two opinions you can’t reconcile. The test cases also help put the story in context with the rest of the application. Programmers can use the tests to help them to code the story correctly. This is the main reason you want to get this done as close to the start of the iteration as you can—before programmers start to code.

Chapter 2, “Ten Principles for Agile Testers,” introduces the “Power of Three” rule.

Don’t forget to ask the programmers what they think you might have missed. What are the high-risk areas of the code? Where do they think the testing should be focused? Getting more technical perspective will help with designing detailed test cases. If you’ve created a test matrix, you may want to review the impacted areas again as well.

One beneficial side effect of reviewing the tests with the programmers is the cross-learning that happens. You as a tester are exposed to what they are thinking, and they learn some techniques for testing that they would not have otherwise encountered. As programmers, they may get a better understanding of what high-level tests they hadn’t considered.

Test Cases as Documentation

High-level test cases, along with the executable tests you’ll write during the iteration, will form the core of your application’s documentation. Requirements will change during and after this iteration, so make sure your executable test cases are easy to maintain. People unfamiliar with agile development often have the misconception that there’s no documentation. In fact, agile projects produce usable documentation that contains executable tests and thus is always up to date.

The great advantage of having executable tests as part of your requirements document is that it’s hard to argue with their results.

Lisa’s Story

Frequently, product owners, plan administrators, or business development managers will come and ask me a question such as, “What’s the system supposed to do if someone submits a loan payment for zero dollars?” or “Why didn’t everyone in this plan get a 3% nonelective contribution?”