Crispin, Lisa - Agile Testing - A Practical Guide for Testers and Agile Teams

Hudson Hacker, who looks for ways to cheat the checkout page

Enrico Executive, who does all his shopping online and ships gifts to all his clients worldwide

Betty Bargain, who’s looking for great deals

Debbie Ditherer, who has a hard time deciding what items she really wants to order

We might hang photos representing these different personas and their biographies in our work area so that we always keep them in mind. We can test the same scenario as each persona in turn and see what different experiences they might encounter.

Another way to approach persona testing, which we learned from Brian Marick and Elisabeth Hendrickson, is to pick a fictional character or famous celebrity and imagine how they would use our application. Would the Queen of England be able to navigate our checkout process? How might Homer Simpson search for the item he wants?

Real World Projects: Personas

The OneNote team at Microsoft uses personas as part of their testing process. Mike Tholfsen [2008], the Test Manager for OneNote, says they use seven personas that might use OneNote, specific customer types such as Attorneys, Students, Real Estate Agents, and Salespersons. The personas they create contain information such as:

• General job description

• “A Day in the Life”

• Primary uses for OneNote

• List of features the persona might use

• Potential notebook structures

• Other applications used

• Configuration and hardware environment

You can also just assume the roles of novice, intermediate, and expert users as you explore the application. Can users figure out what they are supposed to do without instructions? If you have a lot of first-time users, you might need to make the interface very simple.

Janet’s Story

When I first started testing a new production accounting system, I found it very difficult to understand the flow, but the production accountants on the team loved it. After I worked with it for a while, I understood the complexity behind the application and knew why it didn’t have to be intuitive for a first-time user. This was a good lesson for me, because I always assumed applications had to be user-friendly.

—Janet

If your application is custom-built for specific types of users, it might need to be “smart” rather than intuitive. Training sessions might be sufficient to get over the initial lack of usability so that the interface can be designed for maximum efficiency and utility.

Navigation

Navigation is another aspect of usability testing. It’s incredibly important to test links and make sure the tabbing order makes sense. If a user has a choice of applications or websites, and has a bad first experience, they likely won’t use your application again. Some of this testing is automatable, but it’s important to test the actual user experience.

If you have access to the end users, get them involved in testing the navigation. Pair with a real user, or watch one actually use the application and take notes. When you’re designing a new user interface, consider using focus groups to evaluate different interfaces. You can start with mock-ups and flows drawn on paper, get opinions, and try HTML mock-ups next, to get early feedback.

See Chapter 8, “Business-Facing Tests that Support the Team,” for an example of Wizard of Oz testing, which is one approach to designing for usability.

Check Out the Competition

When evaluating your application for usability, think about other applications that are similar. How do they accomplish tasks? Do you consider them user-friendly or intuitive? If you can get access to competing software, take some time to research how those applications work and compare them with your product. For example, you’re testing a user interface that takes a date range, and it has a pop-up calendar feature to select the date. Take a look at how a similar calendar function works on an airline reservation website.

Usability testing is a fairly specialized field. If you’re producing an internal application to be used by a few users who will be trained in its use, you probably don’t need to invest much in usability testing. If you’re writing the online directory assistance for a phone company, usability might be your main focus, so you need to learn as much as you can about it, or bring in a usability expert.

See the bibliography for links to articles by Jeff Patton, Gerard Meszaros, and others on usability testing.

Behind the GUI

In a presentation titled “Man and Machine” [2007], Jonathan Kohl talked about alternatives for testing interfaces. Instead of always thinking about testing through the user interface, consider attacking the problem in other ways. Think about testing the whole system from every angle that you can approach. Consider using tools like simulators or emulators.

Chapter 9, “Toolkit for Business-Facing Tests that Support the Team,” provides more detail about tools that facilitate these tests.

API Testing

In Chapter 8 and Chapter 9, we talked about testing behind the GUI to drive development. In this section, we show that you can extend your tests for the API in order to try different permutations and combinations.

An API (application programming interface) is a collection of functions that can be executed by other software applications or components. The end user is usually never aware that an API exists; she simply interacts with the interface on top.

Each API call has a specific function with a number of parameters that accept different inputs. Each variation will return a different result. The easy tests are simple inputs. The more complicated testing patterns occur when the parameters work together to give many possible variations. Sometimes parameters are optional, so it’s important that you understand the possibilities. Boundary conditions should be considered as well, for both the inputs and expected results. For example, use both valid and invalid strings for parameters, vary the content, and vary the length of the strings’ input.

Another way to test is to vary the order of the API calls. Changing the sequence might produce unexpected results and reveal bugs that would never be found through UI testing. You can control the tests much more easily than when using the UI.

Lisa’s Story

My team was working on a set of stories to enable retirement plan sponsors to upload payroll contribution files. We wrote FitNesse test cases to illustrate the file parsing rules, and the programmer wrote unit tests for those as well. When the coding for the parser was complete, we wanted to throw a lot more combinations of data at the parser, including some really bizarre ones, and see what happened. We could use the same fixture as we used for our tests to drive development, enter all of the crazy combinations we could think of, and see the results. We tested about 100 variations of both valid and invalid data. Figure 10-3 shows an example of just a few of the tests we tried. We found several errors in the code this way.

Figure 10-3 Sample of parsing rules test

We didn’t keep all of these tests in the regression suite because they were just a means of quickly trying every combination we could think of. We could have done these tests in a semi-automated, ad hoc manner too, not bothering to type the expected results into the result checking table, and just eyeballing the outputs to make sure they looked correct.