Crispin, Lisa - Agile Testing - A Practical Guide for Testers and Agile Teams

Showing them a FitNesse test that replicates the scenario is much more powerful than just showing them narrative requirements. Maybe the system wasn’t designed the way it should have been, but the test illustrates how it actually works, because we can clearly see the results of the inputs and operations. This has saved a lot of arguments on the level of “I thought it worked this way.”

If they decide the functionality, as implemented, is incorrect, we can change the expected outputs of the test and write a story to implement code to make the test pass again with the new expectations. You can’t do that with a requirements document.

—Lisa

Organizing the test cases and tests isn’t always straightforward. Many teams document tests and requirements on a wiki. The downside to a wiki’s flexibility is that you can end up with a jumble of hierarchies. You might have trouble finding the particular requirement or example you need.

Lisa’s team periodically revisits its wiki documentation and FitNesse tests, and refactors the way they’re organized. If you’re having trouble organizing your requirements and test cases, budget some time to research new tools that might help. Hiring a skilled technical writer is a good way to get your valuable test cases and examples into a usable repository of easy-to-find information.

Chapter 14, “An Agile Test Automation Strategy,” has more on test management.

Summary

The iteration planning session sets the tone for the whole iteration. In this chapter, we looked at what agile testers do to help kick off the iteration to a good start.

During iteration planning, testers help the team learn about the stories by asking questions and considering all viewpoints.

Task cards need to be written along with development task cards and estimated realistically.

Another way of tackling testing tasks is to write them directly on the developer task cards.

Teams should commit to the work for which they can complete all of the testing tasks, because no story is done until it’s fully tested.

The start of an iteration is the last chance to ensure that the stories are testable and that adequate test data is provided.

Testers collaborate with customers to explore stories in detail and write high-level test cases to let programmers kick off coding.

Testers review high-level tests and requirements with programmers to make sure they are communicating well.

Tests form the core of the application’s documentation.

Chapter 18 Coding and Testing

Our agile tester has helped plan the release, size stories appropriately, and make sure they’re testable. She, along with colleagues on the customer and development team, has turned examples of desired behavior for each story into high-level user acceptance tests. She and her team have lined up the resources and infrastructure needed to deliver business value. Now, team members have picked up task cards and started writing code. What do testers do next, especially before any stories are ready to test?

Driving Development

The beginning of coding is a good time to start writing detailed tests. The high-level tests written before the iteration, or in the first couple days of it, provide enough information for the programmers to start their own test-driven development. So now we have a bit of breathing room, but if we don’t move quickly, coding could get way ahead of testing and go off in the wrong direction.

Now’s the time to start writing executable tests that illustrate the details about a story in order to keep development moving forward smoothly and help testing keep pace with coding. Like the high-level tests, we base detailed tests on examples provided by the customers.

At this point, we’re mainly writing tests that will be automated, but we’re also thinking ahead to the important exploratory testing we need to do as coding is completed.

Start Simple

As testers, we’re easily distracted by interesting code smells and edge cases. However, if we’re using tests to guide coding, we have to start with the basics. Write the simplest happy path test you can in order to show that the core functionality works.

Why executable tests? We’re working on an extremely tight schedule, and neither the programmers nor the testers have time to stop and run manual tests over and over. They do have time to click a button and run an automated test. That test needs to fail in a way that makes the cause as obvious as possible. Ideally, we would give these tests to the programmers so that they could execute them as they code. That is one reason why picking the right automation framework is so important.

Chapter 14, “An Agile Automation Strategy,” gives pointers for selecting the right tools.

For some stories, automating the tests might take a long time. By keeping the first test simple, you keep the focus on designing the automation solution. When the simple test works, it’s worth putting time into more complex test cases.

We stress the importance of automation, but Janet has worked with teams that have successfully used manual tests in the form of checklists or spreadsheets to give the programmers the information they need to start. However, to be successful in the long run, these tests do need to be automated.

Add Complexity

As soon as the happy path test works, start adding more test cases. Add boundary and edge conditions. The tests may show that the programmers misunderstood a requirement, or they may show that a tester did, or maybe the requirement’s true meaning eluded everyone. The important thing is that everyone talks about it and gets on track.

As testers think of new scenarios to validate with executable tests, they also think about potential scenarios for manual exploratory testing. Make a note of these for later pursuit.

Remember the purpose of these tests. They should provide examples that tell the programmers what code to write. As the code evolves, your tests can challenge it more, but resist the temptation to immediately follow smells into edge cases. Get the basics working first. If you think of more cases based on some risk analysis, you can always add extra tests later.

Assess Risk

Testers have used risk analysis to help prioritize testing for a long time, and consideration for risk is already built into agile development. High-risk stories may get higher size estimates, and teams consider risk as they prioritize stories during release and iteration planning.

Some quick risk analysis can help you decide what testing to do first and where to focus your efforts. We never have time to test everything, and we can use risk analysis to figure out how much testing is just enough.

If you have a really complex story, you may want to start by listing all of the potential risks related to the story. These aren’t limited to functionality. Consider security, performance, usability, and other “ilities.” Next, for each item, rate the impact on the business if it were to occur, using a scale of 1 to 5 (or whatever scale works for you): 1 being a low impact, 5 being a critical negative impact.