Crispin, Lisa - Agile Testing - A Practical Guide for Testers and Agile Teams

As we described in the last two chapters, you may have already gone over mock-ups or paper prototypes with your customers. If tasks to mock up a report or interface remain in the iteration plan, remember to keep the process simple. For example, don’t code an HTML prototype when drawing on a whiteboard will do just as well. We want to keep the process as simple as possible; simplicity is a core value.

Show Customers

As soon as a coded user interface or report is ready, even if it’s still rudimentary, lacking all features or displaying hard-coded data, show it to the appropriate customers. Nobody can explain exactly what they want ahead of time. They need to see, feel, and use the application to know if it’s right. You may not be able to implement big changes mid-iteration, but if you start early, there may be time for minor tweaks, and your customers will know what to expect.

The iteration review meeting is a great opportunity to show what the team delivered and get feedback for the next iteration, but don’t wait until then to get input from customers. Keep them involved throughout the iteration.

Understand the Business

Although we get caught up in the fast pace of iterations, we also need to stop and take time to understand the business better. Spend some time talking to business people about their jobs and what aspects might be enhanced with new software features. The better you understand your customer’s business, the better you can be at providing a good product.

Lisa’s Story

My team budgets time for each development team member to sit with the retirement plan administration team members as they do their daily work. Not only do we understand those jobs better, but we often identify small changes in the application that will make the administrator’s work easier.

Simple additions such as a bit of extra data provided, an additional search filter, or changing the order of a display can make a big difference to a tedious and detailed process. We also document what we learn with flow charts and wiki pages so that other team members can benefit.

—Lisa

Some teams actually sit with the business people permanently so that they are involved with the actual business on a daily basis.

Completing Testing Tasks

Agile testers are proactive. We don’t sit and wait for work to come to us. Testers who are accustomed to a waterfall process may feel there’s nothing to do until a story is 100% complete. That’s rarely true during an agile iteration. Work with programmers so that they produce some testable piece of code early on. The shipping cost algorithm presented earlier is a good example. It can be tested completely in isolation, without needing to access the database or the user interface. Alternatively, the user interface could be stubbed out with hard-coded data before the services accessing the real data are complete, and the behavior of the presentation layer can be tested by itself.

Peril: The Testing Crunch

Even experienced agile teams often experience a testing crunch at the end of an iteration. Maybe a story or two turned out to take much longer than expected, or a production problem took time away from development. What happens when tomorrow is the end of your iteration and your task board (real or virtual) is still full of testing cards?

If you see this, recognize it as a bad smell. Work with the team to determine what the problem may be. Are the programmers not working closely enough with the testers? Were there too many interruptions?

The way to address this peril is to involve the whole team. Remember that anyone on the team can sign up for testing tasks. In your daily stand-up, you can evaluate whether the team is on track to finish all of the stories. If multiple stories are in danger of not being completed, choose a story to drop, or reduce the scope on one or more stories. Focus on completing one story at a time. As the end of the iteration approaches, programmers may have to stop working on new features and start picking up testing tasks instead. Missing some functionality from a release is better than missing the entire release because testing couldn’t be completed on all or most stories.

The programmers on Lisa’s team regularly automate behind-the-GUI tests in addition to unit and integration tests. They also often write the functional behind-the-GUI test cases. Sometimes they write the initial happy path executable test so they can coordinate test and code design; then a tester adds more test cases. Occasionally, they write all of the functional test cases, because the testers don’t have the bandwidth to cover all of the test-intensive stories.

Everyone on the team also must be willing to take on manual testing tasks. If your team is just starting and hasn’t been able to address automation needs yet, the whole team should plan time to execute manual regression test scripts as well as manually testing new features. As Lisa’s team can attest, this task provides great motivation for learning how to design the application to facilitate test automation. Other teams tell us this worked for them as well.

Dealing with Bugs

We’ve known many teams that struggle with the question of how to track bugs, or whether to track them at all. As Tom and Mary Poppendieck write in their book Implementing Lean Software Development: From Concept to Cash [2006], defect queues are queues of rework and thus collection points for waste. Some teams simply fix bugs as soon as they’re discovered. They write a unit test to reproduce the bug, fix the code so the test passes, check in the test and the bug fix, and go on. If someone breaks that piece of code later, the test will catch the regression.

Other teams find value in documenting problems and fixes in a defect tracking system (DTS), especially problems that weren’t caught until after code was released. They may even look for patterns in the bugs that got to production and do root cause analysis to learn how to prevent similar issues from recurring. Still, defect systems don’t provide a good forum for face-to-face communication about how to produce higher-quality code.

Chapter 5, “Transitioning Typical Processes,” talks about why your team may or may not want to use a Defect Tracking System.

Lisa and her fellow testers prefer to talk to a programmer as soon as a problem is found. If the programmer can fix it immediately, there’s no need to log the bug anywhere. If no programmer is available immediately to work on the problem, and there’s a possibility the bug might be forgotten, they write a card for it or enter it into their DTS.

We’ve added this section to this chapter because this is when you run into the problem. You have been writing tests first, but are finding problems as you work with the programmer. Do you log a bug? If so, how? You’ve been doing your exploratory testing and found a bug from a story that was marked done. Do you log a bug for that? Let’s discuss more about defects and consider options that are open to you and your team.

Is It a Defect or Is It a Feature?

First, let’s talk about defects versus features. The age-old question in software development is, “What is a bug”? Some answers we’ve heard are: It’s a deviation from the requirements or it’s behavior that is not what was expected. Of course, there are some really obvious defects such as incorrect output or incorrect error messages. But what really matters is the user’s perception of the quality of the product. If the customer says it is a defect, then it is a defect.

In agile, we have the opportunity to work with customers to get things fixed to their satisfaction. Customers don’t have to try to think of every possible feature and detail up front. It is okay for them to change their minds when they see something.