Crispin, Lisa - Agile Testing - A Practical Guide for Testers and Agile Teams

We’ve found that retrospectives are a simple and highly effective way for teams to identify and address issues. The retrospective meeting is a perfect opportunity to raise testing-related issues. Bring up the issues in an objective, non-blaming way. The team can discuss each problem, what might be causing it, and write down some ideas to fix it.

Ideas for Improvements

Let’s take a look at some of those items that made it onto the list for improvement. Too many times, a team will identify really big issues but never follow up and actually do something about them. For example, maybe a lot of unit-level bugs are discovered after the programmers have claimed coding was complete.

The team may decide the programmers aren’t covering enough code with unit tests. They might write an action item to run the code coverage tool before they check in new code, or start writing a “unit tests” task card for each story to make sure they’re completed. Perhaps the team didn’t finish all the test automation tasks before the iteration ended. As they discuss the problem, the team finds that the initial executable tests were too complex, and they need to focus on writing and automating a simple test first, or pair for a better test design. Make sure the action items are concrete.

Agile teams try to solve their own problems and set guidelines to help themselves improve. Action items aimed at one problem may help with others. When Lisa’s team had trouble finishing stories and getting them tested during each iteration, it came up with various rules over the course of a few retrospectives:

Finish high-level test cases for all stories by the fourth day of the iteration.

Deliver one story to test by the fourth day of the iteration.

Focus on finishing one story at a time.

100% of features must be checked in by close of business on the next-to-last day of the iteration.

These rules did more than help the team finish testing tasks. They facilitated a flow and rhythm that helped the team work at a steady, sustainable pace over the course of each iteration.

Begin the next retrospective meeting by reviewing the action items to see what items were beneficial. Lisa’s team puts happy, sad, or neutral faces next to items to denote whether the team tried them and found them successful. The team should figure out the reasons behind any sad faces. Were some items simply forgotten? Did time constraints keep the team from trying a new activity? Did it just seem to be less of a good idea later? These discussions might lead to changing the improvement item or evolving it into a new one.

When the actions for improvement become a habit to the team, they no longer need to be written on the “stop, start, and continue” list. “Start” items that work well may be moved to the “Continue” column. Some ideas don’t work, or prove to be unnecessary, and those can also be taken off the list for the next iteration.

Refer to your ideas for improvement and action items during the iteration. Post them in a location (on a wall or online) where everyone sees them often. Lisa’s team sometimes goes through the list during a mid-iteration stand-up meeting. If you think of new improvement ideas during the iteration, write them down, possibly even on the existing list, so you won’t forget for the next iteration.

It’s a good idea to keep track of things that get in your way throughout the iteration. Keep an impediment backlog on some big visible chart. Talk about the impediments in each iteration, and write task cards or take action to eliminate them.

An Approach to Process Improvement

Rafael Santos, VP of Software Development and Chief ScrumMaster at Ultimate Software, and Jason Holzer, the Chief PSR (Performance, Security, Reliability) Architect, explained to us that their teams found retrospectives that used the “stop, start, and continue” model ineffective. They made “stop, start, and continue” lists, but those didn’t provide enough focus to address issues.

Instead, the ScrumMaster kept an impediment backlog, and the team found that worked better than retrospectives. Impediments may be related to testing or tools.

They also do value stream mapping to find the biggest “wait time,” and use the “five whys” from Toyota to understand which impediment is the biggest or which constraint needs to be addressed.

See the bibliography for good resources for lean development practices.

One example shared was that in a team with three programmers and one tester, the biggest problem was a testing bottleneck. Rafael asked the team what the tester does and wrote those items on a whiteboard. Then he asked the programmers which of those things on the board they couldn’t do. There was only one item they felt they couldn’t handle. This helped the programmers understand how everyone on the development team, not only the testers, could be responsible for testing tasks. This was a highly effective exercise.

Creative approaches like this help new agile teams tackle difficult testing challenges. Retrospectives are a good environment for experimenting.

Use retrospectives as an opportunity to raise testing-related issues and get the whole team thinking about possible solutions. We’ve been pleasantly surprised with the innovative ideas that come out of an entire team focusing on how to improve the way it works.

Celebrate Successes

Agile development practices tend to moderate the highs and lows that exist in more traditional or chaotic processes. If your waterfall team finally manages to push a release out the door after a year-long cycle ending in a two-month stressful fix-and-test cycle, everyone may be ready to celebrate the event with a big party—or they might just collapse for a couple of weeks. Agile teams that release every two weeks tend to stay in their normal coding and testing groove, starting on the next set of stories after drawing just enough breath to hold an iteration review and retrospective. This is nice, but you know what they say about all work and no play.

Make sure your team takes at least a little time to pat itself on the back and recognize its achievements. Even small successes deserve a reward. Enjoyment is a vital agile value, and a little motivation helps your team continue on its successful path. For some reason, this can be hard to do. Many agile teams have trouble taking time to celebrate success. Sometimes you’re eager to get going with the next iteration and don’t take time to congratulate yourselves on the previous accomplishments.

Lisa’s team ends an iteration every other Thursday and conducts its retrospective, iteration review, and release the following day. After their meetings conclude, they usually engage in something they call “Friday Fun.” This sometimes consists of playing a silly trivia or board game, going out for a drink, or playing a round of miniature golf. Getting a chance to relax and have a good laugh has a team-building side benefit.

For bigger milestones, such as a big release or achieving a test coverage goal, the whole company has a party to celebrate, bringing in catered food or going out to a restaurant on Friday afternoon. This is a nice reward and recognizes for everyone on both the business and technical teams.

If yours is a new agile team, motivate yourselves by rewarding small accomplishments. Cheer the rising number of unit tests passing in each build. Oooh and aaah over the chart showing actual burn down matching the projected burn down. Ring a bell when the broken unit tests in the build are fixed (okay, that one might be annoying, but recognize it in some way.)