Crispin, Lisa - Agile Testing - A Practical Guide for Testers and Agile Teams

Some teams keep testing tasks to a day’s work or less and don’t bother to write estimated hours on the card. If a task card is still around after a day’s work, the team talks about why that happened and writes new cards to go forward. This might cut down on overhead and record-keeping, but if you are entering tasks into your electronic system, it may not. Do what makes sense for your team.

Estimating time for bug fixing is always tricky as well. If existing defects are pulled in as stories, it is pretty simple. But what about the bugs that are found as part of the iteration?

Janet’s Story

With new agile teams, I found that they always seem to end up with time spent on bugs that wasn’t allotted as part of their estimates for the stories. Over time, programmers learn how much time they typically spend fixing bugs from a story, and can just add half a day or a couple hours to their tasks for that purpose. Retesting bug fixes adds time to tester’s estimates as well.

Until the team members get a handle on this, it may be appropriate to track the time spent on fixing and testing bugs separately. My current team adds a story in XPlanner with tasks for fixing and testing those bugs that didn’t get caught immediately. We are tracking the time so we can better estimate down the road.

—Janet

However your team chooses to estimate time spent for fixing defects during the iteration, whether it is included in the story estimate or tracked separately, make sure it is done consistently.

Another item to consider when estimating testing tasks is test data. The beginning of an iteration is almost too late to think about the data you need to test with. As we mentioned in Chapter 15, “Tester Activities in Release or Theme Planning,” think about test data during release planning, and ask the customers to help identify and obtain it. Certainly think about it as you prep for the next iteration. When the iteration starts, whatever test data is missing must be created or obtained, so don’t forget to allow for this in estimates.

Lisa’s Story

We worked on a theme related to quarterly account statements for participants in retirement plans. We were modifying a monthly job that takes a “snapshot” of each participant’s account on the specified date. The snapshot relies on a huge amount of data in the production database, including thousands of daily transactions. We planned ahead.

For the first iteration, we did a few stories in the theme knowing we could only test a few cases using some individual retirement plans for which we had data in the test database. We also knew we needed a larger-scale test, with all of the retirement plans in the database and at least an entire month’s worth of data. We wrote a task card to copy enough production data to produce one monthly “snapshot” and made sure the data was scrubbed to protect privacy.

Then we planned the full-blown test in the next iteration. This data enabled the testers to find problems that were undetectable earlier when only partial data was available. It was a nice balance of “just enough” data to do most of the coding and the full amount available in time to verify the complete functionality. Because the team planned ahead, the bugs were fixed in time for the critical release.

—Lisa

Deciding on Workload

We, as the technical team, control our own workload. As we write tasks for each story and post them on our (real or virtual) story board, we add up the estimated hours or visually check the number of cards. How much work can we take on? In XP, we can’t exceed the number of story points we completed in the last iteration. In Scrum, we commit to a set of stories based on the actual time we think we need to complete them.

Lisa’s current team has several years of experience in their agile process and finds they sometimes waste time writing task cards for stories they may not have time to do during the iteration. They start with enough stories to keep everyone busy. As people start to free up, they pull in more stories and plan the related tasks. They might have some stories ready “on deck” to bring in as soon as they finish the initial ones. This sounds easy, but it is difficult to do until you’ve learned enough to be more confident about story sizes and team velocity, and know what your team can and cannot do in a given amount of time and in specific circumstances.

Your job as tester is to make sure enough time is allocated to testing, and to remind the team that testing and quality are the responsibility of the whole team. When the team decides how many stories they can deliver in the iteration, the question isn’t “How much coding can we finish?” but “How much coding and testing can we complete?” There will be times when a story is easy to code but the testing will be very time consuming. As a tester, it is important that you only accept as many stories into the iteration as can be tested.

If you have to commit, commit conservatively. It’s always better to bring in another story than to have to drop one. If you have high-risk stories that are hard to estimate, or some tasks are unknown or need more research, write task cards for an extra story or two and have them ready on the sidelines to bring in mid-iteration.

As a team, we’re always going to do our best. We need to remember that no story is done until it’s tested, so plan accordingly.

Testable Stories

When you are looking at stories, and the programmers start to think about implementation, always think how you can test them. An example goes a long way toward “testing the testability.” What impact will it have on my testing? Part III, “The Agile Testing Quadrants,” gives a lot of examples of how to design the application to enable effective testing. This is your last opportunity to think about testability of a story before coding begins.

Janet’s Story

One team I worked with told me about issues they had in the previous release. The team was rewriting the first step of a multistep process. What they didn’t anticipate was that when the development on the new step started, the rest of the process broke. No testing could be done on any other changes in that iteration until the whole first step was finished.

Testability had not been considered when planning the story. In the next release, when they decided to rewrite the second step, they learned from their previous mistake. The programmers created an extra button on the page that allowed the testers to either call the new page (in flux) or the old page to allow them test other stories.

Remember to ask, “How can we test this?” if it is not obvious to you.

—Janet

During iteration planning, think about what kind of variations you will need to test. That may drive other questions.

Janet’s Story

During one iteration planning meeting that I was in, the programmers started talking about implementation and drawing pictures on the whiteboard to show what they were thinking.

I thought about it for a bit and asked the question, “Can it be done more simply? The permutations and combinations for testing your proposed implementation will make testing horrendous.”

The programmers thought about it for a couple of minutes and suggested an alternative that not only met the customer’s needs, but was simpler and easier to test. It was a win-win combination for everyone.

—Janet

When testability is an issue, make it the team’s problem to solve. Teams that start their planning by writing test task cards probably have an advantage here, because as they think about their testing tasks, they’ll ask how the story can be tested. Can any functionality be tested behind the GUI? Is it possible to do the business-facing tests at the unit level? Every agile team should be thinking test-first. As your team writes developer task cards for a story, think about how to test the story and how to automate testing for it. If the programmers aren’t yet in the habit of coding TDD or automating unit tests, try writing a “XUnit” task card for each story. Write programming task cards for any test automation fixtures that will be needed. Think about application changes that could help with testing, such as runtime properties and APIs.