Crispin, Lisa - Agile Testing - A Practical Guide for Testers and Agile Teams

The testers may lobby for getting an end-to-end tracer bullet through the code quickly, so they can build an automation framework, and then flesh it out as the story development proceeds. If there are stories that present a big testing challenge, it might be good to do those early on. For example, if the release includes implementing a new third-party tool to create documents from templates and dynamic data, there are many permutations to test. If the team is unfamiliar with the tool, the testers can ask the team to consider doing those stories in the first iteration of the release.

What’s in Scope?

Agile teams continually manage scope in order to meet business deadlines while preserving quality. High-value stories are the first priority. Stories that are “nice-to-haves” might be elbowed out of the release.

Lisa’s Story

Our team’s customers list their stories in priority order and then draw a line between the stories that must be done before the release can occur, and the ones that could safely be put off. They call the less important stories “below the line,” and those stories may never get done.

For example, when we undertook the theme to allow retirement plan participants to borrow money from their retirement accounts, there was a “below the line” story to send emails to any participants whose loans are changing status to “pending default” or “default.” When the loan is in “default” status, the borrower must pay taxes and penalties on the balance. The email would be extremely helpful to the borrowers, but it wasn’t as important to our business as the software to request, approve and distribute loans, or process loan payments.

The email story didn’t make it into the release. It wasn’t done until more than two years later, after enough complaints from people who didn’t know their loans were going into default until it was too late.

—Lisa

Janet worked with a team whose customers were under the misplaced assumption that all of the features would get into their release and that when they were prioritizing, they were just picking which stories got done first. When the rest of the team realized the misunderstanding, they also implemented the idea of stories above and below the line. It helped to track progress as well as make the stories that were dropped below the line very visible.

Deadlines and Timelines

Many domains revolve around fixed dates on the calendar. Retail businesses make most of their profit during the holiday season. An Internet retail site is smart to have all new features implemented by October 1. Implementing a new feature close to the peak buying period is risky. Lisa’s company’s customers must complete government-required tasks during certain periods of the year. When it’s too late for a feature to get released this year, it often gets put off for the next year, because more urgent priorities must be addressed. Regulatory changes have specific timelines, and organizations have no choice about the timeline.

Janet’s Story

While working on this book, I was planning a release with my team at WestJet. We had several possible stories and worked with the customers to decide what the release would look like. We had one regulatory change that was light work for the programmers, but heavy for the testers. It needed to be in production by a certain date, so the other stories we were considering for the release took that into consideration.

We decided to create a small maintenance release with just that one major feature, along with a few bugs from the backlog so the release of the regulatory change would not be jeopardized. While the testers completed their testing, the rest of the team started some of elaboration stories for the next release.

An alternative plan could have been that the programmers chip in and help test and fit in more features. However, the whole team decided that this plan would work the best with the least amount of risk.

—Janet

Focus on Value

It’s rather easy for a team to start discussing a complex story and lose sight of what value the features actually deliver. Release planning is the time to start asking for examples and use cases of how the features will be used, and what value they’ll provide. Drawing flowcharts or sample calculations on the whiteboard can help pinpoint the core functionality.

Lisa’s Story

Our product owner wrote a story to provide a warning if an employer overrides the date a participant becomes eligible to contribute to a retirement account after the participant has already made contributions.

The warning needed to be incorporated into the legacy UI code, which didn’t easily accommodate it. The team discussed how it might be implemented, but every option was fairly costly. Not only would coding be tricky, but a lot of time was needed to test it adequately and update existing automated tests. This feature wouldn’t provide much value to the business, just a bit of help to the end users. The release was already pretty close to the limit on features.

One of the programmers suggested providing a report of participants who met the criteria so the plan administrators could simply call the employers who may need to make corrections. The report story was much smaller than the warning story, could easily fit into the release, and was acceptable to the customer.

—Lisa

There is no guarantee that these initial “guesstimates” at what will be in a given release will hold up over time. That is why customers needs to understand their priorities, take checkpoints at the end of every iteration, and reevaluate the priorities of remaining stories.

System-Wide Impact

One of our jobs as testers is to keep the big picture in mind. The agile tester thinks about how each story might affect the system as a whole, or other systems that ours has to work with. For example, if the toy warehouse makes a change to its inventory software, and the new code has a bug that overstates the number of items in stock, the website might sell more of the hot new doll than there are available, disappointing thousands of children and their parents at Christmas. When risk is high, listing areas of the system that might be affected for a theme or group of stories might be a worthwhile exercise even during release planning.

We talked about the “ripple effects” in Chapter 8, “Business Facing Tests that Support the Team.”

Contact points between our system and that of partners or vendors always merit consideration. Even a minor change to a csv or

Figure 15-5 shows a simplified diagram of a new system that touches many pieces of the existing system. Different tools might be needed to test the integrations.

Figure 15-5 System impacts

Testers who have worked with some of the other systems or understand what testing needs to happen on those systems can offer valuable insight into the impact of a new story. Often, stories will need to be delayed until a future release if the impact has not been explored. This is a good time to recall previous releases that didn’t end so well.

Third-Party Involvement

Working with vendor tools, partners, or other contractor teams on a big project complicates release planning. If anyone outside your team is responsible for some part of the project, that’s one piece that’s out of your control. If you need to coordinate with others, including possible new users of the system, it’s best to start early.

Lisa’s team has written several interfaces to allow users to upload data to their systems. In each case, they had to get the proposed file format out to the users early to make sure it would work for them. Other projects involved sending data to partners or vendors. These required extra planning to arrange testing with their test systems and getting their feedback on whether data was valid and correctly formatted.