Crispin, Lisa - Agile Testing - A Practical Guide for Testers and Agile Teams

Bob Galen, an agile coach and end-game expert, observes that agile development may not have seeped into every organizational nook and cranny. He notes, “Agile testers can serve as a conduit or facilitator when it comes to physical delivery of the software.”

Planning Enough Time for Testing

Because testing and coding are part of one process in agile development, we’d prefer not to make special plans for extra testing time, but in real life we might need some extra time.

Most teams accumulate some technical debt, despite the best intentions, especially if they’re working with legacy code. To maintain velocity, your team may need to plan a refactoring iteration at regular intervals to add tests, upgrade tools, and reduce technical debt. Lisa’s team conducts a refactoring sprint about every six months. While the business doesn’t usually receive any direct benefits at the end of a refactoring sprint, the business experts understand that these special sprints result in better test coverage, a solid base for future development, reduced technical debt, and a higher overall team velocity.

Some teams resort to “hardening” iterations, where they spend time only finding and fixing bugs, and they don’t introduce any new functionality. This is a last resort for keeping the application and its infrastructure solid. New teams may need an extra iteration to complete testing tasks, and if so, they budget time for that in the release plan.

Use retrospectives and other process improvement practices to learn ways to integrate testing and coding so that the code produced in each iteration is production-ready. When that goal is achieved, work to ensure that a stable build that could be released to production is available every day. Lisa’s team members thought that this was an unattainable goal in the days when they struggled to get any stable build before release, but it was only a couple of years before almost every build was release-worthy.

When your build is stable, you are ready to enter the “End Game.”

The End Game

What is the end game? We’ve heard people call the time right before delivery many things, but the “end game” seems to fit best. It’s the time when the team applies the finishing touches to the product. You’re dotting your i’s and crossing your t’s. It’s the last stretch before the delivery finish line. It’s not meant to be a bug-fix cycle, because you shouldn’t have any outstanding bugs by then, but that doesn’t mean you might not have one or two to fix.

You might have groups in your organization that you didn’t involve in your earlier planning. Now it’s time to work closely with the folks that administer the staging and production environments, the configuration managers, the database administrators outside of your team, and everyone who plays a role in moving the software from development to staging and production. If you weren’t working with them early this time, consider talking to these folks during your next release planning sessions, and keep in touch with them throughout the development cycle.

Bob Galen tells us that the testers on his team have partnered with the operations group that manages the staging and production environments. Because the operations group is remote, it finds that having guidance from the agile team is particularly valuable.

There are always system-level tests that can’t be automated, or are not worth automating. More often than not, your staging environment is the only place where you can do some system-level integration tests or system-level load and stress testing. We suggest that you allot some time after development for these types of finishing tasks. Don’t code right up to the end.

Plan as much time for the end game as you need. Janet has found that the length of time needed for the end game varies with the maturity of the team and the size of the application. It may be that only one day is needed to finish the extra tasks, but it may be one week or sometimes as much as a whole two-week iteration. The team from the example used in Chapter 12, “Summary of Testing Quadrants,” scheduled two weeks, because it was a complex system that required a fair bit of setup and system testing.

Lisa’s Story

When I worked on a team developing applications for a client, we had to follow the client’s release schedule. Testing with other parts of the larger system was only possible during certain two-week windows, every six or eight weeks. Our team completed two or three iterations, finishing all of the stories for each as if they were releasing each iteration.

Then we entered a testing window where we could coordinate system testing with other development teams, assist the client with UAT, and plan the actual release. This constituted our end game.

—Lisa

If you have a large organization, you might have ten or fifteen teams developing software for individual products or for separate areas of functionality for the same application. These areas or products may all need to release together, so an integrated end game is necessary. This does not mean that you leave the integration until the very end. Coordination with the other teams will be critical all along your development cycle, and if you have a test integration system, we recommend that you be sure that you have tried to integrate long before the end game.

You also may have considerations beyond your team, for example, working with software delivered by external teams at the enterprise level.

Use this end-game time to do some final exploratory testing. Step back and look at the whole system and do some end-to-end scenarios. Such testing will confirm that the application is working correctly, give you added confidence in the product, and provide information for the next iteration or release.

Testing the Release Candidate

We recommend that the automated regression testing be done against every release candidate. If you’re following our recommendation to run automated regression tests continually on each new build, or at least daily, you’ve already done this. If some of your regression tests are manual, you’ll need to plan time for those or they might not get done. A risk assessment based on changes made to each build will determine what tests need to be run if there is more than one release candidate.

Test on a Staging Environment

Whether you are using traditional or agile development processes, a staging environment that mimics production is vital for final testing before release, as well as for testing the release process itself. As part of the end game, your application should be deployed to staging just like you would deploy it to production, or as your customers would on their environments. In many organizations that Janet has seen, the staging environment is usually shared among multiple projects, and the deployment must be scheduled as part of the release planning. Consider ahead of time how to handle dependencies, integrating with other teams using the staging environment, and working with external third parties. It might feel like “traditional” test planning, but you might be dealing with teams that haven’t embraced agile development.

Although agile promotes continuous integration, it is often difficult to integrate with third-party products or other applications outside your project’s control. Staging environments can have better controls so that external applications may connect and have access to third-party test environments. Staging environments can also be used for load and performance testing, mock deploys, fail-over testing, and manual regression tests and exploratory functional testing. There are always configuration differences between environments so your staging environment is a good place to test these.