Crispin, Lisa - Agile Testing - A Practical Guide for Testers and Agile Teams

The term test-driven development misleads practitioners who don’t understand that it’s more about design than testing. Code developed test-first is naturally designed for testability. Quadrant 1 activities are all aimed at producing software with the highest possible internal quality.

When teams practice TDD, they minimize the number of bugs that have to be caught later on. Most unit-level bugs are prevented by writing the test before the code. Thinking through the design by writing the unit test means the system is more likely to meet customer requirements. When post-development testing time is occupied with finding and fixing bugs that could have been detected by programmer tests, there’s no time to find the serious issues that might adversely affect the business. The more bugs that leak out of our coding process, the slower our delivery will be, and in the end, it is the quality that will suffer. That’s why the programmer tests in Quadrant 1 are so critical. While every team should adopt practices that work for its situation, a team without these core agile practices is unlikely to benefit much from agile values and principles.

Supporting Infrastructure

Solid source code control, configuration management, and continuous integration are essential to getting value from programmer tests that guide development. They enable the team to always know exactly what’s being tested. Continuous integration gives us a way to run tests every time new code is checked in. When a test fails, we know who checked in the change that caused the failure, and that person can quickly fix the problem. Continuous integration saves time and motivates each programmer to run the tests before checking in the new code. A continuous integration and build process delivers a deployable package of code for us to test.

Agile projects that lack these core agile practices tend to turn into “mini-waterfalls.” The development cycles are shorter, but code is still being thrown “over the wall” to testers who run out of time to test because the code is of poor quality. The term waterfall isn’t necessarily derogatory. We’ve worked on successful “waterfall” projects where the programmers stringently automate unit tests, practice continuous integration, and use automated builds to run tests. These successful “waterfall” projects also involve customers and testers throughout the development cycle. When we code without appropriate practices and tools, regardless of what we call the process, we’re not going to deliver high-quality code in a timely manner.

Why Write and Execute These Tests?

We’re not going into any details here about how to do TDD, or the best ways to write unit and component tests. There are several excellent books on those subjects. Our goal is to explain why these activities are important to agile testers. Let’s explore some reasons to use technology-facing tests that support the team.

Lets Us Go Faster and Do More

Speed should never be the end goal of an agile development team. Trying to do things fast and meet tight deadlines without thinking about the quality causes us to cut corners and revert to old, bad habits. If we cut corners, we’ll build up more technical debt, and probably miss the deadline anyway. Happily, though, speed is a long-term side effect of producing code with the highest possible internal quality. Continuous builds running unit tests notify the team of failure within a few minutes of the problem check-in, and the mistake can be found and fixed quickly. A safety net of automated unit and code integration tests enables the programmers to refactor frequently. This keeps the code at a reasonable standard of maintainability and delivers the best value for the time invested. Technical debt is kept as low as possible.

If you’ve worked as a tester on a project where unit testing was neglected, you know how easy it is to spend all of your time finding unit-level defects. You might find so many bugs while testing the “happy path” that you never have time to test more complex scenarios and edge cases. The release deadline is pushed back as the “find and fix” cycle drags on, or testing is just stopped and a buggy product is foisted off on unsuspecting customers.

Our years on agile teams have been Utopian in contrast to this scenario. Driving coding practices with tests means that the programmers probably understood the story’s requirements reasonably well. They’ve talked extensively with the customers and testers to clarify the desired behaviors. All parties understand the changes being made. By the time the team has completed all of the task cards for coding a story, or a thin, testable slice of one, the feature has been well covered by unit and component tests. Usually the programmers have made sure at least one path through the story works end to end.

This means that we, as testers, waste little time finding low-level bugs. We’re likely to try scenarios the programmers hadn’t thought of and to spend our time on higher-level business functionality. Well-designed code is usually robust and testable. If we find a defect, we show it to the programmer, who writes a unit test to reproduce the bug and then fixes it quickly. We actually have time to focus on exploratory testing and the other types of in-depth tests to give the code a good workout and learn more about how it should work. Often, the only “bugs” we find are requirements that everyone on our team missed or misunderstood. Even those are found quickly if the customer is involved and has regular demos and test opportunities. After a development team has mastered TDD, the focus for improvement shifts from bug prevention to figuring out better ways to elicit and capture requirements before coding.

Test-First Development vs. Test-Driven Development

Gerard Meszaros [Meszaros 2007, pp. 813–814] offers the following description of how test-first development differs from test-driven development:

“Unlike test-driven development, test-first development merely says that the tests are written before the production code; it does not imply that the production code is made to work one test at a time (emergent design). Test-first development can be applied at the unit test or customer test level, depending on which tests we have chosen to automate.”

Erik Bos [2008] observes that test-first development involves both test-first programming and test-first design, but there’s a subtle difference:

“With test-first design, the design follows the tests, whereas you can do test-first programming of a design that you first write down on a whiteboard. On larger projects, we tend to do more design via whiteboard discussions; the team discusses the architecture around a whiteboard, and codes test-first based on this design. On smaller projects, we do practice test-driven design.”

There are several different philosophies about when to write tests and for what purpose. It’s up to each team to agree on the approach that helps it achieve its quality objectives, although there is common agreement in the agile community that TDD definitely helps a team achieve better-quality software. This is an important way that programmer tests support the team. Let’s look at some more ways.

Making Testers’ Jobs Easier

The core practices related to programmer tests make lots of testing activities easier to accomplish. Programmers work in their own sandboxes, where they can test new code without affecting anyone else’s work. They don’t check in code until it has passed a suite of regression tests in their sandbox.

The team thinks about test environments and what to use for test data. Unit tests usually work with fake or mock objects instead of actual databases for speed, but programmers still need to test against realistic data. Testers can help them identify good test data. If the unit tests represent real-life data, fewer issues will be found later.