Crispin, Lisa - Agile Testing - A Practical Guide for Testers and Agile Teams

Lisa’s team decided on a different but equally effective approach. The team members started “strangling” the legacy code by writing all new features in a new test-friendly architecture. They’re gradually replacing all of the old code with code written test-first. When they do work on old code to fix bugs, or in the cases where the old code needs updating, they simply add unit tests for all of the code they change. A GUI smoke test suite covers the critical functions of the rest of the legacy system that has no unit tests.

Chapter 7, “Technology-Facing Tests that Support the Team,” goes into more detail about different agile approaches to legacy code.

As with any automation project, approach the hard-to-automate code one piece at a time, and address the highest risk areas first. Solve the testability problem and find a way to write unit-level tests. The effort will pay off.

Developing an Automation Strategy—Where Do We Start?

A simple, step-by-step approach sounds incompatible with an automation strategy, but in agile testing we try to understand the problem first. Deciding where and how to start with automation requires a bit of thought and discussion. As your team looks at testing challenges, you’ll need to consider where automation is appropriate. Before you start searching for a particular automation tool, you’ll want to identify your requirements.

You need to understand what problem you are trying to solve. What are you trying to automate? For example, if you have no test automation of any kind, and you start by buying an expensive commercial test tool thinking it will automate all your functional tests, you may be starting in the wrong place.

We suggest you start at the beginning. Look for your biggest gain. The biggest bang for the buck is definitely the unit tests that the programmers can do. Instead of starting at the top of the test pyramid, you may want to start at the bottom, making sure that the basics are in place. You also need to consider the different types of tests you need to automate, and when you’ll need to have tools ready to use.

In this section, we assume you have automated Quadrant 1 unit and component tests in place, and are looking to automate your business-facing tests in Quadrants 2 and 3, or your Quadrant 4 technology-facing tests that critique the product. We’ll help you design a good strategy for building your automation resources.

Think about the skills and experience on your team. Who needs the automation, and why? What goals are you trying to achieve? Understanding some of these issues may affect your choice of tools and what effort you expend. There is a section on evaluating tools at the end of this chapter.

Automation is scary, especially if you’re starting from scratch, so where do we begin?

Where Does It Hurt the Most?

To figure out where to focus your automation efforts next, ask your team, “What’s the greatest area of pain?” or, for some teams, “What’s the greatest area of boredom?” Can you even get code deployed in order to test it? Do team members feel confident about changing the code, or do they lack any safety net of automated tests? Maybe your team members are more advanced, have mastered TDD, and have a full suite of unit tests. But they don’t have a good framework for specifying business-facing tests, or can’t quite get a handle on automating them. Perhaps you do have some GUI tests, but they’re extremely slow and are costing a lot to maintain.

Peril: Trying to Test Everything Manually

If you’re spending all your time retesting features that you’ve tested before, not getting to new features, and needing to add more and more testing, you’re suffering from a severe lack of test automation. This peril means that testers don’t have time to participate in design and implementation discussions, regression bugs may creep in unnoticed, testing can’t keep up anymore with development, and testers get stuck in a rut. Developers aren’t getting involved in the business-facing testing, and testers don’t have time to figure out a better way to solve the testing problems.

Your team can fix this by developing an automation strategy, as we describe in this chapter. The team starts designing for testability and chooses and implements appropriate automation tools. Testers get an opportunity to develop their technical skills.

Wherever it hurts the most, that’s the place to start your automation efforts. For example, if your team is struggling to even deliver deployable code, you need to implement an automated build process. Nothing’s worse than twiddling your thumbs while you wait for some code to test.

But, if performance puts the existence of your organization in danger, performance testing has to be the top priority. It’s back to understanding what problem you are trying to solve. Risk analysis is your friend here.

Chapter 18, “Coding and Testing,” has more information on a simple approach to risk analysis.

Janet’s Story

I worked on a legacy system that was trying to address some quality issues as well as add new features for our main customer. There were no automated unit or functional tests for the existing application, but we needed to refactor the code to address the quality issues. The team members decided to tackle it one piece at a time. As they chose a chunk of functionality to refactor, the programmers wrote unit tests, made sure they passed, and then rewrote the code until the tests passed again. At the end of the refactoring, they had testable, well-written code and the tests to go with them. The testers wrote the higher-level functional tests at the same time. Within a year, most of the poor-quality legacy code had been rewritten, and the team had achieved good test coverage just by tackling one chunk at a time.

—Janet

Test automation won’t pay off unless other good development practices are in place. Continuous integration running a robust suite of unit tests is a first step toward automating other tests. Code that’s continually refactored for maintainability and good design will help increase the ROI on automation. Refactoring can’t happen without that good unit test coverage. These development practices also need to be applied to the automated functional test scripts.

Multi-Layered Approach

While we recommend mastering one tool at a time, don’t expect too much out of any one tool. Use the right tool for each need. The tool that works best for unit tests may or may not be appropriate to automate functional tests. GUI, load, performance, and security testing may each require a different tool or tools.

Mike Cohn’s test pyramid concept (see Figure 14-2) has helped our teams put their automation efforts where they do the most good. We want to maximize the tests that have the best ROI. If the system architecture is designed for testability, test automation will be less expensive, especially at the unit level.

Tests that go through the user interface usually have the lowest ROI, because they’re expensive to maintain, but we do need them. They make up the small tip of our pyramid We may choose to automate some of these tests, but the majority of GUI tests are defined in business terms and probably are best left as human interaction tests (i.e., manual tests).

The middle layer represents the functional tests that work directly with production code, without a GUI or other layer in between. While they’re not as inexpensive to automate as unit-level tests, and provide feedback a bit more slowly, the right tools allow them to have a good ROI. The fact that these tests can be written in a language the business experts understand adds to their value.

There are many different layers in the application that can be tested independently. In his book xUnit Test Patterns [2007], Gerard Meszaros refers to this as the Layer Test pattern. He cautions that when trying to test all of the layers of the application separately, we still have to verify that the layers are hooked up correctly, and this may require at least one test of the business logic through the presentation layer.