Crispin, Lisa - Agile Testing - A Practical Guide for Testers and Agile Teams

In Chapter 14, “An Agile Test Automation Strategy,” we’ll look at ways to organize automated tests.

Legacy Code

In our experience, it’s much easier to get traction on automation if you’re writing brand new code in an architecture designed with testing in mind. Writing tests for existing code that has few or no tests is a daunting task at best. It seems virtually impossible to a team new to agile and new to test automation.

It is sometimes a Catch-22. You want to automate tests so you can refactor some of the legacy code, but the legacy code isn’t designed for testability, so it is hard to automate tests even at the unit level.

If your team faces this type of challenge and doesn’t budget plenty of time to brainstorm about how to tackle it, it’ll be tough to start automating tests effectively. Chapter 14 gives strategies to address these issues.

Fear

Test automation is scary to those who’ve never mastered it, and even to some who have. Programmers may be good at writing production code, but they might not be very experienced at writing automated tests. Testers may not have a strong programming background, and they don’t trust their potential test automation skills.

Non-programming testers have often gotten the message that they have nothing to offer in the agile world. We believe otherwise. No individual tester should need to worry about how to do automation. It’s a team problem, and there are usually plenty of programmers on the team who can help. The trick is to embrace learning new ideas. Take one day at a time.

Old Habits

When iterations don’t proceed smoothly and the team can’t complete all of the programming and testing tasks by the end of an iteration, team members may panic. We’ve observed that when people go into panic mode, they fall into comfortable old habits, even if those habits never produced good results.

So we may say, “We are supposed to deliver on February 1. If we want to meet that date, we don’t have time to automate any tests. We’ll have to do whatever manual tests can be done in that amount of time and hope for the best. We can always automate the tests later.”

This is the road to perdition. Some manual tests can get done, but maybe not the important manual exploratory tests that would have found the bug that cost the company hundreds of thousands of dollars in lost sales. Then, because we didn’t finish with our test automation tasks, those tasks carry over to the next iteration, reducing the amount of business value we can deliver. As iterations proceed, the situation continues to deteriorate.

Can We Overcome These Barriers?

The agile whole-team approach is the foundation to overcoming automation challenges. Programmers who are new to agile are probably used to being rewarded for delivering code, whether it’s buggy or not, as long as they meet deadlines. Test-driven development is oriented more toward design than testing, so business-facing tests may still not enter their consciousness. It takes leadership and a team commitment to quality to get everyone thinking about how to write, use, and run both technology-facing and business-facing tests. Getting the whole team involved in test automation may be a cultural challenge.

See Chapter 3, “Cultural Challenges,” for some ideas on making changes to the team culture in order to facilitate agile practices.

In the next chapter, we show how to use agile values and principles to overcome some of the problems we’ve described in this chapter.

Summary

In this chapter, we analyzed some important factors related to test automation:

We need automation to provide a safety net, provide us with essential feedback, keep technical debt to a minimum, and help drive coding.

Fear, lack of knowledge, negative past experiences with automation, rapidly changing code, and legacy code are among the common barriers to automation.

Automating regression tests, running them in an automated build process, and fixing root causes of defects reduces technical debt and permits growth of solid code.

Automating regression tests and tedious manual tasks frees the team for more important work, such as exploratory testing.

Teams with automated tests and automated build processes enjoy a more stable velocity.

Without automated regression tests, manual regression testing will continue to grow in scope and eventually may simply be ignored.

Team culture and history may make it harder for programmers to prioritize automation of business-facing tests than coding new features. Using agile principles and values helps the whole team overcome barriers to test automation.

Chapter 14 An Agile Test Automation Strategy

As we explored each of the Agile Testing Quadrants in Part III, we gave examples of tools that can help those different testing efforts succeed. Many of those tools are for automating tests. As we described in the previous chapter, teams face plenty of obstacles in their quest for successful test automation. Better tools become available all the time, but the trick is to choose the right tools and learn to use them effectively. Test automation requires thoughtful investment and incremental improvement. In this chapter, we explain how you can apply agile values and principles to get traction in starting or improving your automation efforts.

An Agile Approach to Test Automation

Here you are, reading this chapter on how to get your test automation strategy working, maybe hoping for that silver bullet, or an answer to all your questions. We hate to disappoint you, but we need to tell you right up front, there is no silver bullet. There is no one answer that works for every team. Don’t lose heart though, because we have some ideas to help you get started.

First, we suggest approaching your automation problems as you would any problem. Define the problem you are trying to solve. To help you figure that out, we first talk about some basics of test automation and reintroduce some terms.

Automation Test Categories

In Part III, we introduced the Agile Testing Quadrants and talked about each quadrant and the purpose of the tests in each quadrant. In this section, we look at the quadrants in a different light. Let’s look carefully at the quadrants (see Figure 14-1).

Figure 14-1 Agile Testing Quadrants

You can see that we’ve labeled both quadrants that support the team (Q1 and Q2) as using automation. In Quadrant 4, the tools used for critiquing the product from a technology point of view also usually require automated tools. In Chapter 9, “Toolkit for Business-Facing Tests that Support the Team,” we discussed some of the tools that can be used for automating business-facing tests in the quest for supporting the team. In fact, the only quadrant that is not labeled as using automation is Quadrant 3—the business-facing tests that critique the product. However, as we discussed in Chapter 10, “Business-Facing Tests that Critique the Product,” tools may be useful for some of that testing. For example, automation can help set up test data and user scenarios, and analyze logged activity.