Crispin, Lisa - Agile Testing - A Practical Guide for Testers and Agile Teams

Invest in a test architecture that accommodates the complexity of the system under test. Plan to obtain necessary resources and expertise at the right time for specialized tests. For each type of test, your team should work together to choose tools that solve your testing problems. Use retrospectives to continually evaluate whether your team has the resources it needs to succeed, and whether all necessary tests are being specified in time to serve their purpose, and automated appropriately.

Does end-to-end testing seem impossible to do? Is your team finding it hard to write unit tests? As Janet’s team did, get everyone experimenting with different approaches and tools. The quadrants provide a framework for productive brainstorming on creative ways to achieve the testing that will let the team deliver value to the business.

Summary

In this chapter, we described a real project that used tests from all four agile testing quadrants to overcome difficult testing challenges. We used examples from this project to show how teams can succeed with all types of testing. Some important lessons from the Remote Data Monitoring System project are:

The whole team should choose or create tools that solve each testing problem.

Combinations of common business tools such as spreadsheets and custom-written test scripts may be needed to accomplish complex tests.

Invest time in building the right test architecture that works for all team members.

Find ways to keep customers involved in all types of testing, even if they’re in a remote location.

Report test results in a way that keeps all stakeholders informed about the iteration and project progress.

Don’t forget to document . . . but only what is useful.

Think about all four quadrants of testing throughout your development cycles.

Use lessons learned during testing to critique the product in order to drive development in subsequent iterations.

Part IV Automation

Test automation is a core agile practice. Agile projects depend on automation. Good-enough automation frees the team to deliver high-quality code frequently. It provides a framework that lets the team maximize its velocity while maintaining a high standard. Source code control, automated builds and test suites, deployment, monitoring, and a variety of scripts and tools eliminate tedium, ensure reliability, and allow the team to do its best work at all times.

Automation is also a vast topic. It includes tasks like writing simple shell scripts, setting up session properties, and creating robust automated tests. The range and number of automated tools seem to grow exponentially as we learn about better ways to produce software. Happily, the number of excellent books that teach ways to automate appears to grow just as fast.

This book is focused on the tester’s role in agile development. Because automation is key to successful agile development, we need to talk about it, but we can’t begin to cover every aspect of the subject. What we do want to explain is why you, as a tester, must embrace automation, and how you and your team can overcome the many obstacles that can hamper your automation efforts. This section describes how you can apply agile values, principles, and practices to grow a practical automation strategy, overcome barriers, and get traction on test automation.

Chapter 13 Why We Want to Automate Tests and What Holds Us Back

Why do we automate testing, the build process, deployment, and other tasks? Agile teams focus on always having working software, which enables them to release production-ready software as often as needed. Achieving this goal requires constant testing. In this chapter, we look at reasons we want to automate and the challenges that make it hard to get traction on automation.

Why Automate?

There are multiple reasons to automate besides our saying you need to have automation to be successful using agile. Our list includes the following:

Manual testing takes too long.

Manual processes are error prone.

Automation frees people to do their best work.

Automated regression tests provide a safety net.

Automated tests give feedback early and often.

Tests and examples that drive coding can do more.

Tests provide documentation.

Automation can be a good return on investment.

Let’s explore each of these in a little more detail.

Manual Testing Takes Too Long

The most basic reason a team wants to automate is that it simply takes too long to complete all of the necessary testing manually. As your application gets bigger and bigger, the time to test everything grows longer and longer, sometimes exponentially, depending on the complexity of the AUT (Application under test).

Agile teams are able to deliver production-ready software at the end of each short iteration by having production-ready software every day. Running a full suite of passing regression tests at least daily is an indispensable practice, and you can’t do it with manual regression testing. If you don’t have any automation now, you’ll have to regression test manually, but don’t let that stop you from starting to automate it.

If you execute your regression testing manually, it takes more and more time testing every day, every iteration. In order for testing to keep pace with coding, either the programmers have to take time to help with manual regression testing, or the team has to hire more testers. Inevitably, both technical debt and frustration will grow.

If the code doesn’t even have to pass a suite of automated unit level regression tests, the testers will probably spend much of their time researching, trying to reproduce and report those simple bugs, and less time finding potentially serious system level bugs. In addition, because the team isn’t doing test-first development, code design is more likely to be less testable and may not provide the functionality desired by the business.

Manually testing a number of different scenarios can take a lot of time, especially if you’re keying inputs into a user interface. Setting up data for a variety of complex scenarios can be an overwhelming task if you have no automated way to speed it up. As a result, only a limited number of scenarios may be tested, and important defects can be missed.