Crispin, Lisa - Agile Testing - A Practical Guide for Testers and Agile Teams

You don’t have to be an expert on how to do technology-facing testing that critiques the product to help your team plan for it and execute it. Your team can evaluate what tests it needs from this quadrant. Talk about these tests as you plan your release; you can create a test plan specifically for performance and load if you’ve not done it before. You will need time to obtain the expertise needed, either by acquiring it through identifying and learning the skills, or by bringing in outside help. As with all development efforts, break technology-facing tests into small tasks that can be addressed and built upon each iteration.

Summary

In this chapter, we’ve explored the fourth agile testing quadrant, the technology-facing tests that critique the product.

The developer team should evaluate whether it has, or can acquire, the expertise to do these tests, or if it needs to plan to bring in external resources.

An incremental approach to these tests, completing tasks in each iteration, ensures time to address any issues that arise and avoid production problems.

The team should consider various types of “ility” testing, including security, maintainability, interoperability, compatibility, reliability, and installability testing, and should execute these tests at appropriate times.

Performance, scalability, stress, and load testing should be done from the beginning of the project.

Research the memory management issues that might impact your product, and plan tests to verify the application is free of memory issues.

Chapter 12 Summary of Testing Quadrants

In Chapter 6, we introduced the testing quadrants, and in the chapters that followed we talked about how to use the concepts in your agile project. In this chapter, we’ll bring it all together with an example of an agile team that used tests from all four quadrants.

Review of the Testing Quadrants

We’ve just spent five chapters talking about each of the quadrants (see Figure 12-1) and examples of tools you can use for the different types of testing. The next trick is to know which tests your project needs and when to do them. In this chapter, we’ll walk you through a real-life example of an agile project that used tests from all four agile testing quadrants.

Figure 12-1 Agile Testing Quadrants

A System Test Example

The following story is about one organization’s success in testing its whole system using a variety of home-grown and open source tools. Janet worked with this team, and Paul Rogers was the primary test architect. This is Paul’s story.

The Application

The system solves the problem of monitoring remote oil and gas production wells. The solution combines a remote monitoring device that can transmit data and receive adjustments from a central monitoring station using a proprietary protocol over a satellite communication channel.

Figure 12-2 shows the architecture of the Remote Data Monitoring system. The measurement devices on the oil wells, Remote Terminal Units (RTU), use a variety of protocols to communicate with the measurement device. This data from each RTU is transmitted via satellite to servers located at the client’s main office. It is then made available to users via a web interface. A notification system, via email, fax, or phone, is available when a particular reading is outside of normal operational limits. A Java Message Service (JMS) feed and web services are also available to help integration with clients’ other applications.

Figure 12-2 Remote data monitoring system architecture

The software application was a huge legacy system that had few unit tests. The team was slowly rebuilding the application with new technology.

The Team and the Process

The team consisted of four software programmers, two firmware programmers, three to four testers, a product engineer, and an off-site manager. The “real” customer was in another country. The development team uses XP practices, including pair programming and TDD. The customer team used the defect-tracking system for the backlog, but most of the visibility of the stories was through index cards. Story cards were used during iteration planning meetings, and the task board tracked the progress.

Scrum was used as the outside reporting mechanism to the organization and the customers. The team had two week iterations and released the product about every four months. This varied depending on the functionality being developed. Retrospectives were held as part of every iteration planning session, and action was taken on the top three priority items discussed.

Continuous integration through CruiseControl provided constant builds for the testers and the demonstrations held at the end of every iteration. Each tester had a local environment for testing the web application, but there were three test environments available to the system. The first one was to test new stories and was updated as needed with the latest build. The second one was for testing client-reported issues, because it had the last version released to the clients. The third environment was a full stand-alone test environment that was available for testing full deploys, communication links, and the firmware and hardware. It was on this environment that we ran our load and reliability tests.

Tests Driving Development

The tests driving development included unit test and acceptance tests.

Unit Tests

Unit tests are technology-facing tests that support programming. Those that are developed as part of test-driven development not only help the programmer get the story right but also help to design the system.

Chapter 7, “Technology-Facing Tests that Support the Team,” explains more about unit testing and TDD.

The programmers on the Remote Data Monitoring project bought into Test Driven Development (TDD) and pair programming wholeheartedly. All new functionality was developed and tested using pair programming. All stories delivered to the testers were supported by unit tests, and very few bugs were found after coding was complete. The bugs that were found were generally integration-related.

However, when the team first started, the legacy system had few unit tests to support refactoring. As process changes were implemented, the developers decided to start fixing the problem. Every time they touched a piece of code in the legacy system, they added unit tests and refactored the code as necessary. Gradually, the legacy system became more stable and was able to withstand major refactoring when it was needed. We experienced the power of unit tests!

Acceptance Tests

The product engineer (the customer proxy) took ownership of creating the acceptance tests. These tests varied in format depending on the actual story. Although he struggled at first, the product engineer got pretty good at giving the tests to the programmers before they started coding. The team created a test template, which evolved over time, that met both the programmers’ and the testers’ needs.