Crispin, Lisa - Agile Testing - A Practical Guide for Testers and Agile Teams

Lisa’s Story

Early on in our agile efforts, my team wasn’t fixing broken tests fast enough. I wrote “Tests are not temporary!” on the whiteboard to remind everyone that once a test passes, it needs to keep passing. A few days later, the words “but testers are!” had been added to get back at me. We did get much better at keeping our builds “green” after that.

—Lisa

One passing test leads to another. Keep your tests current and maintainable with refactoring. Extend them to cover other test cases. The various combinations and scenarios might or might not become part of the regression suite after they pass. We want our regression suite to run in a timely manner, and having too many tests for edge cases would slow it down.

Use Appropriate Test Design Patterns

When designing tests, look at different patterns and choose the ones that work for you. Keep them as simple as you can. Before you can design tests, you have to identify the ones you need. Pierre Veragen coined the term test genesis patterns to note the patterns that help you think of tests. Examples and use cases feed into our test genesis patterns.

Build/Operate/Check

Lisa’s team often goes with a build/operate/check pattern: Build the input data, in memory or actually in the database, depending on the purpose of the test; invoke the production code to operate on those inputs; and check the results of that operation. Some teams call this setup/execute/validate. For example, to test the invoice presented to a new account holder, set up the fees to be charged, input the properties of the account that relate to fee amounts, run the code that calculates the fees, and then check to see what fees were actually charged. See Figure 9-9 for an example of a test that sets up a loan with a specified amount, interest rate, term, payment frequency, and service start date and then checks the resulting amortization schedule. The test data is built in memory, which makes for a speedy test. A “teardown” fixture (not shown) removes the test data from memory so it won’t interfere with subsequent tests.

Figure 9-9 Example test with build/operate/check pattern

If there’s a need to test the application’s data access layer, tests can run using an actual database. Each test can insert the test data it needs, operate on it, check results, and delete the data. Testing with data in a real database can be a means of automating a test against legacy code whose data access and business logic layers aren’t easily separated.

Notice that the “check” table in the example uses a declarative style, with each row forming an independent test case, without changing the state of the system. Each row in our example tests a line in the loan amortization schedule. In the next section, we’ll look at patterns that are in a procedural style, with steps that change or test the state of the system.

Time-Based, Activity, and Event Patterns

Sometimes a timeline-based procedural pattern reflects the business better. For example, when testing a loan, we want to make sure interest and principal are applied correctly for each payment. The amount of interest depends on the date the payment was received and the date of the last payment processed. We want a test that simulates taking out a loan for a certain dollar amount, interest rate, and time period, and then over time simulates the borrower sending in payments, which are received and processed. Figure 9-10 shows a simple example of a FitLibrary “DoFixture” test that takes out a loan, checks the payment amount, posts the borrower’s payments, receives the payments and processes them, and then checks the interest, principal, and loan balance amount. It also checks the loan default state.

Figure 9-10 Sample time-based test

Depending on the domain, a time- or event-based approach might simulate the actual business processes better and be more understandable to business experts than a declarative type test. Other customers might find the declarative table style simpler to understand, because it hides the procedural details. Different patterns work best for different situations, so experiment with them.

Learning More

Your team should educate itself on test patterns that help drive programming. Finding the right pattern for each type of test ensures the test communicates clearly, is easy to maintain, and runs in an optimal amount of time. See the bibliography for more invaluable resources on test design, such as Gerard Meszaros’s xUnit Test Patterns: Refactoring Test Code .

Bring programmers and testers together to brainstorm test approaches and to help decide what tests can be automated and how the code should be designed to support testing. Business logic and algorithms should be accessible by test fixtures, without having to go through a user interface or batch scheduling process. This enables test-driven development, which in turn produces testable architecture.

A common approach to automating tests is by driving tests with keywords or action words. This can be used with tools such as Fit and FitNesse, or Ruby with Watir. We’ll explain this next.

Keyword and Data-Driven Tests

Data-driven testing is a tool that can help reduce test maintenance and enable you to share your test automation with manual testers. There are many times when you want to run the same test code over and over, repeating only the inputs and expected results. Spreadsheets or tables, such as those supported by Fit, are excellent ways to specify inputs. The test fixture, method, or script can loop through each data value one at a time, matching expected results to actual results. By using data-driven tests, you are actually using examples to show what the application is supposed to do.

Keyword-driven testing is another tool used in automated testing, where predefined keywords are used to define actions. These actions correspond to a process related to the application. It is the first step in creating a domain testing language. These keywords (or action words) represent a very simple specification language that non-programmers can use to develop automated tests. You still need programmers or technical automation specialists to implement the fixtures that the action words act on. If these keywords are extended to emulate the domain language, customers and nontechnical testers can specify tests that map to the workflow more easily.

The sample spreadsheet in Figure 9-11 shows how one company used action words to automate their test setup. The same action words can be used to test. The words Signup, Signoff, and CCDeposit are words that are domain-specific. Their users could easily write tests without understanding the underlying code.

Figure 9-11 Sample test spreadsheet with action words

Combining data-driven and keyword-driven testing techniques can be very powerful. Fit and FitNesse use both keywords and data to drive tests. The other tools we’ve described in this chapter can also accommodate this approach.

Any test strategy can run into trouble if the code isn’t designed to be easily tested. Let’s take a look at testability concerns.

Testability