Crispin, Lisa - Agile Testing - A Practical Guide for Testers and Agile Teams

More tools that are geared for direct use by product owners and business experts are becoming available, and many teams develop their own. Tools such as Fit (Framework for Integrated Tests) and FitNesse were designed to facilitate collaboration and communication between the customer and development teams. We’re hearing about more teams where the customers actually write the tests in a tool such as those.

Notes from a Distributed Team

Pierre Veragen and Erika Boyer of iLevel by Weyerhaeuser told us that every iteration begins with everyone on the team writing acceptance tests. That’s how they start their iteration planning. Most interesting is the fact that their product owners, who are mechanical engineers, write FitNesse tests themselves. Pierre explains that an advantage of a tool such as FitNesse is the ability to use their own domain language in the FitNesse tests. It doesn’t matter what they end up choosing as a UI. They can test all of their complex calculations in the tests.

With this process, tests can be written before writing the testing code or the system under test. It’s true test-driven development. Behavior changes and bug fixes can follow.

Some teams build their own frameworks that allow customers, business analysts, and testers to document examples that can be directly turned into executable tests. These are often based on open source tools such as xUnit, Fit, Selenium, and Watir. We like this approach, because it saves time and resources. When you’re delivering production-ready code in short iterations, you need a streamlined process.

Online forum tools are a good alternative to email conversations for ongoing discussions about features or technical concerns, especially for teams that don’t all sit together. Emails often get missed or lost, people have to remember to choose “Reply all,” and it can be hard to put together the details of the discussion later. Lisa’s team uses an online forum to elicit opinions about different tools, propose different behavior for features, and conduct philosophical discussions such as whether to track defects.

Finding the right electronic tools is particularly vital for distributed teams. Instant messaging, the telephone, VoIP, and Skype help us communicate, but they lack the visual component. Some global teams ask their members to meet at nonstandard hours so that they can have real-time conversations, but frameworks for written and visual communication are still critical.

Wikis are a common tool used to enhance communication and record discussions and decisions. Wikis enable users to edit web page content in a web browser. Users can add hyperlinks and easily create new pages. You can upload mock-ups, samples, and pictures of whiteboard drawings and make them easily visible on Wiki pages. The hierarchical organization can get tricky to maintain, but there are lots of open source and vendor wiki software packages available that make managing your knowledgebase and sharing information easier to administer. If your wiki knowledgebase has grown to the point where it’s hard to find anything, hire a technical writer to transform it into organized, usable documentation.

Open source and commercial tools provide ways to let teams collaborate on requirements and test cases online. We can’t emphasize enough the need for you to identify tools that might be helpful, to experiment with them for a few iterations, and to decide how well they work for you. Your team’s needs will change with time, so always be open to trying new techniques and frameworks.

These tools help create the conversation about the story. With these techniques, and as much real-time conversation and visual sharing as we can manage, we can define the right product from the get-go.

Tools for Automating Tests Based on Examples

What about test tools? We like the collaboration inherent with tools such as Fit and FitNesse. However, in our opinion, any tool that gets testers and programmers, programmers and customers, and testers and customers talking is a great one. We know teams where customers actually write tests in Fit, FitNesse, Expect, or other tools. This works when the tool has been set up in a manner that’s clear to everyone writing tests, with the domain language easy to understand and the appropriate fixtures provided.

Tools to Test below the GUI and API Level

There are a multitude of open source tools that enable you to test below the GUI or at the API layer. We are listing just a few, but your team will need to determine the right tool for you.

Unit-Level Test Tools

Some teams use the xUnit tools such as JUnit or NUnit for business-facing tests as well as technology-facing tests. If the testers and customers are comfortable with these tools, and they provide for all of the functional testing behind the GUI needed, they’re fine. To make these tools more customer-friendly, teams might build a framework on top of the unit-level tools that testers and customers can use to specify tests.

Janet has worked on a couple of applications like that. One was a message handling system that was being deployed in an organization. The programmers used JUnit for all of the component and integration testing. They built a load test framework that could make use of the JUnit tests, so no other testing tools were needed. The GUI front end was so small that Janet was able to test it manually. It made no sense to automate the GUI testing in this case.

Behavior-driven development (BDD) tools are also suited to this purpose, because they use a more natural language for specifying the tests. Behavior-driven development is a variation of test-driven development, pioneered by Dan North [2006], and evolved by many others. It’s related to domain-driven design, with a focus on the domain rather than on the technology, and driving design with a model. Instead of the word “test” or “assert,” BDD uses the word “should.” By thinking in terms of behavior, it’s natural to write specifications ahead of code. Test specifications use a domain-specific language to provide tests that customers can read but that can also be easily automated.

Some of the many BDD tools available as of this writing include easyb and JBehave for the Java platform, NBehave and NSpec for .NET, and RSpec for Ruby. These tools, like the XUnit tools, are intended for use by programmers to guide coding, but they can also be used to express business-facing tests that drive development, involving customers more closely in the development process.

Behavior-Driven Development

Andrew Glover, president of Stelligent Incorporated and author of books including Continuous Integration and Java Testing Patterns , explains the thinking behind one of the BDD tools, easyb.

assertEquals(42.50, order.price(), 0.0). Without examining the context in which this statement appears, this code is somewhat incomprehensible. Now imagine you don’t even read code—that is, you are a stakeholder asking (actually paying ) for new features. The previous code statement might as well be Farsi (assuming you can’t actually read Farsi!).

order.price().shouldBe 42.50. While the context in which this statement appears is still absent, this line of code is a bit more coherent. In fact, it reads like a normal sentence (and this time knowledge of Farsi isn’t required!). Stakeholders, in this case, could understand this code if they chose to read it; on top of that, it turns out that this line of code essentially matches what they asked for in the first place. This line of code describes behavior in a more literal manner too—the code uses a normal everyday phrase like shouldBe, which is distinctly different than the previously written assertEquals.