Crispin, Lisa - Agile Testing - A Practical Guide for Testers and Agile Teams

Security Testing Perspectives

Security testing is a vast topic on its own. Grig Gheorghiu shares some highlights about resources that can help agile teams with security testing.

Just like functional testing, security testing can be done from two perspectives: from the inside out (white-box testing) and from the outside in (black-box testing). Inside-out security testing assumes that the source code for the application under test is available to the testers. The code can be analyzed statically with a variety of tools that try to discover common coding errors that can make the application vulnerable to attacks such as buffer overflows or format string attacks.

See http://en.wikipedia.org/wiki/Buffer_overflow and http://en.wikipedia.org/wiki/Format_string_vulnerabilities for more information.

See http://en.wikipedia.org/wiki/List_of_tools_for_static_code_analysis for a list of tools that can be used for static code analysis.

The fact that the testers have access to the source code of the application also means that they can map what some books call “the attack surface” of the application, which is the list of all of the inputs and resources used by the program under test. Armed with a knowledge of the attack surface, testers can then apply a variety of techniques that attempt to break the security of the application. A very effective class of such techniques is called fuzzing and is based on fault injection. Using this technique, the testers try to make the application fail by feeding it various types of inputs (hence the term fault injection ). These inputs can be carefully crafted strings used in SQL injection attacks, random byte changes in given input files, or random strings fed as command line arguments.

More resources on this subject can be found at: www.fuzzing.org/category/fuzzing-book/ and www.fuzzing.org/fuzzing-software

The outside-in approach is the one mostly used by attackers who try to penetrate into the servers or the network hosting your application. As a security tester, you need to have the same mind-set that attackers do, which means that you have to use your creativity in discovering and exploiting vulnerabilities in your own application. You also need to stay up-to-date with the latest security news and updates related to the platform/operating system your application runs on, which is not an easy task.

So what are agile testers to do when faced with the apparently insurmountable task of testing the security of their application? Here are some practical, pragmatic steps that anybody can follow:

1.Adopt a continuous integration (CI) process that periodically runs a suite of automated tests against your application.

2.Learn how to use one or more open source static code analysis tools. Add a step to your CI process that consists of running these tools against your application code. Mark the step as failed if the tools find any critical vulnerabilities.

3.Install an automated security vulnerability scanner such as Nessus (http://www.nessus.org/nessus/). Nessus can be run in a command-line, non-GUI mode, which makes it suitable for inclusion in a CI tool. Add a step to your CI process that consists of running Nessus against your application. Capture the Nessus output in a file and parse that file for any high-importance security holes found by the scanner. Mark the step as failed when any such holes are found.

4.Learn how to use one or more open source fuzzing tools. Add a step to your CI process that consists of running these tools against your application code. Mark the step as failed if the tools find any critical vulnerabilities.

As with any automated testing effort, running these tools is no guarantee that your code and your application will be free of security defects. However, running these tools will go a long way toward improving the quality of your application in terms of security. As always, the 80/20 rule applies. These tools will probably find the 80% most common security bugs out there while requiring 20% of your security budget.

To find the remaining 20% of the security defects, you’re well advised to spend the other 80% of your security budget on high-quality security experts. They will be able to test your application security thoroughly by the use of techniques such as SQL injection, code injection, remote code inclusion, and cross-site scripting. While there are some tools that try to automate some of these techniques, they are no match for a trained professional who takes the time to understand the inner workings of your application in order to craft the perfect attack against it.

Security testing can be intimidating, so budget time to adopt a hacker mind-set and decide on the right approach to the task at hand. Use the resources Grig suggests to educate yourself. Take advantage of these tools and techniques in order to achieve security tests with a reasonable return on investment.

Just this brief look at security testing shows why specialized training and tools are so important to do a good job of it. For most organizations, this testing is absolutely required. One security intrusion might be enough to take a company out of business. Even if the probability were low, the stakes are too high to put off these tests.

Code that costs a lot to maintain might not kill an organization’s profitability as quickly as a security breach, but it could lead to a long, slow death. In the next section we consider ways to verify maintainability.

Maintainability

Maintainability is not something that is easy to test. In traditional projects, it’s often done by the use of full code reviews or inspections. Agile teams often use pair programming, which has built-in continual code review. There are other ways to make sure the code and tests stay maintainable.

We encourage development teams to develop standards and guidelines that they follow for application code, the test frameworks, and the tests themselves. Teams that develop their own standards, rather than having them set by some other independent team, will be more likely to follow them because they make sense to them.

The kinds of standards we mean include naming conventions for method names or test names. All guidelines should be simple to follow and make maintainability easier. Examples are: “Success is always zero and failure must be a negative value,” “Each class or module should have only one single responsibility,” or “All functions must be single entry, single exit.”

Standards for developing the GUI also make the application more testable and maintainable, because testers know what to expect and don’t need to wonder whether a behavior is right or wrong. It also adds to testability if you are automating tests from the GUI. Simple standards such as, “Use names for all GUI objects rather than defaulting to the computer assigned identifier” or “You cannot have two fields with the same name on a page” help the team achieve a level where the code is maintainable, as are the automated tests that provide coverage for it.

Maintainable code supports shared code ownership. It is much easier for a programmer to move from one area to another if all code is written in the same style and easily understood by everyone on the team. Complexity adds risk and also makes code harder to understand. The XP value of simplicity should be applied to code. Simple coding standards can also include guidelines such as, “Avoid duplication—Don’t copy-paste methods.” These same concepts apply to test frameworks and the tests themselves.

Maintainability is an important factor for automated tests as well. Test tools have lagged behind programming tools in features that make them easy to maintain, such as IDE plug-ins to make writing and maintaining test scripts simpler and more efficient. That’s changing fast, so look for tools that provide easy refactoring and search-and-replace, and for other utilities that make it easy to modify the scripts.