Improving the Quality of Your Tests Through Page Coverage

Programmers often write tests and have a lot of powerful frameworks and tools at their disposal. One technique, Code coverage, provides you with the actual lines of code that are executed when the tests are run. This information is usually condensed into a percentage number. The ideal amount of code coverage has been debated for a long time and I guess there is no right answer. I feel that code coverage, if used correctly, is one very useful technique to help determine the quality of the tests and ultimately the quality of the software being tested. What I find most interesting is the actual lines of code that have been covered or to be more precise the lines of code (or branches) that have not been covered. A quick look at the code coverage source code report often reveals if any important test cases are missing. I think that is valuable information.

Take a look at the Wikipedia article on code coverage if you are interested.

Based on the technology used, code coverage tools are available even for web applications. These tools would help the programmers creating them. Testers usually would not find these tools very interesting. Therein comes Page coverage. Page coverage in software testing does not seem to be standard terminology yet. My understanding of Page coverage is that it is a visual representation of the web page showing which parts of the page are touched by the tests. It is not a silver bullet, but it should help to identify if any test cases have been missed. It can also keep managers happy. I have not seen any open source tool to provide you with page coverage, though I have heard a few people are working on it. If you use Selenium 2 (Web Driver), you should have the basic building blocks required to build such a tool yourself. David Burns gave a lightning talk on the subject and mentioned that he has a working proof of concept.

Selenium IDE users are often left out of many cool tools. The good news is that I have been working on a Page Coverage plugin for Selenium IDE and it should be available any day now.

If you like this post, you might find my previous blog posts under testing techniques useful.