Laura Bright, McAfee
Measuring code coverage is a popular way to ensure that software is being adequately tested and gives software teams confidence in the quality of their product. In particular, decision coverage of both automated and manual tests is a good way to ensure that as many paths as possible are covered in testing and the team is not missing possible defects. Management likes to report high decision coverage numbers as an indicator of product quality, and many QA teams set a goal of reaching a target percentage as a way of demonstrating that the code has been thoroughly tested.
But what does high decision coverage mean in terms of overall product quality? Clearly it is good to cover as many decisions as reasonably possible. A good set of functional test cases will cover the “low-hanging fruit”, but may leave many uncovered decisions. Some of these uncovered decisions may be difficult or impossible to cover under normal testing conditions, but many can be covered by adding automated functional tests, unit tests, and manual tests. Hard to reach “corner cases” can be covered by setting up special testing environments. Some of these new tests may cover important use cases while others may provide little benefit. The question this paper aims to answer is, how do you determine where to invest your efforts to increase decision coverage?
This paper presents strategies to increase your decision coverage by targeting areas of the code with the highest ROI. Rather than focus on reaching a target percentage, we show how you can set realistic decision coverage goals that will improve your overall software quality. We present specific examples of how this approach has benefitted our team, including:
- Identifying automated functional test cases that were overlooked by QA
- Removing dead or obsolete code
- Finding product defects that cause some code to not get executed
- Identifying manual test cases to cover parts of code that are difficult to test through automation
- Determining areas of the code that would benefit from increased unit testing
Through these efforts, our team has increased our percentage decision coverage, but more importantly, has improved overall quality by finding defects that otherwise may have been missed.
Laura Bright, 2014 Technical Paper, Paper, Slides
