Michael Huber, Steven Loucks, Andrea Pound, Deian Tabakov, Schlumberger Information Solutions
Designing tests and determining their overall coverage can be a tough challenge for test engineers. When performing manual tests, engineers usually do not have sufficient visibility into the features that have already been covered by unit tests. Test engineers also have no means to confidently measure the amount of code that has been executed by their own testing efforts, neither for an individual function nor for the software application as a whole.
To improve test design, code quality and collaboration, a software development team needs to have a common means of evaluating and discussing test completeness. Although software developers have long used code coverage measurements to understand, communicate and increase the scope of their unit tests, manual testers still tend to be locked firmly into a world of scripted tests in which metrics are typically based on simply counting test cases. Little to no analysis is performed on the true coverage of these manual tests. This poster presentation introduces a process that encourages and enables all members of the software development team (developers, architects, manual testers and test automation engineers) to collect code coverage data consistently during their respective test activities.
The coverage data is compiled into a single file that visually illustrates the areas of the code that are covered by at least one testing approach, as well as the areas that remain completely uncovered. The software development team analyzes and discusses this information at regular intervals during the development cycle. With all players at the same table, this approach results in a healthier overall view of the testing strategy and provides clearer insight into how each member of the software development team can contribute to a higher quality software product. In addition to providing more accurate coverage metrics, this process results in the development of a spirit of joint ownership in relation to code quality, better communication and collaboration between the groups and skill development for all members of the team, especially the manual testers.
A case study of a real software development project in which this workflow was successfully implemented will be presented. The technical realization of this process in Visual Studio will be explained in detail so that conference participants will be able to easily implement this approach in their organizations. A live demo is also planned, making the presentation a practical, hands-on experience for the audience.
Michael Huber, Steven Loucks, Andrea Pound, Deian Tabakov, Schlumberger Information Solutions, 2011 Poster Paper, Abstract
