Jeffrey Weston, Microsoft
Microsoft Office for Mac has a very efficient developer process for porting code across platforms (from Windows to Mac). Unfortunately for testers, it creates many new bugs across our entire suite of applications, requiring a large amount of resources and time to test and find them all. Because the bugs appear at random each time we port code, test cost reductions are incredibly difficult.
To overcome this problem, we investigated a sample size of bugs to find the root cause of the defect. We identified the exact code change that introduced the defect, collected data on the process the developer was following and on the code context in which the bug existed. Categorizing the bugs by common attributes produced bug patterns that we could then act upon.
This technique provided valuable information that allowed us to extinguish (or at least severely limit) an entire category of defects, scope our testing to only the riskiest areas, and identify which test tools and techniques would give us the most value in terms of finding the bugs after future merges.
This paper describes an overview of the project as well as a list of requirements, pre-conditions and a template that other teams can use to reproduce this root cause analysis method, enabling teams to identify their own bug patterns and make investments to help find the bugs more efficiently.
Using this root cause analysis method is an effective means to help test organizations reduce costs while increasing engineering and product quality.
Jeffrey Weston, 2012 Technical Paper, Abstract, Paper, Slides, Notes
