Michael L. Hoffman, Consultant
The aspiration of any software company or organization is the delivery of software products within defined goals of scope and constraints of schedule and resources. The competitive nature of business requires that organizations live up to these goals, and work within these constraints more effectively and efficiently over product lifecycle generations.
What constitutes improving over time for a software QA organization varies depending on perspective:
- For a tester it may mean more quickly differentiating between improper and expected behavior.
- For a customer support analyst it may mean less issues found in the field.
- For a test engineer it may mean optimizing test coverage based on risk assessment.
- For a quality engineer it may mean identifying issues in earlier lifecycle phases.
- For a QA manager it may mean making sure a quality/test team has the proper knowledge and tools in time to validate new technologies.
- For a QA director it may mean doing more with less.
The ability to consolidate so many perspectives into a comprehensive evaluation of improvement at the organizational level is a complex challenge. A QA organization needs to regularly ask itself these questions:
- How do we know we are doing the right thing?
- How do we know we are improving over time?
Realizing quantitatively whether your software QA organization lives up to the dynamic needs of your business is not as simple as looking at defect trends over time. An appropriate evaluation involves various aspects of the software QA function, including operational behaviors, talent, customers and budgets.
This paper chronicles an approach taken by one software QA organization to evaluate its own health through establishing goals, benchmarking, defining organization-level metrics, and on-going self-assessment.
2010 Technical Paper, Michael L. Hoffman, Abstract, Paper, Slides
