Darryl Nicholson, Vesta
Walk a mile in my in my shoes. Calibrated risk taking is one of your core skills. Time to market is a measure of success. Your product is not tangible but rather a service you provide. Your skills as a QA engineer are primarily measured by your ability to accurately articulate the risk of a production release in a Software as a Service (SaaS) environment. Your clients are fixated on schedules; yet they demand and expect the delivery of high quality software as a matter of course. You are responsible for risk management and release readiness all the while knowing that your QA team’s core competency is judged by how well the team gauges what is “just enough” regression testing. In such an environment, how do you define your methodology so that you can determine acceptable risk?
Do you find my shoes comfortable or are you developing blisters? Well, we routinely deliver software which successfully navigates between the financial dictates of swiftness to market and the professional QA engineer’s passion for perfection. This paper will outline how our software development life cycle copes with the new QA reality of SaaS. I will specify how we use software tools and procedures to measure, define and mitigate risk. I will outline how we traverse our path to production and balance speed with quality.
“Just enough” regression testing relies on the idea that there exists a “Minimum Regression Set” (MRS) that is quantifiable and that deterministically exercises the System Under Test (SUT). In my paper, I will explain the whys, the hows and specifics of this methodology. MRS is defined and measured per production release and allows you to evaluate the regression risk of any given software deployment at any point in time. Basically, the risk statement is the percentage of executed test cases derived from calibrated set where one measures API and code coverage. This paper outlines the methodologies used to reduce the MRS to smallest set of test cases needed to provide “just enough” testing so you can deliver high quality software as quickly as possible…ahead of your competitors.
Darryl Nicholson, 2012 Technical Paper, Abstract, Paper, Slides, Notes
