Jonathan Li On Wing, Groupon
Have I been approaching testing all wrong this whole time?
In my earlier years of being a test engineer, I was taught to write tests that are repeatable: given a certain set of parameters and inputs, map out the expected output – repeatable and simple. However with a developer-test ratio of 10 to 1, I was quickly falling behind. I found myself having to decide between spending time on my automation test suite and exploratory testing. The former gave great reliability regarding regression, but the latter caught most of the bugs. The former was repeatable and could run practically 24 hours a day, the latter only when I found time. How about if we implemented some of the exploratory concepts in the automated test suite?
In this paper, I will discuss about my experience in adding variability to test automation and creating an oracle to be able to determine correctness. I will demonstrate that, in my experience, adding randomness to the tests can help quicken delivery time, improve the discovery of bugs, and increase confidence, without compromising repeatability, when compared to traditional automation methods.
Target Audience: Introductory
2015 Technical Paper, Jonathan Li On Wing, Paper, Slides, Notes, Video.
