PNSQC Marketing Chair Moss Drake recently interviewed upcoming Keynote Erik Van Veenendaal, who will be presenting “Building on Success, Beyond the Obvious” at PNSQC 2021, on October 11-13. Erik, an international consultant, and trainer, has received both the European Testing Excellence Award and the ISTQB International Testing Excellence Award for his contributions to testing. In this interview, he discusses his expertise in testing methodologies and test improvement models.
PNSQC: Some people might be worried about your stance on “good enough testing.” How would you respond to that?
Erik Van Veenendaal: Some people may well be correct in being worried, it depends on the context and what you are testing. In the safety-critical environment, “good enough” may not be sufficient. However, most of us do not work in a safety-critical environment, but in an organization where there is a constant balancing between time-to-market and quality. In these organizations, testing is about understanding the business drivers and translating this into an adequate test approach. The test approach should fit the balance with time-to-market, sometimes missing one or two (minor) defects is ok (although as testers we do not like this situation, which is why we are testers). This is what I refer to when I’m presenting and discussing good enough testing. Not everyone will always be able to achieve 100% coverage of all requirements, all code, and in all combinations.
PNSQC: Often bugs appear in production because of configuration or external changes. Afterward, management asks “Why wasn’t this found in testing?” Any suggestions on how to answer that question?
EVV: An initial response might be, but who made the error that eventually resulted in this failure? Thereafter, I will start explaining that exhaustive testing doesn’t exist and that testing is often squeezed and not provided the time that was requested. Especially, I like to explain the principle that exhaustive [testing] does not exist and try to relate it to an example from their own application. For instance, an input with various fields where they are asked to provide examples of what should be tested. I then show that testing these examples in combination very quickly leads to an enormous amount of input possibilities. No way can we test all of those, we have to make choices. In the discussion with them, I may well end up coming across risk-based testing and its important role in this. If you tell us what are the main risks and what is important to the business, we are able to focus our testing and set the right priorities. Remember, we cannot test everything, but will aim to test the most important parts to the best of our ability.
PNSQC: In your ideal organization, who creates the test plans?
EVV: Not sure what the ideal organization looks like, in which context they operate, what product they are developing, and which lifecycle they are using. These are all items that could or would influence the way I should answer this question. Without knowing this, let me still have a go at answering this question. A test plan to me is a logical part of a project management plan, but with a specific focus on testing. So basically it is the responsibility of the project manager, however, is all is organized well he or she is supported by a test lead. This test lead understands testing, product risk analysis, how to define a test approach and exit. The test lead should be able to establish a test plan that has merit and good valuable content. Of course, all of this is very different in an Agile where the team will together define the “plan” for the upcoming iterations. The test plan then becomes a team effort where the testers in the team provide guidance.
PNSQC: The title of your talk is “Building on Success, Beyond the Obvious.” How do you define success?
EVV: Happy stakeholders, happy customers, and happy users and as a result a happy business or organization. Success also means being on time, within budget, and product quality as expected. In this specific presentation “Building on Success, Beyond the Obvious” I talk about practices I applied in testing throughout my career that really contributed to achieving to happy stakeholders and ultimately delivering more or less on time, within budget and also product quality being more or less as expected. Practices that made the difference, practices that made it happen.
PNSQC: Apparently you’re a world champion table tennis player. Do you bring any of your quality and testing expertise to that game?
EVV: Well, let me correct: I’m not a former world champion, but I did participate in two table tennis world championships. Sure, those are things you can take to the game — getting better requires dedication, winning requires focus — but, also, in order to be successful, thorough preparation is needed. The opponent is analyzed for his strong and especially weak spots, where can I find the “defects,” how can I score the points. Perhaps the best analogy can be through the pesticide paradox, if the same tests are repeated over and over again, eventually, these tests no longer find any new defects. If the points are played with the same strategy over and over again, eventually this strategy will no longer be successful. Your opponent will be prepared for your game. To keep scoring points, the existing strategy may need changing and adapting, to win the match.
Thank you, Erik!
