Dawid Pacia
We all know that a modern tester should have both analytical skills and basic knowledge of programming as well as soft skills. If we add experience in agile software development, we get an almost perfect candidate in the eyes of employers.
However, what if one day your employer says:
Our sensor has been recently working too short. Please, do the battery life tests. Current functionalities of our product will be enhanced soon. Please, write some tests for the embedded platform. Clients are complaining about our system working too slowly. Please, prepare the testing environment for performance tests of microprocessor memory.
Apart from the word „please”, above sentences have one more thing in common – for a long time neither of them was of any interest to the software tester.
Nowadays, the world is changing and very soon testers can face such challenges but it is still not so common to make them aware of it during QA’s conferences. The content of this presentation consists of 3 major parts:
- Risks, challenges and safety aspect the modern tester should be aware of
- The real-life examples of safety vulnerabilities and its impact on customers (and the scale of it!) What makes testing so exciting despite so much responsibility and so many challenges
Risks, Challenges, and Safety Aspect:
- Be aware of flaky tests and their reasons
Now we live in an era of continuously delivered things. That causes even more and more test automation that we rely on. However, can we still treat every environment as safe enough? We should be still aware of false negatives (the test is marked as Passed when the functionality is not working) and false positives (the test is marked as Failed when the functionality is working). There is also something even worse – flaky tests (tests failing intermittently). And there are a lot of reasons behind that… (both, human and environmental ones)
- Updates (OTA) as the riskiest area
One of the most used “functionalities” by (aware or not) customers. When you think about SW update, usually that’s not something to be considered and safety-critical. For web applications, every blocker can be fixed by fast deployment or bugfix. For mobile apps, it’s a matter of releasing a new version to store. Even for a desktop, you can force your user to reinstall it once again. But what if something goes wrong for IoT/HW devices? Is there any safety “belt” or procedure to prevent your firmware from turning into a brick?
- Scale matters
New smart devices are very often responsible for public and private safety. To cover dedicated area (or just get your customers involved) there are a lot of them needed – sometimes thousands of them! Do you think that scale can be skipped during testing? How to ensure safe infrastructure when your network is growing up? Which parameters should be considered as important (e.g connectivity issues, configuration, geolocation, external influences, wireless signals multiplication)?
Key takeaways include:
- Understanding how product risk influences safety issues
- Knowledge of how to identify (and mitigate) major safety problems
- Understanding what safety means to the customer and how to incorporate and transfer it into testing activities
- Perception of how to deal with “undiscovered” problems even without substantive experience
Dawid Pacia, 2019 Technical Presentation, Paper, Slides
