Engin Uzuncaova, Microsoft
Software testing, in general, is a human-centric process where creativity and technical skills mix and mingle in unique ways to both understand and evaluate software systems. When we are faced with new and original testing challenges, it is usually the human element in the process that makes it possible for us to repackage the existing tools and methods, again, in new and original ways for viable solutions. As the complexity of a system under test increases, this task becomes harder to accomplish. What do we do when the level of understanding required to properly analyze and test such systems is beyond our comfort zone?
Testing geo-spatial data and route planning algorithms is a good example for this; the inherent complexity of geo-spatial data, the representation of the data and also novel algorithms that utilize this data make testing more of a daunting (or fun?) task at both levels: understanding and evaluating. This paper presents a collective testing approach that we have developed at Bing Maps to attack the complexity of our route planning product focusing on the following areas:
- Heuristic-based tests to identify defects that are otherwise hard to find,
- Efficient test infrastructure to promote agility,
- Intuitive visualization solutions for complex geo-spatial entities and scenarios and, finally,
- Integration of complex statistical analysis techniques with the testing process.
This approach has provided improved coverage in our testing and enabled us to identify more quality issues during development. We also benefited from a more efficient evaluation mechanism for complex issues found during testing. As we head towards providing far more geo-spatial coverage and enhanced features to our customers, our testing approach will have to adapt and evolve accordingly; this paper presents some pointers in this area as well.
2010 Technical Paper, Engin Uzuncaova, Abstract, Paper
