Sharookh Daruwalla, Intel Corporation
The scope of software validation is always expanding with the ever-increasing test matrix making automation a necessary tool to alleviate the unmanageable workload on validation teams. However, extending automation for Virtualization OSes (e.g. VMWare ESXi) come with unique challenges not seen with traditional OSes (e.g. Windows, Linux). Most test execution engines support only one specific traditional OS running on a single System under Test (SUT). Further, they are also designed for static resource management and usually support only the handful of scripting languages and communication protocols preferred by the supported OS. This paradigm does not work for virtualization, where a SUT is only a host or manager of virtual machines (VM) running different Guest OSes. In addition, virtualization test times are also longer due to the additional steps required to setup VMs. This paper presents a detailed look into the challenges faced when automating test validation for Virtualization OSes and the design choices used to overcome them.
The proposed ITE-Berta test automation framework implements dynamic resource management within its libraries to support VMs created within tests, adds communication protocol support for multiple OSes, and uses cross-script execution to extend support for languages not natively supported in the test execution engine. To reduce virtualization test times, we implement a Test Continuation Mechanism that maintains setups and performs selective clean-up between similar tests. Finally, we analyze our proposed solution and how it may be applicable to different Virtualization OSes and test frameworks.
