Maros Kutschy – Ness
Along with Functional testing of the UI with Selenium and Layout testing of the UI with Galen framework, it is essential to perform also Visual UI testing. Among the tools on the market, I chose Percy https://percy.io/. Even with the free plan, you have 5000 screenshots per month. In these days full of challenges related to the Covid crisis when companies try to automate everything that is possible, I think performing automated visual checks instead of the time-consuming manual check is a very good strategy for companies to be more effective and productive.
On our project, we had to decide how to automate specific applications, which have many visual elements. We are using the Java Cucumber automation framework for UI and API testing. For UI testing we are using Selenium, but this tool is not very suitable for Visual testing. Therefore, we decided to do POC for Visual testing of this specific application with Percy https://percy.io/
We didn’t have many experiences with visual testing, we played in the past with Percy’s main competitor – Applitools Eyes, but we didn’t invest much time into this. The biggest challenge was to find out how to integrate running Percy tests into CI Jenkins build, how to get Slack notifications to be notified about Percy validations results and how to stabilize the tests. Another problem was related to big refactor of the framework, after which Percy integration stopped working.
The problem with CI Jenkins integration: we are using Java Cucumber framework, but Percy requires Node Js installed. We solved the problem with communication between Jenkins slave and Jenkins master by special setup of Jenkins job and also one-time setup on Jenkins slave. Slack notification: as you do not see results of Percy validations in the framework reports, we integrated Slack notification after each Percy job build. Stabilization of tests: We regularly improve tests to be more stable Problems after the big refactor of the framework: we created a temporary git branch without refactor from which we run the tests.
Lessons learned from this experience: Integration of Percy into our automation framework was a good decision. POC was successful and we have now already around 180 scenarios testing this specific application. We learned that this tool serves good to our needs but it is necessary to invest time to properly understand how it works and to integrate it to the framework. This time investment paid off.
I will share my experience with Percy including live demo about how to integrate it in any framework, how to run the tests, how to check the results and how to integrate it in CI process.
Maros Kutschy, 2021 Technical Presentation, Presentation
