Alexa, How to test Alexa? - An Automation approach

This paper presents an idea or a concept for test automation on gadgets to use voice assistance.

Nowadays the Voice assistants, like Alexa, are taking over households, in the USA anyway, where one in four households owns at least one device.  Those gadgets provide an increasing part of the added value of modern gadget systems and thus become more complex to test.

Where one of the most common general problems with the test automation framework working with hardware and software to test the voice assistance seems to be forgetting that any test automation project is a software project in its own right. Software projects fail if they do not follow processes and are not managed adequately, and test automation framework projects are not different. Of all people, test engineers ought to realize how important it is to have a disciplined approach to software development.

This paper focuses on developing a Proof Of Concept to approach an automation framework running a script on Python and using different libraries that strictly follow the QA best practices and coding to execute a common E2E Test Scenario, where the main goal is to approach a quality strategy to test the Alexa voice assistant on real android gadgets to validate Amazon's wake word "Alexa" triggered in different acoustic noise conditions and different utterances to validate how many times is successfully detected; looking to help on QA community sharing knowledge to foster a software testing quality.

Takeaways:

  • A glimpse of what can be achieved to test Alexa voice assistance using Python and different libraries on the End-to-End testing.
  • Analyze the requirements and challenges of a test automation framework which must be identified as a software development project.
  • If you are a manual tester, starting with Python could be a good option to start your career as SDET, due it simple, enjoyable, easy to use, and it's easy to learn.
  • Learn Automation Framework with different tools to show the whole umbrella of Voice assistance testing
  • Show a demo running scripts to validate the triggering 'Hey Alexa' with the technique End-to-End testing, developed on Python.
  • Testing -- The testing designing techniques considered that can be used for designing test automation architecture
  • Android Testing -- How to implement an android test automation framework approaching the testing in voice assistance on devices using Alexa.
  • DevOps (CI and CD) -- How to integrate test automation into your continuous integration process.
  • Agile -- How to approach test automation using a Scrum-based model.


Juan de Dios Delgado BernalJuan Delgado, Consultant in test, Mobica

QA Manager and Test consultant with experienced Software Development Engineer in Test with a track record of delivering major test solutions for the most important telecommunication companies across the whole US in projects on behalf of Mobica. With strong project management skills, key areas of expertise include understanding complex business requirements & formulating robust test strategies; developing automated test solutions with the ability to interface between Development, Project, QA, and the Teams to ensure execution of test strategy.

Extensive software engineering skills in Automation creating house automation frameworks in different models (usually page object model) implemented with different programming languages such as Python, java, selenium, cucumber, and different tools like Browserstack and integrated with Continuous Integration tools (CI/CD) and version controller as Git.

Involved in the test management of Agile/Waterfall projects throughout the Software Testing Life Cycle (STLC). Also creating structured processes & best practices to manage defects right through to resolution.

Excellent knowledge and implementation skills working on Manual and Automation testing of Web apps Server, Mobile apps, Web services applications (Client/Server), and Cloud(AWS). Experience in various domains(telecommunication, E-commerce, Banking, Healthcare, etc.).

Professor at Universidad Tecnológica de Aguascalientes focused on the subjects of Project Management and Project development methodologies. Also, a Professor at Universidad Panamericana in the faculty of Engineering focused on Software Quality Assurance.

I have been Speaker at the most famous software testing conferences about relevant topics to inspire the QA community. relevant participations: ConTest in New York, PNSQC in Portland (3 times - 2018, 2019 and 2020)
https://www.pnsqc.org/juan-de-dios-delgado-bernal/

Deanna Raven, Senior Project Manager, Mobica

Experienced Technical Project Manager with a demonstrated history of working in the computer software industry. Skilled in Bug Tracking, Databases, Management, Report Writing, and Software Documentation. Strong program and project management professional with a Certified Paralegal focused in Canadian Law from Stratford Career Institute.

Piotr Wroblewski Divisional Delivery Director North America Mobica

Senior Manager with a demonstrable track record in IT Projects, Transitions & Program Management areas.

Successfully delivered number of complex programs, transitioned and transformed organizations, defined organizational structures, team functions. Integrated multiple technologies, operational processes and procedures to drive efficiency and cost savings while developing and motivating the teams to exceed business objectives.

Currently focusing on software delivery, seeking new ventures and opportunities within software development projects and portfolio management.

Specialties: Specific skills include: software delivery and engineering, Project/Transition Management, Offshore/Nearshore Transitions and Operations Management

Looking for objective driven project, program and delivery manager with a bigger picture of your portfolio and business strategy able to lead your team towards it? Lets have a chat."

Paper
Presentation