Robotics Software Reliability:
An FTC robotics team uses software fault tree analysis (FTA) and software failure modes and effects analysis (SFMEA) to predict uncertainties in the physical world.
PNSQC would like to present this article by the First Tech Challenge (FTC) robotics team The Bytes of Kitkats. This team of high school students is presenting their first paper at PNSQC. The team formed in 2013 and in 2016 they received the Oregon championship award in FIRST LEGO League (FLL). They have been recognized for exemplary robot software development and innovative designs in Oregon, US FTC West Regionals and FTC world Championship events. In addition to striving for hardware and software excellence, the Bytes of Kitkats do outreach by educating young students in Portland communities, other US states and countries about robotics through camps, specific topic workshops, community open houses, and mentoring younger robotics teams.
The Mission
Robotics software reliability is crucial to controlling robot hardware while interacting with the physical world to accomplish a set of missions.
Reliable software needs to handle sensor errors, incorrect functionalities, software exceptions and uncertainties from the external environment. At the Pacific NW Software Quality Conference (PNSQC), we will share our experiences tackling these challenges in Android robotics as a FIRST Tech Challenge (FTC) robotics team: the Bytes of Kitkats.
The Model
Linking robotics and artificial intelligence will be crucial when humanity takes the next steps towards setting up a base on the Moon or setting up a permanent colony on Mars. Smart robots will be required in such extremely hostile environments. However, we first need to ensure that the software running these robots meets exceptional quality through effective quality assurance. The team has created an interesting approach to improving software reliability in Android robots by integrating robot software and physical worlds together.
The software fault tree analysis (FTA) and software failure modes and effects analysis (SFMEA) provide the statistical testing to evaluate and improve software reliability. The shortcut model predicts software reliability growth through the stochastic process. Our objectives are to build the software reliability model with finite or fixed number of inherent defects and accelerate the software reliability improvement process to ensure the robot behaves consistently in the field for FIRST Tech Challenge (FTC) robot competitions.
The Method
By building and testing Android robots with different motors, sensors and 3D-printed attachments, we could categorize sensor reading uncertainties (e.g., the position of the robot and surrounding objects) and multiple uncontrollable environmental factors from the physical world into software reliability models. Comparing the software reliability model data and the robot performance in the field, we found that the model is a powerful tool to guide the team to shrink software reliability gaps in critical areas for consistent robot performances and direct the robot redesign strategy and timing for performance improvements. All the methods that we shared can be used in many industries to improve software reliabilities when handling the physical world uncertainties and complicities.
The Team
This is the first technical paper the team has presented at PNSQC. We are thankful to the PNSQC program committee for their support and help during the paper review process. We look forward to meeting software quality professionals and enthusiasts, sharing our experiences and learning from other experts. Quality Assurance for software reliability is one of the biggest challenges. The world of integrated robot software and physical world complexity, the fast pace of competition and high robot performance expectations, the balance between the robot evolving functionalities and testing, all of them make software reliability even more challenging and also more exciting than ever.
Our FTC team members are multi-functional. Reliability is considered in every software functional design discussion and review. We evaluate different user stories, design the test strategy, and compare different approaches to software reliabilities. Software reliability becomes an essential part of QA, robot design and the team competition strategy definition process.
Developing reliable software faster and better within a complex physical world is critical for Quality Assurance. We have to be ever faster to face the challenge of increasing complexity and high-performance expectations. As a result, we need to build more advanced software reliability techniques, tools and testing processes on our road to software reliability.
Bo Li works in software development and management for Logic Technology Development in Technology and Manufacturing Group (TMG) at Intel. He joined Intel in 1999 after receiving Ph.D. from Georgia Institute of Technology. Outside of work, he volunteers to mentor multiple robotics teams in the community for high school and middle school students to foster more passions and knowledge in Science, Technology, Engineering and Mathematics (STEM).
Catch the team and their presentation at PNSQC in October this fall.
