View on GitHub

AVVRV2045

This is the project page for the Autonomy V&V Roadmap and Vision 2045

2022 Autonomy Verification & Validation Workshop

In order to support the development of a V&V roadmap (Vision 2045 of V&V for Autonomy) for Autonomy, Boeing and NASA are holding an Autonomy Verification & Validation Workshop on May 23, 2022 in Boeing’s El Segundo site in California. The workshop is open to all (industry, academics, and government). More information about the workshop (registration, agenda, location, …) can be found here.

Note that the workshop is being held just before the NASA Formal Methods (NFM) Symposium on May 24-27, 2022 in Pasadena, California.

Vision 2045 of Verification & Validation for Autonomy

Industry is developing autonomous solutions that expand the role of aviation in our daily lives and deliver faster services to society at large. Many of those new aeronautical-related services will start with humans in control; but are expected to evolve towards increasingly autonomous solutions in order to remain economically viable. The roles of humans will shift from controlling to monitoring as technical solutions rely more and more on technologies such as Artificial Intelligence, i.e., machine learning, reinforcement learning, multi-objective optimization, and planning & scheduling. The shift will be progressive and require increasingly pervasive use of runtime monitoring. Moreover, as “smarter” technologies are increasingly utilized on both ground and airborne systems, the assurance needs will also evolve. As design-time and operational safety merge together, V&V will have to expand and morph into novel runtime solutions. As the landscape of autonomy changes so does safety assurance and software V&V.

The goal of this effort “Vision 2045 of Verification & Validation for Autonomy” is to build that comprehensive list of V&V needs, develop a roadmap to achieve them, identify the services they can enable, and point to the certification gaps they fill. This effort will serve as support for NASA’s internal, long range, planning exercises which will inform the funding requirements that support this evolution of technology. Questions that need to be addressed include:

  1. What are the new technologies needed to enable increasingly autonomous air services? Each new technology should be accompanied by examples of services it enables.
  2. What are the limitations of current V&V capabilities with respect to these new technologies? For example, are current testing techniques sufficient for deploying ML-enabled systems? Answers should consider the projected use of these technologies as well as their possible re-use in different contexts (applications, environments, and so on).
  3. Where do the current certification standards (such as DO-178C) fail to address assurance needs for these technologies or fail to take into account V&V results associated with the new technologies?
  4. What role can new certification approaches such as Overarching Properties or safety cases play in the certification of increasingly autonomous systems?
  5. What paradigm shifts in the existing regulations are necessary to implement new certification approaches?
  6. How will the assurance of cyber-physical-human systems need to change to accommodate shifting roles and responsibilities between humans and automation? What new technologies will be required to assure anticipated systems architectures? How will we account for the implicit contributions to safety that are provided by humans in today’s systems?

While the individual needs of external customers/organizations/agencies and application domains will vary greatly, they need to be identified so that the community including industry, academia and government agencies can each determine what they can address given their means and their particular focus. To identify these threads, a realistic definition of a “Vision 2045 of V&V for Autonomy” is expected to be built. This effort will involve development of a mapping from V&V needs to technologies to enabled services with a particular interest in how new V&V technologies can fill existing or anticipated gaps in the regulatory framework. Furthermore, since the increasing use of autonomy implies a shift in the role played by humans, new V&V approaches addressing the changing roles and responsibilities should be addressed. At a minimum, the aforementioned questions should be answered with defensible positions.

Autonomy V&V Roadmap and Vision 2045 project

To create the Vision 2045 of V&V for Autonomy and a roadmap to achieve this vision, a collaboration team of recognized researchers from industry and academia was built, with support from government agencies.

The team

Team meetings

We have regular meetings to discuss different topics of autonomy V&V:

Expected Outcomes of this project include:

Autonomy Verification and Validation Workshop Information

To support this project, an Autonomy Verification \& Validation workshop will be held on May 23, 2022 in Boeing’ El Segundo site. The participants are not limited to our project team members, but a visitor’s badge is required, and should be applied 2 weeks before the workshop.

Time of the workshop

May 23, 1:00-5:45pm Pacific time

Registration

Registration is required to attend the workshop. Please send an email to Huafeng Yu (huafeng.yu@boeing.com) for registration with your full legal name and also indicate if a parking place is needed.

Venue and access requirement

Boeing El Segundo S30 Building

1st Floor Tracy Center (PRS16105)

Building S30 located adjacent to 915 North Selby St.

El Segundo, CA 90245

Visitor badge required (application 2 weeks before the workshop)

Schedule of the workshop

Details of the the deep dive discussions:

Workshop organizers