Human-autonomy teaming
The use of uncrewed systems—on land, at sea and in the air—is accelerating in all aspects of defence and security. The common thread across all uncrewed systems is a human operator monitoring data feeds and controlling mission systems to complete complex tasks. Increasingly, artificial intelligence (AI)-enabled systems, machine learning and other autonomous systems are being paired with human operators to reduce the cognitive workload on the human operator.
Learn more about our defence communications solutions
Innovative solution leverages AI and voice recognition
Measuring human performance alongside automated systems
Human-autonomy team (HAT) concepts support human operators in the execution of their tasks, by doing things like automating pattern detection and providing suggested activities and decisions based on data inputs from the environment. This has applications for training where data comes from a synthetic training environment, or real-time operations where real data is collected from mission systems.
HAT concepts have significant potential to advance the state of the art by integrating autonomous systems with human operators to improve overall performance for complex tasks. Calian provided an innovative solution for effective and trusted human-autonomy teaming (HAT) support in a complex air operations environment for NORAD continental surveillance. It measured human performance alongside automated systems using artificial intelligence (AI) and voice recognition. This technology has the potential to be used for a range of aerospace command and control (C2) applications, including for NORAD monitoring and space situational awareness operations.
The challenge
This first part of the challenge was to test the viability of using HAT for timely and effective C2 in complex, dynamic environments with a high volume of multi-source data feeding into a headquarters. The challenge was for continental defence in a North American Aerospace Defence Command (NORAD) headquarters. NORAD is responsible for maintaining situational awareness of all aerospace traffic and maritime approaches to North America. This involves integrating a range of data inputs from military, civil and commercial sources to build and maintain situational awareness.
The second part of the challenge was to measure the trust from human operators toward their AI-enabled autonomous assistant in a synthetic environment. Establishing trust between the human and the AI-enabled assistant is essential to harnessing the full potential of HAT. This is especially important in complex and dynamic environments where AI-enabled tools may identify patterns and solutions that even experienced human operators may not immediately identify.
The innovation project
Two operators, with experience working in NORAD, participated in the evaluation. Two Royal Canadian Air Force (RCAF) airfields with multiple runways were created in a simulated environment to test the technology and to measure human performance. The operators used the HAT technology and were prompted to make timely decisions based on inputs from radar displays, radio communication, visual displays and machine-generated speech. Decisionmaking was captured along with data on the mental and emotional state of the operators using the black cap.
The results
The technology solution provided the appropriate feedback to human operators in a realistic, simulated environment. The team measured the cognitive and emotional responses of experienced operators at multiple airfields, including monitoring incoming aircraft using sensors, visually identifying targets, and having verbal communication with simulated entities. Calian thanks its partner BMU for their contribution and tireless support throughout this project.
The single greatest challenge was the trust response from operators—specifically, developing and establishing the trust of the human operator in the automated component.
- The emotional responses from the participants showed some frustration with engaging with human-machine automated counterparts. Whether this experience is a bellwether for the anticipated experience is yet to be determined.
- With new technologies, human operators often identify challenges because new technologies sometimes require different processes and procedures for maximum effectiveness. This may require experienced operators to change their approach, which requires moving away from existing knowledge.
- Human operators have hundreds—if not, thousands—of hours of experience, resulting in a high level of confidence with existing processes. AI-enabled technology needs to connect with operators in a way that fosters trust that the technology will either improve performance, reduce time and cognitive load for tasks, or both.
Making sense of some of the cognitive data was also more challenging than anticipated because of the relatively small sample size. The purpose was met because we conclusively showed that we can collect cognitive and emotional data.
- The emotional responses varied at different points in the scenario, as did visual and behaviour cues from the operator.
- The emotional measurements resulting from the initial stimulus and the higher-stress stimulus were generally aligned with expectations.
What’s Next?
The results of this innovation project were promising. Human operators delivered on the objectives while data on their cognitive and emotional states were measured. The results show that trust is central to any human-autonomy teaming. This area needs more research. Any technology solution will underperform if the operators using it don’t trust the information they are getting from it. A better understanding of the barriers to trust and what fosters trust is essential to improving the human side of human-autonomy teaming. The success of this project provides an opportunity to expand the scope of this research to better understand how HAT can be integrated into operational settings.
The first potential follow-on activity for future HAT work is to integrate the test bed with other in-service technologies. Using a systems agnostic approach means identifying additional in-service technologies suitable for the test bed to broaden our understanding of how HAT can reduce workload and improve operational effectiveness.
The second follow-on activity is to expand the number of operators involved and to broaden the types of applications. To provide a more representative sample for HAT analysis, expanding the number of operators to 12 would be valuable. A more representative and comprehensive dataset may provide additional insights. The approach used for this project could be expanded into other job roles. ATC was used for this project, and the same approach could be used to evaluate the potential of machine-enabled systems for airborne warning and control systems crew members and aerospace controllers.
Third, the space domain is a high-growth and high‑importance domain where research could benefit operational effectiveness. Human use of space is accelerating every year, including for defence and national security applications. Satellites are being launched into space at a record rate, with new records being set every year. Orbits are becoming increasingly crowded, and maintaining situational awareness and exercising effective C2 over space assets will become increasingly challenging.
To better manage the complexity of space situational awareness and space operations, HAT solutions could reduce cognitive loads on operators and improve overall performance.
Lastly, more research should be conducted using realtime data. The data provided in training is synthetic and based on real-world experience and scenarios. To validate the potential application of HAT solutions for operational purposes, future research could connect real world data to validate the applicability of HAT to operational missions.