By Chris Thatcher | January 29, 2019 Estimated reading time 6 minutes, 55 seconds.
Artificial intelligence (AI) may someday transform how aircraft fly, but one of the early beneficiaries of the emerging technology could be more elemental–flight instructors.
Imagine sitting behind a student pilot in a cockpit simulator and, on a tablet in real time, having precise biometric and other performance data as she executes a difficult procedure. Was the student calm or was her heart rate significantly elevated? Were her eyes scanning the correct displays and controls or looking elsewhere?
Flight simulators already provide instructors with plenty of information on a student’s performance. And a lab can gather biometric data from an array of body sensors. Combining the two in a way that doesn’t distract the student from the task at hand is another matter. Enter AI.
Thales is developing a non-invasive method to monitor performance, using cameras and AI to provide instructors with insight into the mental and physical state of a student during critical moments in training.
The application, called HuMANS, synchronizes and contextualizes what might otherwise be an overload of data, explained Christophe Bruzy, the technical director of simulators for the French aerospace, defence, transportation and security company.
It employs an array of seven cameras to monitor and track the eye movements and other physical responses of a student that the AI then displays in graphical detail. In a debrief, instructors can select specific phases of the flight to show a student exactly what has happened, simultaneously presenting a map of the flight, a video of the student in the cockpit, and a graphic of their eye movement, among other information. Instructor notes made on the tablet during the session are also synched with the flight.
The system can help an instructor identify and correct “inappropriate behaviour,” said Bruzy, and is not platform agnostic, meaning it could be adapted to any flight or vehicle training system.
Using similar technology, Thales is also developing a training system for air traffic controllers (ATCs) that could significantly improve their effectiveness while reducing the time and cost of instruction.
Faced with a shortage of ATCs and a flight volume that is expected to double in Europe over the next two decades, “this is our solution to cope with the increase in training needs raised by this huge increase in traffic load that is coming,” said Jean-Phillipe Pinheiro, manager of Thales’ SkyCentre air traffic management innovation lab.
Called SkyCentre-SimDebrief, the system tracks the actions of a trainee. It monitors eye movement and heart rate and captures voice cadence and duration of calls with pilots as the student manages simulated aircraft separation. All of that information is available to assist the instructor in real-time.
“If you were an instructor doing a simulation with your trainee, the electronic instructor assistant would be like another instructor sitting alongside you,” said Pinheiro. “It would be looking at a lot of data, detecting if something is going well, going bad, and touching you on the shoulder [to say] there is something happening at this position.”
He compared the experience to the motivating effect of a smartwatch that synthesizes a wealth of data on a person’s physical activity and encourages them to keep going.
To refine the AI of each system, Thales is using machine learning based on instructor grading and feedback.
The presentations were two of more than a dozen panels and demonstrations of AI applications in transport, defence, cybersecurity, healthcare, and cities during a media day hosted at the company’s new Centre of Research and Technology in Artificial Intelligence eXpertise (CortAIx) in Montreal.
Thales typically invests about 20 per cent of its revenue in research and development and has been exploring AI for almost 30 years, said chairman and CEO Patrice Caine. Since 2016, though, it has invested over $10 billion in AI, connectivity, big data and cybersecurity. It opened Digital Factory, a digital incubator and cyber security hub, in Paris in 2017, and CortAIx in 2018.
The interest in Canada is well timed. In late December, the federal government announced nearly $230 million for a Quebec-based AI-powered supply chain supercluster, called SCALE.AI. That followed the creation in early December of an international panel on AI to share research results and best practices and convene international AI initiatives. The panel was first announced by Prime Minister Justin Trudeau and French president Emmanuel Macron on June 7, 2018.
In a wide-ranging discussion before the demonstrations, Caine acknowledged the risks associated with AI–he specifically pledged not to invest in autonomous killing machines: “It’s not that difficult to say no to killer robots,” he said. But he argued more transparency is needed to “build trust” and understanding in the decisions AI applications make if they are to be readily accepted.
“I don’t think the solution is to put AI in a cage and make everything secret,” he said. But companies like Thales will have to contribute to accepted standards and rules as a framework for the governance AI. “At the moment AI is seen as a black box … [We need] to be able to explain how and why the machine did or recommended what it did.”
As a company involved in a number of the demonstration projects for autonomous vehicles and urban air taxis (Caine wouldn’t specify with whom he has partnered), Thales has an obvious stake in building the public trust that will be critical for such aircraft to take flight.
