Text description of graphics and narrator voice in the film AI enhanced Air Traffic Control

1. Graphic 1
a. Two cars are shown with signals emanating from them
The speaker voice says: “Today, autonomous cars are feasible in a chaotic environment,
so we asked ourselves:”

2. Graphic 2
a. An aircraft is shown with signals emanating from it
The speaker voice says: “could autonomous air traffic control be feasible, in a controlled
environment? At LFV, we have made an effort to get closer to the answer, let’s look at
how we did it...”

3. Graphic 3
a. Text is shown with “Technology” as an overarching title and “Radars”, “Radios”,
“Network” and “Technology” as subtitles.
i. Transition to:
b. An illustration of a human is shown and placed in front of a radar screen and desk.
Communication with an aircraft is shown and “cogs turning” to show that the human
is thinking. Parts of the cogs are shown to be alleviated by “technology” which
reduces the amount of thought needed by the human through the use of a pie chart
illustration.
The speaker voice says: “Air Traffic Control is relying heavily on technology already, with
radars, radios, networks and navigational aids, but at its core, are humans. The humans
are there for safety reasons, and their capability of managing complex situations. But are
we now at a point in technological achievement where humans could be alleviated from
a considerable portion of their complex tasks, even in Air Traffic Control?”

4. Graphic 4
a. Two aircraft are shown separated in altitude
The speaker voice says: “The primary purpose of ATC worldwide is to prevent collisions,
organize and expedite the flow of air traffic, and provide information and other support
for pilots. Air traffic controllers monitor the location of aircraft..”

5. Graphic 5
a. The two aircraft transition to their respective latitude and longitude positions over a
map of Europe, with additional aircraft shown, flying across Europe.
The speaker voice says: “.. in their assigned airspace by radar and communicate with the
pilots by radio. To prevent collisions, ATC enforces traffic separation rules..”

6. Graphic 6
a. The text “Traffic separation rules” is shown with three checkboxes appearing
underneath.
b. A box illustrating airspace around an aircraft appears, illustrating separation in three
dimensions.
The speaker voice says: “..which ensure each aircraft maintains a minimum amount of
empty space around it at all times.”

7. Graphic 7
a. A human is illustrated with a pillar indicating 100% work capacity, a computer chip
with AI written on it appears and a new pillar appears on top of the previous pillar,
indicating more than 100% work capacity.
b. Transition to focus on the computer chip. And boxes with following content:
Explainability of AI, Human machine interaction, safety aspects, infinite traffic
scenarios, weather conditions, financial costs, redundancy.
The speaker voice says: “Air Traffic Controllers, like every other human being, have a
certain work capacity. So in this project we raised a hypothesis: if we provide Air Traffic
Controllers with Artificial Intelligence to automate some of their tasks, they could
potentially increase their work capacity and handle more flights. To reach this vision
there are challenges on the way. We have to take many aspects into consideration, like
explainability of AI, human-machine interaction, safety aspects, infinite traffic scenarios
to potentially train on, weather conditions, financial costs and redundancy.”

8. Graphic 8
a. The logos of LFV and IBM are shown.
The speaker voice says: “to address these challenges we have partnered in this project
with the IBM Garage and IBM research.”

9. Graphic 9
a. A lattice is shown with 5 layers, a title is shown at the very end of the illustration
with the text “Advanced autoplanner”. From bottom to top the layers shown are:
i. Enterprise Design Thinking
1. Co-created with air traffic controllers and IBM researchers
2. Defined minimum viable product
ii. AI modelling
iii. Backend application
iv. IBM streams
v. NARSIM, Air traffic control simulator
The speaker voice says: “We have performed Design Thinking Workshops to co-create the
challenges and solutions together with Air traffic controllers and data science experts. The
workshops were performed to define a Minimum Viable Product, the smallest and most well
defined solution to mitigate the biggest risks. The project team then entered the AI
modelling phase and created the backend application, these were then deployed and
connected to Narsim, an Air Traffic Control simulator. With the help of IBM Streams service,
we were able to test how the AI model behaves in the Narsim simulator. We named our
solution Advanced Autoplanner (AAP).”

10. Graphic 10
a. Text is shown illustrating that a “safety first approach” has been used, two subtitles
are indicated, they are:
i. Phase 1: Forecast airspace in future
ii. Phase 2: Course of action recommendations
The speaker voice says: “The Advanced Autoplanner AI model is designed to use a safety-first
approach and operates in two phases: phase one forecasts the future state of the airspace,
to determine safe actions that avoid future conflicts, and phase two provides best course of
action based on ranking safe actions from understanding common aircraft characteristics.”

11. Graphic 11
a. Text “Phase 1: Forecast airspace in future” is shown and an illustration with aircraft
flying towards each other is shown from above, with forecasted trajectories. The
illustration transitions to showing the airspace and forecasted trajectories from the
side, indicating complexities in an airspace with descending and ascending traffic.
b. Boxes surrounding the aircraft appear, showing the required separation of airspace
around each aircraft. Two of the boxes flash in red.
c. An illustration of a human appears.
d. The human disappears and is replaced by multiple alternatives for various course of
actions that could be applied. The title on the page transitions from “Phase 1” to
“Phase 2”.
e. The alternative courses of action are shown to be prioritized.
The speaker voice says: “The AI model forecasts locations of all the aircrafts in real-time
and determines continuously the future state of the airspace that an Air Traffic
Controller is responsible for. The AI model captures the complex three dimensional
trajectories including the waypoints in a flight plan (for a given aircraft). In phase 1 the AI
model uses a lattice-based search space exploration technique with early cutoffs for
determining safe actions. This allows the AI model to offer both explanations why to take
an action and counter-factual explanations why to not take an action, to an air traffic
controller. In phase 2 the AI model uses a rule based approach to rank actions by
optimality from the set of safe actions identified in phase 1. The rules capture common
aircraft characteristics such as preference to increase in altitude, because at higher
altitude, the air is thinner and thus results in improved fuel efficiency. A machine
learning approach could be adopted in the future for action re-ranking based on fine
grained aircraft characteristics such as body type and maneuvering capabilities.”

12. Graphic 12
a. Two aircraft are shown heading towards each other, one of them is deviated in order
for the aircraft not to collide and then resumes its trajectory.
The speaker voice says: “When the AI model recommends a successful action to the
pilot, the AI model tracks when the aircraft is safe to be brought back to its original
course.”

13. Graphic 13
a. Three scenarios are outlined on the screen, they are:
i. Scenario 1 – Advanced autoplanner identifies, assesses and solves a conflict
between two aircraft using speed.
ii. Scenario 2 – Advanced autoplanner solves conflict by using turn, speed and
altitude.
iii. Scenario 3 – Demonstration of what happens if pilots do not follow the
instructions given by the Advanced autoplanner
The speaker voice says: “We would now like to show you three scenarios, handled by the
AutoPlanner solution in the Narsim Simulator. The separation minima needed in this sector is
5.0 nautical miles, but we have added a buffer, the AAP is not allowed to go below 6 nautical
miles separation.”

14. Graphic 14
a. A video displaying separation management, as seen in Narsim between two aircraft,
by Autoplanner using speed.
The speaker voice says: “First, we will see how the Advanced Autoplanner identifies,
assesses and solves a conflict between two aircraft using speed. At this time, we can see
that both LFV042 och LFV050 have a conflict in about 12 minutes, inside sector Whisky.
Advanced Autoplanner is using its conflict search algorithm: identifying the conflict and
looking for solutions on how to solve it. The two aircraft are now separated by 5.9
nautical miles on the same level, which is just below the buffer AAP is using for safe
separation. When aircraft LFV042 enters the sector, 9 minutes before loss of separation,
Advanced Autoplanner issues a clearance to increase the speed to Mach .78, a slight
increase of the ground speed by 4 knots. The use of speed to solve this conflict is based
on LFVs own ranking of how the Advanced Autoplanner should weigh the different
solutions: turn, speed and altitude. We can see that the small increase of speed has also
increased the predicted separation to 6.7 nautical miles, which is more than the
minimum separation. 9 minutes later, we can see that the issued clearance, for increase
of speed, ensured separation between the two aircraft.”

15. Graphic 15
a. A video displaying separation management, as seen in Narsim between three
aircraft, by Autoplanner using speed, turn and altitude.
The speaker voice says: “In scenario 2 the Advanced Autoplanner is using all three tools
available: turn, speed and altitude, for solving conflicts between three aircraft, LFV 900,
LFV915 and LFV911, who are all at 40 000ft. LFV900 coming from the north is given a
clearance to increase the speed to M.86, a 10-knots speed increment, and to turn 5
degrees to the right. This is to avoid the conflict with LFV915 coming from the south. The
separation between the two aircraft went from 6.1 nautical miles, which is on the verge
of autoplanners minimum allowed distance, to around 13 nautical miles, which is well
above the required distance. Simultaneously LFV911 from the east is given a climb to
41 000 ft to avoid both LFV900 from the north and LFV915 from the south. Advanced
Autoplanner now regards that LFV900 is clear of LFV915, and gives LFV900 a new route
clearance back to waypoint WOODY, outside Antwerp in Belgium. The aircraft uses its
own navigation back to the original flightplan. The predicted separation is now 10
nautical miles, well above the safety margin Advanced Autoplanner is using for
separation, but within what we in the project regard as efficient flight trajectories. We
can now see that all three aircraft are separated according to aviation rules, either by at
least 5 nautical miles, as between LFV900 and LFV915 or by at least 1000ft as between
LFV911 and the other two aircraft.”

16. Graphic 16
a. A video displaying separation management, as seen in Narsim between two aircraft,
by Autoplanner using turn and altitude whilst experiencing lack of adherence to
given instructions by pilots.
The speaker voice says: “In scenario 3, we would like to show you what happens if the
pilots do not follow the instructions given by the Advanced Autoplanner. LFV957 in
northeast is given the instruction “5 degrees right” to turn behind LFV970 coming from
the west. The pilot on LFV957 however, only turns 2 degrees, for unknown reasons. The
separation went from 2.9 to 4.4 nautical miles, which is still below the separation minima
needed in this sector. The Advanced Autoplanner identifies that the previous clearance is
not sufficient, and turns LFV957 another 5 degrees to the right, but again the pilot only
turns by 2 degrees. The predicted separation is now 5.8 nautical miles, which is above
the minimum separation needed, but still below the buffer used by AAP. LFV970 is now
entering the sector, and is promptly given a clearance to climb 1000ft to avoid LFV957,
but again the pilot does not adhere to the clearance, and stays at his altitude. Advanced
Autoplanner now assesses that LFV970 is not adhering to its clearance, and turns LFV957
a third time to the right, and this time the turn is sufficient to maintain separation
according to the defined buffer. This shows that the Advanced Autoplanner follows up its
instructions and is constantly making sure that these instructions are adequate to
maintain the separations required.”

17. Graphic 17
a. Text displayed stating “What LFV learned”, additional text says “AI model
successfully separates aircraft and controls them safely and efficiently through the
sector”
The speaker voice says: “The project has developed a model that successfully separates
aircraft and controls them safely and efficiently through the sector.”

18. Graphic 18
a. Text displayed:
i. What are the next steps?
1. Increased complexity in AI modelling
a. Climbing and descending traffic
b. Various wind conditions
b. A human in front of a desk and a screen appears as well as additional text:
i. Human machine interface (HMI)
1. Presented proposed instructions
2. Presented time for execution
The speaker voice says: “In the next phase we will include some of the more complex
realities of air traffic control, including climbing and descending traffic as well as various wind
conditions and we may enable the AI model to learn based on the interactions and choices
by the Air traffic controllers as well as simulators to evolve and make better
recommendations. We will also provide the controller with a Human Machine interface
(HMI) presenting proposed instructions and time for execution.”

19. Graphic 19
a. Text displayed: Thank you for watching

20. Graphic 20
a. Text displayed showing who has been involved in the teams from LFV and IBM.

21. Graphic 21
a. LFV and IBM logos are shown in the middle of the screen.