A Deep Ensemble Method for Multi-Agent Reinforcement Learning: A Case Study on Air Traffic Control

Supriyo Ghosh, Sean Laguna, Shiau Hong Lim, Laura Wynter, Hasan Poonawala

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

3 Scopus citations

Abstract

Reinforcement learning (RL), a promising framework for data-driven decision making in an uncertain environment, has successfully been applied in many real-world operation and control problems. However, the application of RL in a large-scale decentralized multi-agent environment remains a challenging problem due to the partial observability and limited communications between agents. In this paper, we develop a model-based kernel RL approach and a model-free deep RL approach for learning a decentralized, shared policy among homogeneous agents. By leveraging the strengths of both these methods, we further propose a novel deep ensemble multi-agent reinforcement learning (MARL) method that efficiently learns to arbitrate between the decisions of the local kernel-based RL model and the wider-reaching deep RL model. We validate the proposed deep ensemble method on a highly challenging real-world air traffic control problem, where the goal is to provide effective guidance to aircraft to avoid air traffic congestion, conflicting situations, and to improve arrival timeliness, by dynamically recommending adjustments of aircraft speeds in real-time. Extensive empirical results from an open-source air traffic management simulation model, developed by Eurocontrol and built on a real-world data set including thousands of aircrafts, demonstrate that our proposed deep ensemble MARL method significantly outperforms three state-of-the-art benchmark approaches.

Original languageEnglish
Title of host publication31st International Conference on Automated Planning and Scheduling, ICAPS 2021
EditorsSusanne Biundo, Minh Do, Robert Goldman, Michael Katz, Qiang Yang, Hankz Hankui Zhuo
Pages468-476
Number of pages9
ISBN (Electronic)9781713832317
StatePublished - 2021
Event31st International Conference on Automated Planning and Scheduling, ICAPS 2021 - Guangzhou, Virtual, China
Duration: Aug 2 2021Aug 13 2021

Publication series

NameProceedings International Conference on Automated Planning and Scheduling, ICAPS
Volume2021-August
ISSN (Print)2334-0835
ISSN (Electronic)2334-0843

Conference

Conference31st International Conference on Automated Planning and Scheduling, ICAPS 2021
Country/TerritoryChina
CityGuangzhou, Virtual
Period8/2/218/13/21

Bibliographical note

Publisher Copyright:
Copyright © 2021, Association for the Advancement of Artificial Intelligence (www.aaai.org). All rights reserved.

ASJC Scopus subject areas

  • Artificial Intelligence
  • Computer Science Applications
  • Information Systems and Management

Fingerprint

Dive into the research topics of 'A Deep Ensemble Method for Multi-Agent Reinforcement Learning: A Case Study on Air Traffic Control'. Together they form a unique fingerprint.

Cite this