Smart Edge-Enabled Traffic Light Control: Improving Reward-Communication Trade-offs with Federated Reinforcement Learning

Nathaniel Hudson, Pratham Oza, Hana Khamfroush, Thidapat Chantem

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

Abstract

Traffic congestion is a costly phenomenon of every-day life. Reinforcement Learning (RL) is a promising solution due to its applicability to solving complex decision-making problems in highly dynamic environments. To train smart traffic lights using RL, large amounts of data is required. Recent RL-based approaches consider training to occur on some nearby server or a remote cloud server. However, this requires that traffic lights all communicate their raw data to some central location. For large road systems, communication cost can be impractical, particularly if traffic lights collect heavy data (e.g., video, LIDAR). As such, this work pushes training to the traffic lights directly to reduce communication cost. However, completely independent learning can reduce the performance of trained models. As such, this work considers the recent advent of Federated Reinforcement Learning (FedRL) for edge-enabled traffic lights so they can learn from each other's experience by periodically aggregating locally-learned policy network parameters rather than share raw data, hence keeping communication costs low. To do this, we propose the SEAL framework which uses an intersection-agnostic representation to support FedRL across traffic lights controlling heterogeneous intersection types. We then evaluate our FedRL approach against Centralized and Decentralized RL strategies. We compare the reward-communication trade-offs of these strategies. Our results show that FedRL is able to reduce the communication costs associated with Centralized training by 36.24%; while only seeing a 2.11 % decrease in average reward (i.e., decreased traffic congestion).

Original languageEnglish
Title of host publicationProceedings - 2022 IEEE International Conference on Smart Computing, SMARTCOMP 2022
Pages40-47
Number of pages8
ISBN (Electronic)9781665481526
DOIs
StatePublished - 2022
Event8th IEEE International Conference on Smart Computing, SMARTCOMP 2022 - Espoo, Finland
Duration: Jun 20 2022Jun 24 2022

Publication series

NameProceedings - 2022 IEEE International Conference on Smart Computing, SMARTCOMP 2022

Conference

Conference8th IEEE International Conference on Smart Computing, SMARTCOMP 2022
Country/TerritoryFinland
CityEspoo
Period6/20/226/24/22

Bibliographical note

Publisher Copyright:
© 2022 IEEE.

Keywords

  • Edge Computing
  • Federated Learning
  • Reinforcement Learning
  • Smart Traffic
  • Traffic Light Control

ASJC Scopus subject areas

  • Artificial Intelligence
  • Computer Science Applications
  • Computer Vision and Pattern Recognition

Fingerprint

Dive into the research topics of 'Smart Edge-Enabled Traffic Light Control: Improving Reward-Communication Trade-offs with Federated Reinforcement Learning'. Together they form a unique fingerprint.

Cite this