QoS-aware edge AI placement and scheduling with multiple implementations in FaaS-based edge computing

Nathaniel Hudson, Hana Khamfroush, Matt Baughman, Daniel E. Lucani, Kyle Chard, Ian Foster

Research output: Contribution to journalArticlepeer-review

Abstract

Resource constraints on the computing continuum require that we make smart decisions for serving AI-based services at the network edge. AI-based services typically have multiple implementations (e.g., image classification implementations include SqueezeNet, DenseNet, and others) with varying trade-offs (e.g., latency and accuracy). The question then is how should AI-based services be placed across Function-as-a-Service (FaaS) based edge computing systems in order to maximize total Quality-of-Service (QoS). To address this question, we propose a problem that jointly aims to solve (i) edge AI service placement and (ii) request scheduling. These are done across two time-scales (one for placement and one for scheduling). We first cast the problem as an integer linear program. We then decompose the problem into separate placement and scheduling subproblems and prove that both are NP-hard. We then propose a novel placement algorithm that places services while considering device-to-device communication across edge clouds to offload requests to one another. Our results show that the proposed placement algorithm is able to outperform a state-of-the-art placement algorithm for AI-based services, and other baseline heuristics, with regard to maximizing total QoS. Additionally, we present a federated learning-based framework, FLIES, to predict the future incoming service requests and their QoS requirements. Our results also show that our FLIES algorithm is able to outperform a standard decentralized learning baseline for predicting incoming requests and show comparable predictive performance when compared to centralized training.

Original languageEnglish
Pages (from-to)250-263
Number of pages14
JournalFuture Generation Computer Systems
Volume157
DOIs
StatePublished - Aug 2024

Bibliographical note

Publisher Copyright:
© 2024

Keywords

  • Edge intelligence
  • Federated learning
  • Quality-of-service
  • Serverless edge computing
  • Service placement

ASJC Scopus subject areas

  • Software
  • Hardware and Architecture
  • Computer Networks and Communications

Fingerprint

Dive into the research topics of 'QoS-aware edge AI placement and scheduling with multiple implementations in FaaS-based edge computing'. Together they form a unique fingerprint.

Cite this