Ir directamente a la navegación principal Ir directamente a la búsqueda Ir directamente al contenido principal

FSL-SAGE: Accelerating Federated Split Learning via Smashed Activation Gradient Estimation

  • Srijith Nair
  • , Michael Lin
  • , Peizhong Ju
  • , Amirreza Talebi
  • , Elizabeth Serena Bentley
  • , Jia Liu

Producción científica: Conference article › revisión exhaustiva

4 Citas (Scopus)

Resumen

Collaborative training methods like Federated Learning (FL) and Split Learning (SL) enable distributed machine learning without sharing raw data. However, FL assumes clients can train entire models, which is infeasible for large-scale models. In contrast, while SL alleviates the client memory constraint in FL by offloading most training to the server, it increases network latency due to its sequential nature. Other methods address the conundrum by using local loss functions for parallel client-side training to improve efficiency, but they lack server feedback and potentially suffer poor accuracy. We propose FSL-SAGE (Federated Split Learning via Smashed Activation Gradient Estimation), a new federated split learning algorithm that estimates server-side gradient feedback via auxiliary models. These auxiliary models periodically adapt to emulate server behavior on local datasets. We show that FSL-SAGE achieves a convergence rate of O(1/√T ), where T is the number of communication rounds. This result matches FedAvg, while significantly reducing communication costs and client memory requirements. Our empirical results also verify that it outperforms existing state-of-the-art FSL methods, offering both communication efficiency and accuracy.

Idioma originalEnglish
Páginas (desde-hasta)45507-45528
Número de páginas22
PublicaciónProceedings of Machine Learning Research
Volumen267
EstadoPublished - 2025
Evento42nd International Conference on Machine Learning, ICML 2025 - Vancouver, Canada
Duración: jul 13 2025 → jul 19 2025

Nota bibliográfica

Publisher Copyright:
© 2025, ML Research Press. All rights reserved.

Financiación

This work is supported in part by ONR grant N0001424-1-2729; NSF grants CAREER 2110259, 2112471, and 2324052; DARPA YFA D24AP00265 and DARPA HR001125-2-0019.

FinanciadoresNúmero del financiador
National Science Foundation Arctic Social Science Program2112471, 2110259, 2324052
Defense Advanced Research Projects AgencyYFA D24AP00265, HR001125-2-0019
Office of Naval Research Naval AcademyN0001424-1-2729

    ASJC Scopus subject areas

    • Software
    • Control and Systems Engineering
    • Statistics and Probability
    • Artificial Intelligence

    Huella

    Profundice en los temas de investigación de 'FSL-SAGE: Accelerating Federated Split Learning via Smashed Activation Gradient Estimation'. En conjunto forman una huella única.

    Citar esto