Throughput-Optimal Topology Design for Cross-Silo Federated Learning - Inria - Institut national de recherche en sciences et technologies du numérique Accéder directement au contenu
Communication Dans Un Congrès Année : 2020

Throughput-Optimal Topology Design for Cross-Silo Federated Learning

Othmane Marfoq
Chuan Xu
Giovanni Neglia
Richard Vidal
  • Fonction : Auteur
  • PersonId : 1081827

Résumé

Federated learning usually employs a client-server architecture where an orchestrator iteratively aggregates model updates from remote clients and pushes them back a refined model. This approach may be inefficient in cross-silo settings, as close-by data silos with high-speed access links may exchange information faster than with the orchestrator, and the orchestrator may become a communication bottleneck. In this paper we define the problem of topology design for cross-silo federated learning using the theory of max-plus linear systems to compute the system throughput---number of communication rounds per time unit. We also propose practical algorithms that, under the knowledge of measurable network characteristics, find a topology with the largest throughput or with provable throughput guarantees. In realistic Internet networks with 10 Gbps access links for silos, our algorithms speed up training by a factor 9 and 1.5 in comparison to the master-slave architecture and to state-of-the-art MATCHA, respectively. Speedups are even larger with slower access links.
Fichier principal
Vignette du fichier
NIPS2020.pdf (12.18 Mo) Télécharger le fichier
Origine : Fichiers produits par l'(les) auteur(s)
Loading...

Dates et versions

hal-03007834 , version 1 (17-11-2020)
hal-03007834 , version 2 (17-11-2020)

Identifiants

Citer

Othmane Marfoq, Chuan Xu, Giovanni Neglia, Richard Vidal. Throughput-Optimal Topology Design for Cross-Silo Federated Learning. NeurIPS 2020 - 34th Conference on Neural Information Processing Systems, Dec 2020, Vancouver / Online, Canada. ⟨hal-03007834v2⟩
168 Consultations
158 Téléchargements

Altmetric

Partager

Gmail Facebook X LinkedIn More