MD-GAN: Multi-Discriminator Generative Adversarial Networks for Distributed Datasets

Corentin Hardy 1, 2 Erwan Le Merrer 3 Bruno Sericola 2
2 DIONYSOS - Dependability Interoperability and perfOrmance aNalYsiS Of networkS
Inria Rennes – Bretagne Atlantique , IRISA_D2 - RÉSEAUX, TÉLÉCOMMUNICATION ET SERVICES
3 WIDE - the World Is Distributed Exploring the tension between scale and coordination
Inria Rennes – Bretagne Atlantique , IRISA_D1 - SYSTÈMES LARGE ÉCHELLE
Abstract : A recent technical breakthrough in the domain of machine learning is the discovery and the multiple applications of Generative Adversarial Networks (GANs). Those generative models are computationally demanding, as a GAN is composed of two deep neural networks, and because it trains on large datasets. A GAN is generally trained on a single server. In this paper, we address the problem of distributing GANs so that they are able to train over datasets that are spread on multiple workers. MD-GAN is exposed as the first solution for this problem: we propose a novel learning procedure for GANs so that they fit this distributed setup. We then compare the performance of MD-GAN to an adapted version of Federated Learning to GANs, using the MNIST and CIFAR10 datasets. MD-GAN exhibits a reduction by a factor of two of the learning complexity on each worker node, while providing better performances than federated learning on both datasets. We finally discuss the practical implications of distributing GANs.
Complete list of metadatas

https://hal.inria.fr/hal-01946665
Contributor : Corentin Hardy <>
Submitted on : Thursday, May 9, 2019 - 5:07:21 PM
Last modification on : Tuesday, May 14, 2019 - 11:05:18 AM

File

MDGAN_lastVersion (1).pdf
Files produced by the author(s)

Identifiers

  • HAL Id : hal-01946665, version 2

Citation

Corentin Hardy, Erwan Le Merrer, Bruno Sericola. MD-GAN: Multi-Discriminator Generative Adversarial Networks for Distributed Datasets. IPDPS 2019 - 33rd IEEE International Parallel and Distributed Processing Syposium, May 2019, Rio de Janeiro, Brazil. pp.1-12. ⟨hal-01946665v2⟩

Share

Metrics

Record views

74

Files downloads

261