Gossiping GANs

Corentin Hardy 1, 2 Erwan Le Merrer 3 Bruno Sericola 2
2 DIONYSOS - Dependability Interoperability and perfOrmance aNalYsiS Of networkS
Inria Rennes – Bretagne Atlantique , IRISA-D2 - RÉSEAUX, TÉLÉCOMMUNICATION ET SERVICES
3 WIDE - the World Is Distributed Exploring the tension between scale and coordination
Inria Rennes – Bretagne Atlantique , IRISA-D1 - SYSTÈMES LARGE ÉCHELLE
Abstract : A recently celebrated kind of deep neural networks is Generative Adversarial Networks. GANs are generators of samples from a distribution that has been learned; they are up to now centrally trained from local data on a single location. We question the performance of training GANs using a spread dataset over a set of distributed machines, using a gossip approach shown to work on standard neural networks [1]. This performance is compared to the federated learning distributed method, that has the drawback of sending model data to a server. We also propose a gossip variant, where GAN components are gossiped independently. Experiments are conducted with Tensorflow with up to 100 emulated machines, on the canonical MNIST dataset. The position of this paper is to provide a first evidence that gossip performances for GAN training are close to the ones of federated learning, while operating in a fully decentralized setup. Second, to highlight that for GANs, the distribution of data on machines is critical (i.e., i.i.d. or not). Third, to illustrate that the gossip variant, despite proposing data diversity to the learning phase, brings only marginal improvements over the classic gossip approach.
Complete list of metadatas

https://hal.inria.fr/hal-01923742
Contributor : Corentin Hardy <>
Submitted on : Thursday, November 15, 2018 - 2:02:19 PM
Last modification on : Friday, September 13, 2019 - 9:51:33 AM

Identifiers

Citation

Corentin Hardy, Erwan Le Merrer, Bruno Sericola. Gossiping GANs. DIDL 2018 - Second Workshop on Distributed Infrastructures for Deep Learning, Dec 2018, Rennes, France. ⟨10.1145/3286490.3286563⟩. ⟨hal-01923742⟩

Share

Metrics

Record views

110