M. Abadi, A. Chu, I. Goodfellow, H. B. Mcmahan, I. Mironov et al., Deep Learning with Differential Privacy, Proceedings of the 2016 ACM SIGSAC Conference on Computer and Communications Security, pp.308-318, 2016.

J. Patrick, C. J. Andersen, and . Ras, Algorithms for Euclidean Degree Bounded Spanning Tree Problems, Int. J. Comput. Geometry Appl, vol.29, pp.121-160, 2019.

J. Patrick, C. J. Andersen, and . Ras, Minimum bottleneck spanning trees with degree bounds, Networks 68, vol.4, pp.302-314, 2016.

D. L. Applegate, The Traveling Salesman Problem: A Computational Study, Princeton Series in Applied Mathematics), p.691129932, 2007.

M. Assran, 2019 International Conference on Deep Learning and Machine Learning in Emerging Applications (Deep-ML), Proceedings of the 36th International Conference on Machine Learning, ICML 2019, vol.97, pp.344-353, 2019.

F. Baccelli, Synchronization and Linearity -An Algebra for Discrete Event Systems, The Journal of the Operational Research Society, vol.45, 1994.

A. Bellet, Personalized and Private Peer-to-Peer Machine Learning, 2018.
URL : https://hal.archives-ouvertes.fr/hal-01745796

K. Bonawitz, V. Ivanov, B. Kreuter, A. Marcedone, H. B. Mcmahan et al., Practical Secure Aggregation for Privacy-Preserving Machine Learning, Proceedings of the 2017 ACM SIGSAC Conference on Computer and Communications Security, pp.1175-1191, 2017.

K. Bonawitz, Towards Federated Learning at Scale: System Design, 2019.

R. Bost, R. A. Popa, S. Tu, and S. Goldwasser, Machine Learning Classification over Encrypted Data, Proceedings 2015 Network and Distributed System Security Symposium, vol.4324, p.4325, 2015.

U. Brandes, On variants of shortest-path betweenness centrality and their generic computation, Social Networks, vol.30, issue.2, pp.136-145, 2008.

T. Brunsch, J. Raisch, and L. Hardouin, Modeling and control of high-throughput screening systems, Control Engineering Practice, vol.20, issue.1, pp.14-23, 2012.
URL : https://hal.archives-ouvertes.fr/hal-00846116

S. Caldas, Preprint repository arXiv achieves milestone million uploads, Physics Today, 2014.

S. Caldas, LEAF: A Benchmark for Federated Settings, 2018.

L. Robert, M. E. Carter, and . Crovella, Measuring bottleneck link speed in packet-switched networks, Performance Evaluation, pp.90032-90034, 1996.

V. Chandra, . Zhongdong-huang, and R. Kumar, Automated control synthesis for an assembly line using discrete event system control theory, IEEE Transactions on Systems, Man and Cybernetics, Part C (Applications and Reviews), vol.33, issue.2, pp.284-289, 2003.

K. Cho, B. Van-merrienboer, D. Bahdanau, and Y. Bengio, On the Properties of Neural Machine Translation: Encoder?Decoder Approaches, Proceedings of SSST-8, Eighth Workshop on Syntax, Semantics and Structure in Statistical Translation, pp.103-111, 2014.

I. Colin, Gossip Dual Averaging for Decentralized Optimization of Pairwise Functions, Proceedings of the 33rd International Conference on International Conference on Machine Learning, vol.48, pp.1388-1396, 2016.
URL : https://hal.archives-ouvertes.fr/hal-01329315

P. Courtiol, C. Maussion, M. Moarii, E. Pronier, S. Pilcer et al., Deep learning-based classification of mesothelioma improves prediction of patient outcome, Nature Medicine, vol.25, issue.10, pp.1519-1525, 2019.

A. Dasdan and R. K. Gupta, Faster maximum and minimum mean cycle algorithms for system-performance analysis, IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems, vol.17, issue.10, pp.889-899, 1998.

X. David-henriet, L. Hardouin, J. Raisch, and B. Cottenceau, Holding Time Maximization Preserving Output Performance for Timed Event Graphs, IEEE Transactions on Automatic Control, vol.59, issue.7, pp.1968-1973, 2014.
URL : https://hal.archives-ouvertes.fr/hal-02535574

J. Deng, ImageNet: A Large-Scale Hierarchical Image Database, p.9, 2009.

P. , D. Lorenzo, and G. Scutari, Distributed nonconvex optimization over time-varying networks, 2016 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp.4124-4128, 2016.

J. C. Duchi, A. Agarwal, and M. J. Wainwright, Dual Averaging for Distributed Optimization: Convergence Analysis and Network Scaling, IEEE Transactions on Automatic Control, vol.57, issue.3, pp.592-606, 2012.

N. Farhi, M. Goursat, and J. Quadrat, The traffic phases of road networks, Transportation Research Part C: Emerging Technologies, vol.19, issue.1, pp.85-102, 2011.
URL : https://hal.archives-ouvertes.fr/hal-00860175

M. Fredrikson, S. Jha, and T. Ristenpart, Model Inversion Attacks that Exploit Confidence Information and Basic Countermeasures, Proceedings of the 22nd ACM SIGSAC Conference on Computer and Communications Security - CCS '15, pp.1322-1333, 2015.

M. R. Garey and D. S. Johnson, Computers and Intractability: A Guide to the Theory of NP-Completeness (Series of Books in the Mathematical Sciences). First Edition

S. Gaubert, Resource optimization and (min,+) spectral theory, IEEE Transactions on Automatic Control, vol.40, issue.11, pp.1931-1934, 1995.

A. Go, R. Bhayani, and L. Huang, Twitter Sentiment Classification using Distant Supervision, Processing, pp.1-6, 2009.

M. P. Rob and . Goverde, The max-plus algebra approach to railway timetable design, WIT Transactions on The Built Environment, vol.37, 1998.

B. Gueye, A. Ziviani, M. Crovella, and S. Fdida, Constraint-based geolocation of internet hosts, Proceedings of the 4th ACM SIGCOMM conference on Internet measurement - IMC '04, pp.288-293, 2004.
URL : https://hal.archives-ouvertes.fr/hal-01520105

G. Gutin and . Abraham-p-punnen, The traveling salesman problem and its variations, vol.12, 2006.

A. Aric, D. A. Hagberg, P. J. Schult, and . Swart, Exploring Network Structure, Dynamics, and Function using NetworkX, Proceedings of the 7th Python in Science Conference, pp.11-15, 2008.

K. He, X. Zhang, S. Ren, and J. Sun, Deep Residual Learning for Image Recognition, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp.770-778, 2016.

M. Himsolt, GML: A portable graph file format, 1997.

S. Hochreiter and J. Schmidhuber, Long Short-Term Memory, Neural Computation 9, vol.8, pp.1735-1780, 1997.

K. Hsieh, Gaia: Geo-Distributed Machine Learning Approaching LAN Speeds, Proceedings of the 14th USENIX Conference on Networked Systems Design and Implementation. NSDI'17, p.9781931971379, 2017.

M. Jain and C. Dovrolis, End-to-end available bandwidth, ACM SIGCOMM Computer Communication Review, vol.32, issue.4, pp.295-308, 2002.

, Commun. Rev, vol.32, pp.295-308, 2002.

Z. Jiang, Neural Information Processing Systems, The Deep Learning Revolution, pp.5904-5914, 2018.

P. Kairouz, Advances and Open Problems in Federated Learning, 2019.
URL : https://hal.archives-ouvertes.fr/hal-02406503

J. Kang, Z. Xiong, D. Niyato, H. Yu, Y. Liang et al., Incentive Design for Efficient Federated Learning in Mobile Networks: A Contract Theory Approach, 2019 IEEE VTS Asia Pacific Wireless Communications Symposium (APWCS), 2019.

J. J. Karaganis, On the Cube of a Graph, Canadian Mathematical Bulletin, vol.11, issue.2, pp.295-296, 1968.

C. Karakus, Straggler Mitigation in Distributed Optimization Through Data Encoding, Proc. of NIPS, pp.5434-5442, 2017.

. Sai-praneeth-karimireddy, Stochastic Controlled Averaging for Federated Learning. 2019

R. M. Karp, A characterization of the minimum cycle mean in a digraph, Discrete Mathematics, vol.23, issue.3, pp.309-311, 1978.

P. Kathiravelu, M. Chiesa, P. Marcos, M. Canini, and L. Veiga, Moving Bits with a Fleet of Shared Virtual Routers, 2018 IFIP Networking Conference (IFIP Networking) and Workshops, pp.1-9, 2018.

S. Knight, H. X. Nguyen, N. Falkner, R. Bowden, and M. Roughan, The Internet Topology Zoo, IEEE Journal on Selected Areas in Communications, vol.29, issue.9, pp.1765-1775, 2011.

A. Koloskova, S. Stich, and M. Jaggi, Decentralized Stochastic Optimization and Gossip Algorithms with Compressed Communication, Proceedings of the 36th International Conference on Machine Learning (ICML), vol.97, pp.3478-3487, 2019.

A. Koloskova, A Unified Theory of Decentralized SGD with Changing Topology and Local Updates, 2020.

J. Kone?ný, B. Mcmahan, and D. Ramage, Federated Optimization:Distributed Optimization Beyond the Datacenter, 8th NIPS Workshop on Optimization for Machine Learning (OPT15), 2015.

J. Konecný, Weniger IV-Bezüger dank Eidgenössischem Foliensatz: Wie das BSV die Ärzteschaft bilden will, Schweizerische Ärztezeitung, vol.84, issue.48, pp.02527-02527, 2003.

A. Lalitha, (11173) Jayanderson, Dictionary of Minor Planet Names, pp.755-755

J. Boudec and P. Thiran, Network Calculus, p.354042184, 2001.

S. Li, S. M. Mousavi-kalan, A. S. Avestimehr, and M. Soltanolkotabi, Near-Optimal Straggler Mitigation for Distributed Gradient Methods, 2018 IEEE International Parallel and Distributed Processing Symposium Workshops (IPDPSW), 2018.

T. Li, A. K. Sahu, A. Talwalkar, and V. Smith, Federated Learning: Challenges, Methods, and Future Directions, IEEE Signal Processing Magazine, vol.37, issue.3, pp.50-60, 2020.

T. Li, Federated Optimization in Heterogeneous Networks, Proceedings of the 3rd MLSys Conference, 2020.

X. Li, Communication-Efficient Local Decentralized SGD Methods, arXiv: Machine Learning, 2019.

A. Liakopoulos, B. Maglaris, C. Bouras, and A. Sevasti, Providing and verifying advanced IP services in hierarchical DiffServ networks-the case of GEANT, International Journal of Communication Systems, vol.17, issue.4, pp.321-336, 2004.

X. Lian, Asynchronous Decentralized Parallel Stochastic Gradient Descent, Proceedings of the 35th International Conference on Machine Learning, vol.80, pp.3043-3052, 2018.

X. Lian, Can Decentralized Algorithms Outperform Centralized Algorithms? A Case Study for Decentralized Parallel Stochastic Gradient Descent, Advances in Neural Information Processing Systems, vol.30, pp.5330-5340, 2017.

L. Xiao and S. Boyd, Fast linear iterations for distributed averaging, 42nd IEEE International Conference on Decision and Control, vol.5, pp.4997-5002, 2003.

J. Lin, Divergence measures based on the Shannon entropy, IEEE Transactions on Information Theory, vol.37, issue.1, pp.145-151, 1991.

T. Lin, Don't Use Large Mini-batches, Use Local SGD, International Conference on Learning Representations. 2020

S. Liu and B. Li, <italic>Stemflow:</italic> Software-Defined Inter-Datacenter Overlay as a Service, IEEE Journal on Selected Areas in Communications, vol.35, issue.11, pp.2563-2573, 2017.

Q. Luo, J. Lin, Y. Zhuo, and X. Qian, Hop, Proceedings of the Twenty-Fourth International Conference on Architectural Support for Programming Languages and Operating Systems, pp.893-907, 2019.

O. Zamazal, Machine Learning Support for EU Funding Project Categorization, The Computer Journal, vol.62, issue.11, pp.1684-1694, 2019.

R. Mahajan, N. Spring, D. Wetherall, and T. Anderson, Inferring link weights using end-to-end measurements, Proceedings of the second ACM SIGCOMM Workshop on Internet measurment - IMW '02, 2002.

G. H. Lee and S. Shin, Federated Learning on Clinical Benchmark Data: Performance Assessment (Preprint), 2020.

S. Marcel and Y. Rodriguez, Torchvision the Machine-Vision Package of Torch, Proceedings of the 18th ACM International Conference on Multimedia. MM '10, pp.1485-1488, 2010.

L. Massoulie, Randomized Decentralized Broadcasting Algorithms, Proceedings of the IEEE INFOCOM 2007 -26th IEEE International Conference on Computer Communications, pp.1073-1081, 2007.

B. Mcmahan, Communication-Efficient Learning of Deep Networks from Decentralized Data, Proceedings of the 20th International Conference on Artificial Intelligence and Statistics, AISTATS 2017, vol.54, pp.1273-1282, 2017.

J. Monnot, V. T. Paschos, and S. Toulouse, Approximation algorithms for the traveling salesman problem, Mathematical Methods of Operations Research (ZOR), vol.56, issue.3, pp.387-405, 2003.
URL : https://hal.archives-ouvertes.fr/hal-00003997

. Musketeer and . Url,

A. Nedic, A. Olshevsky, and M. G. Rabbat, Network Topology and Communication-Computation Tradeoffs in Decentralized Optimization, Proceedings of the IEEE, vol.106, issue.5, pp.953-976, 2018.

A. Nedi?, A. Olshevsky, and W. Shi, Achieving Geometric Convergence for Distributed Optimization Over Time-Varying Graphs, SIAM Journal on Optimization, vol.27, issue.4, pp.2597-2633, 2017.

A. Nedic and A. E. Ozdaglar, Distributed Subgradient Methods for Multi-Agent Optimization, IEEE Transactions on Automatic Control, vol.54, issue.1, pp.48-61, 2009.

G. Neglia, G. Calbi, D. Towsley, and G. Vardoyan, The Role of Network Topology for Distributed Machine Learning, IEEE INFOCOM 2019 - IEEE Conference on Computer Communications, pp.2350-2358, 2019.
URL : https://hal.archives-ouvertes.fr/hal-02411164

G. Neglia, Lisan Al-Hal: Jurnal Pengembangan Pemikiran dan Kebudayaan, In: AISTATS 2020 -23rd International Conference on Artificial Intelligence and Statistics

V. Nikolaenko, Privacy-preserving ridge regression on hundreds of millions of records, 2013 IEEE Symposium on Security and Privacy, pp.334-348, 2013.

A. Olshevsky, I. Ch, S. Paschalidis, and . Pu, Asymptotic Network Independence in Distributed Optimization for Machine Learning, 2019.

J. Pennington, R. Socher, and C. Manning, Glove: Global Vectors for Word Representation, Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), vol.14, pp.1532-1543, 2014.

. Valerio-persico, On the performance of the wide-area networks interconnecting publiccloud datacenters around the globe, Computer Networks, vol.112, pp.1389-1286, 2017.

R. Prosad, C. Davrolis, M. Murray, and K. C. Claffy, Bandwidth estimation: metrics, measurement techniques, and tools, IEEE Network, vol.17, issue.6, pp.27-35, 2003.

R. C. Prim, Shortest Connection Networks And Some Generalizations, Bell System Technical Journal, vol.36, issue.6, pp.1389-1401, 1957.

A. Shi-pu, I. Olshevsky, . Ch, and . Paschalidis, Asymptotic Network Independence in Distributed Stochastic Optimization for Machine Learning: Examining Distributed and Centralized Stochastic Gradient Descent, IEEE Signal Process. Mag, vol.37, pp.114-122, 2020.

S. Sundhar-ram, A. Nedi?, and V. V. Veeravalli, A new class of distributed optimization algorithms: application to regression of distributed data, Optimization Methods and Software, vol.27, issue.1, pp.71-88, 2012.

F. Sattler, S. Wiedemann, K. Muller, and W. Samek, Robust and Communication-Efficient Federated Learning From Non-i.i.d. Data, IEEE Transactions on Neural Networks and Learning Systems, vol.31, issue.9, pp.3400-3413, 2020.

K. Scaman, Optimal algorithms for non-smooth distributed optimization in networks, Advances in Neural Information Processing Systems, pp.2740-2749, 2018.
URL : https://hal.archives-ouvertes.fr/hal-01957013

K. Seaman, Optimal algorithms for smooth and strongly convex distributed optimization in networks, Proceedings of the 34th International Conference on Machine Learning, vol.70, pp.3027-3036, 2017.

W. Shi, Q. Ling, G. Wu, and W. Yin, EXTRA: An Exact First-Order Algorithm for Decentralized Consensus Optimization, SIAM Journal on Optimization, vol.25, issue.2, pp.944-966, 2015.

R. Shokri, M. Stronati, C. Song, and V. Shmatikov, Membership Inference Attacks Against Machine Learning Models, 2017 IEEE Symposium on Security and Privacy (SP), pp.3-18, 2017.

S. Silva, B. A. Gutman, E. Romero, P. M. Thompson, A. Altmann et al., Federated Learning in Distributed Medical Databases: Meta-Analysis of Large-Scale Subcortical Brain Data, 2019 IEEE 16th International Symposium on Biomedical Imaging (ISBI 2019), pp.270-274, 2019.
URL : https://hal.archives-ouvertes.fr/hal-01895800

N. Spring, R. Mahajan, D. Wetherall, and T. Anderson, Measuring ISP Topologies With Rocketfuel, IEEE/ACM Transactions on Networking, vol.12, issue.1, pp.2-16, 2004.

H. Tang, D 2 : Decentralized Training over Decentralized Data, Proceedings of the 35th International Conference on Machine Learning, vol.80, pp.4848-4856, 2018.

A. The, . Cloud-in-north, and . America, AWS Global Infrastructure, pp.151-160, 2019.

N. H. Tran, W. Bao, A. Zomaya, M. N. Nguyen, and C. S. Hong, Federated Learning over Wireless Networks: Optimization Model Design and Analysis, IEEE INFOCOM 2019 - IEEE Conference on Computer Communications, pp.1387-1395, 2019.

K. I. Tsianos, S. Lawlor, and M. G. Rabbat, Consensus-based distributed optimization: Practical issues and applications in large-scale machine learning, 2012 50th Annual Allerton Conference on Communication, Control, and Computing (Allerton), pp.1543-1550, 2012.

G. Van-horn, O. Mac-aodha, Y. Song, Y. Cui, C. Sun et al., The iNaturalist Species Classification and Detection Dataset, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp.8769-8778, 2018.

P. Vanhaesebrouck, A. Bellet, and M. Tommasi, Decentralized Collaborative Learning of Personalized Models over Networks, 2017.
URL : https://hal.archives-ouvertes.fr/hal-01533182

H. Wang, Neural Information Processing Systems, The Deep Learning Revolution, pp.9850-9861, 2018.

J. Wang, H. Liang, and G. Joshi, Overlap Local-SGD: An Algorithmic Approach to Hide Communication Delays in Distributed SGD, ICASSP 2020 - 2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2020.

J. Wang and G. Joshi, Cooperative SGD: A unified Framework for the Design and Analysis of Communication-Efficient SGD Algorithms, ICML Workshop, 2019.

J. Wang, A. K. Sahu, Z. Yang, G. Joshi, and S. Kar, MATCHA: Speeding Up Decentralized SGD via Matching Decomposition Sampling, 2019 Sixth Indian Control Conference (ICC), 2019.

J. Wang, Improving Communication-Efficient Distributed SGD with Slow Momentum, 2019.

S. Wang, T. Tuor, T. Salonidis, K. K. Leung, C. Makaya et al., Adaptive Federated Learning in Resource Constrained Edge Computing Systems, IEEE Journal on Selected Areas in Communications, vol.37, issue.6, pp.1205-1221, 2019.

. Webank, https://kujss.iraqjournals.com/pdf_166170_8dd024058ce4abb6c364bec514cecef8.html, Kirkuk University Journal-Scientific Studies, vol.15, issue.2, pp.1-16, 2020.

B. Woodworth, Is Local SGD Better than Minibatch SGD? 2020

J. Zeng-yin and W. , Extrapush for Convex Smooth Decentralized Optimization Over Directed Networks, Journal of Computational Mathematics, vol.35, issue.4, pp.1991-7139, 2017.

K. Yuan, B. Ying, X. Zhao, and A. H. Sayed, Exact Diffusion for Distributed Optimization and Learning?Part I: Algorithm Development, IEEE Transactions on Signal Processing, vol.67, issue.3, pp.708-723, 2019.

H. Zhang, ZipML: Training Linear Models with End-to-End Low Precision, and a Little Bit of Deep Learning, Proceedings of the 34th International Conference on Machine Learning, vol.70, pp.4035-4043, 2017.