Z. Allen-zhu, Z. Qu, P. Richtárik, and Y. Yuan, Even faster accelerated coordinate descent using non-uniform sampling, International Conference on Machine Learning, pp.1110-1119, 2016.

R. Arratia and L. Gordon, Tutorial on large deviations for the binomial distribution, Bulletin of mathematical biology, vol.51, issue.1, pp.125-131, 1989.

F. Baccelli, G. Cohen, G. J. Olsder, and J. Quadrat, Synchronization and linearity: an algebra for discrete event systems, 1992.

S. Boyd, A. Ghosh, B. Prabhakar, and D. Shah, Randomized gossip algorithms, IEEE transactions on information theory, vol.52, pp.2508-2530, 2006.

S. Boyd, N. Parikh, E. Chu, B. Peleato, and J. Eckstein, Distributed optimization and statistical learning via the alternating direction method of multipliers, Foundations and Trends R in Machine learning, vol.3, issue.1, pp.1-122, 2011.

M. Cao, E. M. Daniel-a-spielman, and . Yeh, Accelerated gossip algorithms for distributed computation, Proc. of the 44th Annual Allerton Conference on Communication, Control, and Computation, pp.952-959, 2006.

S. Chatterjee and E. Seneta, Towards consensus: Some convergence theorems on repeated averaging, Journal of Applied Probability, vol.14, issue.1, pp.89-97, 1977.

H. Morris and . Degroot, Reaching a consensus, Journal of the American Statistical Association, vol.69, issue.345, pp.118-121, 1974.

P. Diaconis, S. Holmes, and R. Neal, Analysis of a nonreversible markov chain sampler, Annals of Applied Probability, pp.726-752, 2000.

R. Diekmann, A. Frommer, and B. Monien, Efficient schemes for nearest neighbor load balancing, Parallel computing, vol.25, issue.7, pp.789-812, 1999.
DOI : 10.1016/s0167-8191(99)00018-6
URL : https://doi.org/10.1016/s0167-8191(99)00018-6

. Alexandros-g-dimakis, D. Anand, M. Sarwate, and . Wainwright, Geographic gossip: efficient aggregation for sensor networks, Proceedings of the 5th international conference on Information processing in sensor networks, pp.69-76, 2006.

S. Alexandros-g-dimakis, . Kar, M. F. José, . Moura, A. Michael-g-rabbat et al., Gossip algorithms for distributed signal processing, Proceedings of the IEEE, vol.98, issue.11, pp.1847-1864, 2010.

A. John-c-duchi, M. Agarwal, and . Wainwright, Dual averaging for distributed optimization: Convergence analysis and network scaling, IEEE Transactions on Automatic control, vol.57, issue.3, pp.592-606, 2012.

O. Fercoq and P. Richtárik, Accelerated, parallel, and proximal coordinate descent, SIAM Journal on Optimization, vol.25, issue.4, pp.1997-2023, 2015.
DOI : 10.1137/130949993
URL : http://epubs.siam.org/doi/pdf/10.1137/130949993

E. Ghadimi, I. Shames, and M. Johansson, Multi-step gradient methods for networked optimization, IEEE Trans. Signal Processing, issue.21, pp.5417-5429, 2013.
DOI : 10.1109/tsp.2013.2278149
URL : http://arxiv.org/pdf/1211.2132

F. Robert-m-gower, P. Hanzely, S. Richtárik, and . Stich, Accelerated stochastic matrix inversion: general theory and speeding up bfgs rules for faster second-order optimization, 2018.

R. Gower and P. Richtárik, Stochastic dual ascent for solving linear systems, 2015.

R. Hannah, F. Feng, and W. Yin, A2BCD: An asynchronous accelerated block coordinate descent algorithm with optimal complexity, 2018.

W. Li, H. Dai, and Y. Zhang, Location-aided fast distributed consensus, IEEE Transactions on Information Theory, 2007.
DOI : 10.1109/tit.2010.2081030
URL : http://arxiv.org/pdf/0707.0500

X. Lian, W. Zhang, C. Zhang, and J. Liu, Asynchronous decentralized parallel stochastic gradient descent, 2017.

J. Liu, . Stephen, and . Wright, Asynchronous stochastic coordinate descent: Parallelism and convergence properties, SIAM Journal on Optimization, vol.25, issue.1, pp.351-376, 2015.
DOI : 10.1137/140961134
URL : http://arxiv.org/pdf/1403.3862

J. Liu, D. O. Brian, M. Anderson, A. Cao, and . Morse, Analysis of accelerated gossip algorithms, Automatica, vol.49, issue.4, pp.873-883, 2013.
DOI : 10.1109/cdc.2009.5399566
URL : https://pure.rug.nl/ws/files/2691129/2009ProcCDCLiu.pdf

J. Liu, J. Stephen, C. Wright, V. Ré, S. Bittorf et al., An asynchronous parallel stochastic coordinate descent algorithm, The Journal of Machine Learning Research, vol.16, issue.1, pp.285-322, 2015.

N. Loizou and P. Richtárik, Accelerated gossip via stochastic heavy ball method, 2018.

B. Mohar, Some applications of laplace eigenvalues of graphs, Graph symmetry, pp.225-275, 1997.
DOI : 10.1007/978-94-015-8937-6_6
URL : http://www.ijp.si/ftp/pub/preprints/ps/97/pp535.ps

A. Mokhtari and A. Ribeiro, Dsa: Decentralized double stochastic averaging gradient algorithm, The Journal of Machine Learning Research, vol.17, issue.1, pp.2165-2199, 2016.
DOI : 10.1109/acssc.2015.7421158

I. Necoara, Y. Nesterov, and F. Glineur, Random block coordinate descent methods for linearly constrained optimization over networks, Journal of Optimization Theory and Applications, vol.173, issue.1, pp.227-254, 2017.
DOI : 10.1007/s10957-016-1058-z
URL : http://arxiv.org/pdf/1504.06340

A. Nedic and A. Ozdaglar, Distributed subgradient methods for multi-agent optimization, IEEE Transactions on Automatic Control, vol.54, issue.1, pp.48-61, 2009.
DOI : 10.1109/tac.2008.2009515
URL : http://web.mit.edu/asuman/www/documents/distributed-journal-final.pdf

A. Nedic, A. Olshevsky, and W. Shi, Achieving geometric convergence for distributed optimization over time-varying graphs, SIAM Journal on Optimization, vol.27, issue.4, pp.2597-2633, 2017.

Y. Nesterov, Efficiency of coordinate descent methods on huge-scale optimization problems, SIAM Journal on Optimization, vol.22, issue.2, pp.341-362, 2012.
DOI : 10.1137/100802001
URL : http://www.uclouvain.be/cps/ucl/doc/core/documents/coredp2010_2web.pdf

Y. Nesterov, Introductory lectures on convex optimization: A basic course, vol.87, 2013.
DOI : 10.1007/978-1-4419-8853-9

Y. Nesterov, . Sebastian, and . Stich, Efficiency of the accelerated coordinate descent method on structured optimization problems, SIAM Journal on Optimization, vol.27, issue.1, pp.110-123, 2017.

. Boris-n-oreshkin, J. Mark, . Coates, and . Rabbat, Optimization and analysis of distributed averaging with short node memory, IEEE Transactions on Signal Processing, vol.58, issue.5, pp.2850-2865, 2010.

. S-sundhar-ram, . Nedi?, and . Veeravalli, Asynchronous gossip algorithms for stochastic optimization, Proceedings of the 48th IEEE Conference on, pp.3581-3586, 2009.

A. S-sundhar-ram, . Nedi?, and . Veeravalli, Distributed stochastic subgradient projection algorithms for convex optimization, Journal of optimization theory and applications, vol.147, issue.3, pp.516-545, 2010.

B. Recht, C. Re, S. Wright, and F. Niu, Hogwild: A lock-free approach to parallelizing stochastic gradient descent, Advances in neural information processing systems, pp.693-701, 2011.

P. Richtárik and M. Taká?, Parallel coordinate descent methods for big data optimization, Mathematical Programming, vol.156, issue.1-2, pp.433-484, 2016.

K. Scaman, F. Bach, S. Bubeck, Y. Lee, and L. Massoulié, Optimal algorithms for smooth and strongly convex distributed optimization in networks, International Conference on Machine Learning, pp.3027-3036, 2017.
URL : https://hal.archives-ouvertes.fr/hal-01478317

W. Shi, Q. Ling, G. Wu, and W. Yin, Extra: An exact first-order algorithm for decentralized consensus optimization, SIAM Journal on Optimization, vol.25, issue.2, pp.944-966, 2015.
DOI : 10.1137/14096668x
URL : http://arxiv.org/pdf/1404.6264

H. Tang, X. Lian, M. Yan, C. Zhang, and J. Liu, Decentralized training over decentralized data, vol.2, 2018.

L. Xiao, S. Boyd, and S. Lall, A scheme for robust distributed sensor fusion based on average consensus, Information Processing in Sensor Networks, 2005. IPSN 2005. Fourth International Symposium on, pp.63-70, 2005.

M. Zinkevich, M. Weimer, L. Li, and A. J. Smola, Parallelized stochastic gradient descent, Advances in neural information processing systems, pp.2595-2603, 2010.