D. Arthur and S. Vassilvitskii, K-means++: the advantages of careful seeding, SODA. Society for Industrial and Applied Mathematics, 2007.

H. William, T. Beluch, A. Genewein, J. M. Nürnberger, and . Köhler, The power of ensembles for active learning in image classification, CVPR, 2018.

D. Berthelot, N. Carlini, I. Goodfellow, N. Papernot, A. Oliver et al., Mixmatch: A holistic approach to semi-supervised learning, 2019.

P. Bojanowski and A. Joulin, Unsupervised learning by predicting noise, ICML, 2017.

M. Caron, P. Bojanowski, A. Joulin, and M. Douze, Deep clustering for unsupervised learning of visual features, 2018.

O. Chapelle, B. Scholkopf, and A. Zien, Semi-Supervised Learning, 2006.

B. Cheng, Y. Wei, H. Shi, S. Chang, J. Xiong et al., Revisiting pre-training: An efficient training method for image classification, 2018.

K. Chitta, M. Jose, A. Alvarez, and . Lesnikowski, Large-scale visual active learning with deep probabilistic ensembles, 2019.

C. Doersch, A. Gupta, and A. A. Efros, Unsupervised visual representation learning by context prediction, ICCV, 2015.

M. Ducoffe and F. Precioso, Adversarial active learning for deep networks: a margin based approach, 2018.

Y. Gal, R. Islam, and Z. Ghahramani, Deep bayesian active learning with image data, 2017.

Y. Geifman and R. El-yaniv, Deep active learning over the long tail, 2017.

S. Gidaris, P. Singh, and N. Komodakis, Unsupervised representation learning by predicting image rotations, In ICLR, 2018.
URL : https://hal.archives-ouvertes.fr/hal-01864755

D. Gissin and S. Shalev-shwartz, Discriminative active learning, 2018.

A. Iscen, G. Tolias, Y. Avrithis, and O. Chum, Label propagation for deep semi-supervised learning, CVPR, 2019.
URL : https://hal.archives-ouvertes.fr/hal-02370297

A. Katharopoulos and F. Fleuret, Biased importance sampling for deep neural network training, 2017.

A. Krizhevsky, Learning multiple layers of features from tiny images, 2009.

S. Laine and T. Aila, Temporal ensembling for semisupervised learning, 2016.

S. Laine and T. Aila, Temporal ensembling for semisupervised learning, ICLR, 2017.

Y. Lecun, L. Bottou, Y. Bengio, and P. Haffner, Gradient-based learning applied to document recognition, Proceedings of the IEEE, vol.86, issue.11, pp.2278-2324, 1998.

Y. Li, Y. L. Wang, D. Yu, Y. Ning, P. Hu et al., Ascent: Active supervision for semisupervised learning, IEEE Transactions on Knowledge and Data Engineering, 2019.

J. Long, J. Yin, W. Zhao, and E. Zhu, Graphbased active learning based on label propagation, International Conference on Modeling Decisions for Artificial Intelligence, pp.179-190, 2008.

I. Loshchilov and F. Hutter, Sgdr: Stochastic gradient descent with warm restarts, ICLR, 2017.

C. Mayer and R. Timofte, Adversarial sampling for active learning, 2018.

M. Mccallum and K. Nigam, Employing em in pool-based active learning for text classification, ICML, 1998.

I. Muslea, S. Minton, and C. A. Knoblock, Active+ semi-supervised learning = robust multi-view learning, ICML, 2002.

Y. Netzer, T. Wang, A. Coates, A. Bissacco, B. Wu et al., Reading digits in natural images with unsupervised feature learning, NIPS Workshop on Deep Learning and Unsupervised Feature Learning, 2011.

M. Noroozi and P. Favaro, Unsupervised learning of visual representations by solving jigsaw puzzles, ECCV, 2016.

S. Sylvestre-alvise-rebuffi, K. Ehrhardt, A. Han, A. Vedaldi, and . Zisserman, Semisupervised learning with scarce annotations, 2019.

O. Sener and S. Savarese, Active learning for convolutional neural networks: A core-set approach, 2018.

B. Settles, Active learning literature survey, 2009.

S. Sinha, S. Ebrahimi, and T. Darrell, Variational adversarial active learning, 2019.

A. Tarvainen and H. Valpola, Mean teachers are better role models: Weight-averaged consistency targets improve semi-supervised deep learning results, NIPS, 2017.

V. Verma, A. Lamb, J. Kannala, Y. Bengio, and D. Lopez-paz, Interpolation consistency training for semi-supervised learning, 2019.

K. Wang, D. Zhang, Y. Li, R. Zhang, and L. Lin, Cost-effective active learning for deep image classification, IEEE Trans. CSVT, vol.27, issue.12, pp.2591-2600, 2017.

X. Wang and A. Gupta, Unsupervised learning of visual representations using videos, ICCV, 2015.

W. Yang, L. Jin, D. Tao, Z. Xie, and Z. Feng, Dropsample: A new training method to enhance deep convolutional neural networks for large-scale unconstrained handwritten chinese character recognition, 2015.

D. Zhou, O. Bousquet, J. Thomas-navin-lal, B. Weston, and . Schölkopf, Learning with local and global consistency, NIPS, 2003.

D. Zhou, J. Weston, A. Gretton, O. Bousquet, and B. Schölkopf, Ranking on data manifolds, NIPS, 2003.

Z. Zhou, K. Chen, and Y. Jiang, Exploiting unlabeled data in content-based image retrieval, ECML, 2004.

X. Zhu and Z. Ghahramani, Learning from labeled and unlabeled data with label propagation, 2002.

X. Zhu, J. Lafferty, and Z. Ghahramani, Combining active learning and semi-supervised learning using gaussian fields and harmonic functions, ICML 2003 workshop on the continuum from labeled to unlabeled data in machine learning and data mining, 2003.

, CoreSet

, CoreSet

, Average accuracy and standard deviation for different label budget b and cycle on MNIST and SVHN. Following Algorithm 1, we show the effect of unsupervised pre-training (PRE) and semi-supervised learning (SEMI) compared to the standard baseline, Table 5