F. Daniel, P. Kucherbaev, C. Cappiello, B. Benatallah, and M. Allahbakhsh, Quality control in crowdsourcing: A survey of quality attributes, assessment techniques, and assurance actions, ACM Computing Surveys, vol.51, issue.1, p.7, 2018.

A. Ph, A. M. Dawid, and . Skene, Maximum likelihood estimation of observer errorrates using the em algorithm, J. of the Royal Statistical Society: Series C (Applied Statistics), vol.28, issue.1, pp.20-28, 1979.

G. Demartini, D. E. Difallah, and P. Cudré-mauroux, Zencrowd: leveraging probabilistic reasoning and crowdsourcing techniques for large-scale entity linking, Proc. of WWW 2012, pp.469-478, 2012.

A. P. Dempster, N. M. Laird, and D. B. Rubin, Maximum likelihood from incomplete data via the em algorithm, J. of the Royal Statistical Society: Series B (Methodological), vol.39, issue.1, pp.1-22, 1977.

P. A. Flach, Machine Learning -The Art and Science of Algorithms that Make Sense of Data, 2012.

H. Garcia-molina, M. Joglekar, A. Marcus, A. Parameswaran, and V. Verroios, Challenges in data crowdsourcing, Trans. on Knowledge and Data Engineering, vol.28, issue.4, pp.901-911, 2016.

D. R. Karger, S. Oh, and D. Shah, Iterative learning for reliable crowdsourcing systems, Proc. of NIPS'11, pp.1953-1961, 2011.

J. Le, A. Edmonds, V. Hester, and L. Biewald, Ensuring quality in crowdsourced search relevance evaluation: The effects of training question distribution, SIGIR 2010 workshop on crowdsourcing for search evaluation, vol.2126, pp.22-32, 2010.

Q. Li, Y. Li, J. Gao, B. Zhao, W. Fan et al., Resolving conflicts in heterogeneous data by truth discovery and source reliability estimation, Proc. of SIGMOD'14, pp.1187-1198, 2014.

S. G. Nash, Newton-type minimization via the lanczos method, SIAM J. on Numerical Analysis, vol.21, issue.4, pp.770-788, 1984.

B. Pang and L. Lee, A sentimental education: Sentiment analysis using subjectivity summarization based on minimum cuts, Proc. of the 42nd annual meeting on Association for Computational Linguistics, page 271. Association for Computational Linguistics, 2004.

V. Raykar and P. , Sequential crowdsourced labeling as an epsilon-greedy exploration in a markov decision process, Artificial intelligence and statistics, pp.832-840, 2014.

V. C. Raykar, S. Yu, L. H. Zhao, G. H. Valadez, C. Florin et al., Learning from crowds, J. of Machine Learning Research, vol.11, pp.1297-1322, 2010.

L. Tran-thanh, M. Venanzi, A. Rogers, and N. R. Jennings, Efficient budget allocation with accuracy guarantees for crowdsourcing classification tasks, International Foundation for Autonomous Agents and Multiagent Systems, pp.901-908, 2013.

M. Venanzi, J. Guiver, G. Kazai, P. Kohli, and M. Shokouhi, Community-based bayesian aggregation models for crowdsourcing, Proc. of WWW'14, pp.155-164, 2014.

J. Wang, T. Kraska, M. J. Franklin, and J. Feng, Crowder: Crowdsourcing entity resolution, Proc. of the VLDB Endowment, vol.5, pp.1483-1494, 2012.

P. Welinder, S. Branson, P. Perona, and S. J. Belongie, The multidimensional wisdom of crowds, Proc. Of NIPS'10, pp.2424-2432, 2010.

J. Whitehill, T. Wu, J. Bergsma, J. R. Movellan, and P. L. Ruvolo, Whose vote should count more: Optimal integration of labels from labelers of unknown expertise, Proc. of NIPS'09, pp.2035-2043, 2009.

Y. Zheng, G. Li, Y. Li, C. Shan, and R. Cheng, Truth inference in crowdsourcing: Is the problem solved?, Proc. of the VLDB Endowment, vol.10, pp.541-552, 2017.