Effect of Cognitive Abilities on Crowdsourcing Task Performance (bibtex)
by Hettiachchi, Danula, van Berkel, Niels, Hosio, Simo, Kostakos, Vassilis and Goncalves, Jorge
Abstract:
Matching crowd workers to suitable tasks is highly desirable as it can enhance task performance, reduce the cost for requesters, and increase worker satisfaction. In this paper, we propose a method that considers workers' cognitive ability to predict their suitability for a wide range of crowdsourcing tasks. We measure cognitive ability via fast-paced online cognitive tests with a combined average duration of 6.2 minutes. We then demonstrate that our proposed method can effectively assign or recommend workers to five different popular crowd tasks: Classification, Counting, Proofreading, Sentiment Analysis, and Transcription. Using our approach we demonstrate a significant improvement in the expected overall task accuracy. While previous methods require access to worker history or demographics, our work offers a quick and accurate way to determine which workers are more suitable for which tasks.
Reference:
D. Hettiachchi, N. van Berkel, S. Hosio, V. Kostakos, J. Goncalves, "Effect of Cognitive Abilities on Crowdsourcing Task Performance", in Proceedings of the 17th IFIP TC.13 International Conference on Human-Computer Interaction (INTERACT'19), 2019, 442-464.
Bibtex Entry:
@inproceedings{Hettiachchi2019CognitiveCrowdsourcing,
	Abstract = {Matching crowd workers to suitable tasks is highly desirable as it can enhance task performance, reduce the cost for requesters, and increase worker satisfaction. In this paper, we propose a method that considers workers' cognitive ability to predict their suitability for a wide range of crowdsourcing tasks. We measure cognitive ability via fast-paced online cognitive tests with a combined average duration of 6.2 minutes. We then demonstrate that our proposed method can effectively assign or recommend workers to five different popular crowd tasks: Classification, Counting, Proofreading, Sentiment Analysis, and Transcription. Using our approach we demonstrate a significant improvement in the expected overall task accuracy. While previous methods require access to worker history or demographics, our work offers a quick and accurate way to determine which workers are more suitable for which tasks.},
	Author = {Hettiachchi, Danula and van Berkel, Niels and Hosio, Simo and Kostakos, Vassilis and Goncalves, Jorge},
	Booktitle = {Proceedings of the 17th IFIP TC.13 International Conference on Human-Computer Interaction},
	Doi = {10.1007/978-3-030-29381-9_28},
	Location = {INTERACT'19},
	Pages = {442-464},
	Title = {Effect of Cognitive Abilities on Crowdsourcing Task Performance},
	Type = {Conference Paper},
	Url = {https://nielsvanberkel.com/files/publications/interact2019a.pdf},
	Year = {2019}}
Powered by bibtexbrowser