Context-Informed Scheduling and Analysis: Improving Accuracy of Mobile Self-Reports (bibtex)
by van Berkel, Niels, Goncalves, Jorge, Koval, Peter, Hosio, Simo, Dingler, Tilman, Ferreira, Denzil and Kostakos, Vassilis
Abstract:
Mobile self-reports are a popular technique to collect participant labelled data in the wild. While literature has focused on increasing participant compliance to self-report questionnaires, relatively little work has assessed response accuracy. In this paper, we investigate how participant context can affect response accuracy and help identify strategies to improve the accuracy of mobile self-report data. In a 3-week study we collect over 2,500 questionnaires containing both verifiable and non-verifiable questions. We find that response accuracy is higher for questionnaires that arrive when the phone is not in ongoing or very recent use. Furthermore, our results show that long completion times are an indicator of a lower accuracy. Using contextual mechanisms readily available on smartphones, we are able to explain up to 13% of the variance in participant accuracy. We offer actionable recommendations to assist researchers in their future deployments of mobile self-report studies.
Reference:
N. van Berkel, J. Goncalves, P. Koval, S. Hosio, T. Dingler, D. Ferreira, V. Kostakos, "Context-Informed Scheduling and Analysis: Improving Accuracy of Mobile Self-Reports", in Proceedings of ACM SIGCHI Conference on Human Factors in Computing Systems (CHI'19), 2019, 51:1-51:12.
Bibtex Entry:
@inproceedings{Berkel2019ESMSchedulingAccuracy,
	Abstract = {Mobile self-reports are a popular technique to collect participant labelled data in the wild. While literature has focused on increasing participant compliance to self-report questionnaires, relatively little work has assessed response accuracy. In this paper, we investigate how participant context can affect response accuracy and help identify strategies to improve the accuracy of mobile self-report data. In a 3-week study we collect over 2,500 questionnaires containing both verifiable and non-verifiable questions. We find that response accuracy is higher for questionnaires that arrive when the phone is not in ongoing or very recent use. Furthermore, our results show that long completion times are an indicator of a lower accuracy. Using contextual mechanisms readily available on smartphones, we are able to explain up to 13% of the variance in participant accuracy. We offer actionable recommendations to assist researchers in their future deployments of mobile self-report studies.},
	Author = {van Berkel, Niels and Goncalves, Jorge and Koval, Peter and Hosio, Simo and Dingler, Tilman and Ferreira, Denzil and Kostakos, Vassilis},
	Booktitle = {Proceedings of ACM SIGCHI Conference on Human Factors in Computing Systems},
	Doi = {10.1145/3290605.3300281},
	Location = {CHI'19},
	Pages = {51:1-51:12},
	Title = {Context-Informed Scheduling and Analysis: Improving Accuracy of Mobile Self-Reports},
	Type = {Conference Paper},
	Url = {https://nielsvanberkel.com/files/publications/chi2019a.pdf},
	Year = {2019}}
Powered by bibtexbrowser