Effect of Experience Sampling Schedules on Response Rate and Recall Accuracy of Objective Self-Reports (bibtex)
by van Berkel, Niels, Goncalves, Jorge, Lovén, Lauri, Ferreira, Denzil, Hosio, Simo and Kostakos, Vassilis
Abstract:
The Experience Sampling Method is widely used to collect human labelled data in the wild. Using this methodology, study participants repeatedly answer a set of questions, constructing a rich overview of the studied phenomena. One of the methodological decisions faced by researchers is deciding on the question scheduling. The literature defines three distinct schedule types: randomised, interval-based, or event-based (in our case, smartphone unlock). However, little evidence exists regarding the side-effects of these schedules on response rate and recall accuracy, and how they may bias study findings. We evaluate the effect of these three contingency configurations in a 3-week within-subjects study (N = 20). Participants answered various objective questions regarding their phone usage, while we simultaneously establish a ground-truth through smartphone instrumentation. We find that scheduling questions on phone unlock yields a higher response rate and accuracy. Our study provides empirical evidence for the effects of notification scheduling on participant responses, and informs researchers who conduct experience sampling studies on smartphones.
Reference:
N. van Berkel, J. Goncalves, L. Lovén, D. Ferreira, S. Hosio, V. Kostakos, "Effect of Experience Sampling Schedules on Response Rate and Recall Accuracy of Objective Self-Reports", International Journal of Human-Computer Studies, vol. 125, 2019, 118-128.
Bibtex Entry:
@article{Berkel2019ESMSchedulesEffect,
	Abstract = {The Experience Sampling Method is widely used to collect human labelled data in the wild. Using this methodology, study participants repeatedly answer a set of questions, constructing a rich overview of the studied phenomena. One of the methodological decisions faced by researchers is deciding on the question scheduling. The literature defines three distinct schedule types: randomised, interval-based, or event-based (in our case, smartphone unlock). However, little evidence exists regarding the side-effects of these schedules on response rate and recall accuracy, and how they may bias study findings. We evaluate the effect of these three contingency configurations in a 3-week within-subjects study (N = 20). Participants answered various objective questions regarding their phone usage, while we simultaneously establish a ground-truth through smartphone instrumentation. We find that scheduling questions on phone unlock yields a higher response rate and accuracy. Our study provides empirical evidence for the effects of notification scheduling on participant responses, and informs researchers who conduct experience sampling studies on smartphones.},
	Author = {van Berkel, Niels and Goncalves, Jorge and Lovén, Lauri and Ferreira, Denzil and Hosio, Simo and Kostakos, Vassilis},
	Doi = {10.1016/j.ijhcs.2018.12.002},
	Journal = {International Journal of Human-Computer Studies},
	Pages = {118-128},
	Title = {Effect of Experience Sampling Schedules on Response Rate and Recall Accuracy of Objective Self-Reports},
	Type = {Journal Article},
	Url = {https://nielsvanberkel.com/files/publications/ijhcs2019a.pdf},
	Volume = {125},
	Year = {2019}}
Powered by bibtexbrowser