by van Berkel, Niels, Shalawadi, Sujay, R. Evans, Madeleine, Visuri, Aku and Hosio, Simo
Abstract:
While self-report studies are common in Human-Computer Interaction research, few evaluations have assessed their long term use. We present a longitudinal analysis of a web-based workplace application that collects well-being assessments and offers suggestions to improve individual, team, and organisational performance. Our dataset covers 219 users. We assess their first year of application use, focusing on their usage patterns, well-being evaluations, and behaviour towards notifications. Our results highlight that the drop-off in use was the steepest in the first week (-24.2%). However, substantial breaks in usage were common and did not necessarily result in dropout. We found that latency periods of eight days or more predicted a stronger intention to drop out than stay engaged and that reminder notifications did not result in more completed self-reports but significantly prolonged the usage period. Our work strengthens findings related to high drop out rates, but also provides counter-evidence by showing that despite individuals appearing to drop-off in short-term studies, individuals can and do return to self-report applications after extensive breaks. We contribute an analysis of usage behaviour drivers in the area of technology-enabled well-being measurement, responding to the call for longer-term research to extend the growing literature on self-report studies.
Reference:
N. van Berkel, S. Shalawadi, M. R. Evans, A. Visuri, S. Hosio, "A Longitudinal Analysis of Real-World Self-Report Data", in Proceedings of the 19th IFIP TC.13 International Conference on Human-Computer Interaction (INTERACT'23), 2023, to appear.
Bibtex Entry:
@inproceedings{Berkel2023LongSelfReport,
title = {A Longitudinal Analysis of Real-World Self-Report Data},
author = {van Berkel, Niels and Shalawadi, Sujay and R. Evans, Madeleine and Visuri, Aku and Hosio, Simo},
year = 2023,
booktitle = {Proceedings of the 19th IFIP TC.13 International Conference on Human-Computer Interaction},
location = {INTERACT'23},
pages = {to appear},
abstract = {While self-report studies are common in Human-Computer Interaction research, few evaluations have assessed their long term use. We present a longitudinal analysis of a web-based workplace application that collects well-being assessments and offers suggestions to improve individual, team, and organisational performance. Our dataset covers 219 users. We assess their first year of application use, focusing on their usage patterns, well-being evaluations, and behaviour towards notifications. Our results highlight that the drop-off in use was the steepest in the first week (-24.2%). However, substantial breaks in usage were common and did not necessarily result in dropout. We found that latency periods of eight days or more predicted a stronger intention to drop out than stay engaged and that reminder notifications did not result in more completed self-reports but significantly prolonged the usage period. Our work strengthens findings related to high drop out rates, but also provides counter-evidence by showing that despite individuals appearing to drop-off in short-term studies, individuals can and do return to self-report applications after extensive breaks. We contribute an analysis of usage behaviour drivers in the area of technology-enabled well-being measurement, responding to the call for longer-term research to extend the growing literature on self-report studies.},
type = {Conference Paper}
}