Effect of Information Presentation on Fairness Perceptions of Machine Learning Predictors (bibtex)
by van Berkel, Niels, Goncalves, Jorge, Russo, Daniel, Hosio, Simo and Skov, Mikael B.
Abstract:
The uptake of artificial intelligence-based applications raises concerns about the fairness and transparency of AI behaviour. Consequently, the Computer Science community calls for the involvement of the general public in the design and evaluation of AI systems. Assessing the fairness of individual predictors is an essential step in the development of equitable algorithms. In this study, we evaluate the effect of two common visualisation techniques (text-based and scatterplot) and the display of the outcome information (i.e., ground-truth) on the perceived fairness of predictors. Our results from an online crowdsourcing study (N = 80) show that the chosen visualisation technique significantly alters people's fairness perception and that the presented scenario, as well as the participant's gender and past education, influence perceived fairness. Based on these results we draw recommendations for future work that seeks to involve non-experts in AI fairness evaluations.
Reference:
N. van Berkel, J. Goncalves, D. Russo, S. Hosio, M. B. Skov, "Effect of Information Presentation on Fairness Perceptions of Machine Learning Predictors", in Proceedings of ACM SIGCHI Conference on Human Factors in Computing Systems (CHI'21), 2021, to appear.
Bibtex Entry:
@inproceedings{Berkel2021VisualisationAI,
	Abstract = {The uptake of artificial intelligence-based applications raises concerns about the fairness and transparency of AI behaviour. Consequently, the Computer Science community calls for the involvement of the general public in the design and evaluation of AI systems. Assessing the fairness of individual predictors is an essential step in the development of equitable algorithms. In this study, we evaluate the effect of two common visualisation techniques (text-based and scatterplot) and the display of the outcome information (i.e., ground-truth) on the perceived fairness of predictors. Our results from an online crowdsourcing study (N = 80) show that the chosen visualisation technique significantly alters people's fairness perception and that the presented scenario, as well as the participant's gender and past education, influence perceived fairness. Based on these results we draw recommendations for future work that seeks to involve non-experts in AI fairness evaluations.},
	Author = {van Berkel, Niels and Goncalves, Jorge and Russo, Daniel and Hosio, Simo and Skov, Mikael B.},
	Booktitle = {Proceedings of ACM SIGCHI Conference on Human Factors in Computing Systems},
	Doi = {10.1145/3411764.3445365},
	Location = {CHI'21},
	Pages = {to appear},
	Title = {Effect of Information Presentation on Fairness Perceptions of Machine Learning Predictors},
	Url = {https://nielsvanberkel.com/files/publications/chi2021a.pdf},
	Year = {2021}}
Powered by bibtexbrowser