Research work · 2022

Ethical Data Preprocessing

Data Science Ethics

Abstract

This chapter considers how to measure and ensure fairness of your dataset, in terms of privacy and discrimination against sensitive groups. It demonstrates that simply removing identifiers is not enough to avoid being able to re-identify someone. Privacy can be defined and enhanced by making the dataset k-anonymous, l-diverse or t-close, so as to combat homogeneity and linkage attacks against datasets that can reveal personal and/or sensitive information. This is illustrated with re-identification tales from datasets of movie viewing, search query, and location data. On the additional subject of making predictions, fairness is a major concern. The way in which the target variable is defined can already include bias in the dataset, as discussed in the context of pregnancy prediction and face recognition. Finally, it describes metrics and methods to define and remove such bias from the dataset, so as to avoid discrimination in the resulting predictive model.

DOI: 10.1093/oso/9780192847263.003.0003