Step 1: Understanding bias

We define bias as an unwanted prejudice in the decisions made by an AI system that are systematically disadvantageous to a person or group. Multiple types of bias exist, and can be unknowingly introduced in algorithms at any stage of the development process, whether during data generation or model building. Refer to this page to learn about different types of bias.

In order to measure whether a system treats different groups of people equally, we need to agree on a definition of equality:

• Equality of Outcome: If we select this definition, we ask that all subgroups have equal outcomes. For example, in a recruitment context, we may require that the percentage of applicants hired is consistent across groups (e.g. we want to hire 5% of all female applicants and 5% of all male applicants). Mathematically, this means that the likelihood of a positive outcome is equal for members of each group (regardless of the ground-truth labels):$P(\hat Y=1|G=0)=P(\hat Y=1|G=1).$
• Equality of Opportunity: If we select this definition, we ask that all subgroups are given the same opportunity of outcomes. For example, if we have a face recognition algorithm, we may want the classifier to perform equally well for all ethnicities and genders. Mathematically, the probability of a person in the positive class being correctly assigned a positive outcome and the probability of a person in a negative class being incorrectly assigned a positive outcome should both be the same for privileged and unprivileged group members:$P(\hat Y=1|G=0, Y=y)=P(\hat Y=1|G=1, Y=y), \quad y\in\{0,1\}.$ Notice that here, ground-truth labels are important and necessary.

In this guide, we are mainly concerned with treating different groups of people equally. However, there are applications where we may have to deal with biases against individuals. You can find more information on how to define bias in these instances here.

We will next cover how to measure bias in different situations. You can click here for measuring bias for classification tasks, and here for measuring bias in regression.