Equality of Opportunity metrics

The idea of equality of opportunity metrics, is to compare true positives and/or false positives rates across groups. We will use here min and maj respectively to indicate the unprivileged and privileged groups.

Two common metrics used to convey are:

• Equal Opportunity Difference:

$$EOD=TPR_{min}-TPR_{maj}$$

• Average Odds Difference:

$$AOD=\frac{1}{2}[(FPR_{min}-FPR_{maj})+(TPR_{min}-TPR_{maj})]$$

• False Positives Rate Difference:

$$FPR=(FPR_{min}-FPR_{maj})$$

For all metrics, the ideal value is 0 and fair values are considered between -0.1 and 0.1.

An example of how to measure bias in a binary classification problem in recruitment can be found in our notebook, which can be accessed here or downloaded as the following file:

37KB

Measuring_Bias.ipynb