So How Fair Is Your AI, Exactly?
The use of artificial intelligence (AI) has given rise to new ethical and legal challenges. In my previous article I illustrated why removing the sensitive information from the training data does not promote fairness, but rather the opposite. This article is about identifying the most appropriate fairness definition for an AI application. The bespoken tool has been originally presented in a research paper I co-published on the topic.
Collecting and analyzing data through AI has become standard, and Machine Learning is improving business performances in many areas today. However, numerous cases of AI machine bias have been exposed in recent years, and new examples emerge on a regular basis. This is one of the main pitfalls of AI: if the training data include any kind of bias, the algorithm will incorporate and enforce it – potentially harming sensitive subgroups defined by gender, religion, ethnicity, or age.
The challenge of defining ‘fairness'
Current approaches to mitigate bias are mostly technical and focus on adjusting algorithms or data to satisfy some kind of fairness. However, multiple, conflicting notions of fairness exist and unfortunately there is no universally accepted definition. The most appropriate fairness metric always depends on the context of application. In practice, identifying the fairness objective is complicated because mapping ethical principles to metrics is not a straightforward process. Still, to obtain sustainable solutions for fairer AI, attention must be focused on this problem since the best technical mitigation method will fail when the implemented fairness objective is not aligned with the stakeholders' expectations.
Not on the same page
In many scenarios, this decision is not a trivial one, as has become obvious in the heated debate over the COMPAS algorithm. This one had been developed by the company Northpointe to generate an independent, data-derived "risk score" for several forms of recidivism. Such an algorithm is used in the criminal justice sector in the US to support the judge with particular decisions such as granting of bail or parole. The score is of informative character and the final decision is still up to the judge.
In May 2016, the investigative journalism website ProPublica focused attention on possible racial biases in the COMPAS algorithm. Its main argument was based on analysis of the data which showed that the results were biased. In particular, the false positive rate for people who were black was significantly higher compared to people who were white. As a result, black people were disproportionately often falsely attributed a high risk of recidivism. Northpointe, on the other hand, responded to the accusations by arguing that the algorithm effectively achieved predictive value parity for both groups. In a nutshell, this ensured that risk scores corresponded to probabilities of reoffending, irrespective of any skin colour.
From an objective point of view, it can be stated that both parties make valid and reasonable observations of the data. However, the huge controversy revealed that it is absolutely critical to precisely define and disclose the selected fairness objective for an application. And this decision usually involves arbitration and compromise. In the given scenario, for example, the two fairness objectives could only be mutually satisfied if one of the following conditions was met: Either the base rates of the sensitive subgroups are exactly identical, or the outcome classes are perfectly separable which would allow for creating an ideal classifier that achieves perfect accuracy. Unfortunately, both requirements are very unlikely in real world scenarios.
Navigating metrics
Surprisingly, relatively little research has been conducted on how to streamline the fairness selection process for AI applications in practice. To overcome this challenge, we have developed the Fairness Compass, an experimental tool which seeks to structure the complex landscape of fairness metrics. Based on a set of concrete questions regarding the nature of their data, beliefs in its correctness, fairness policies, and whether the focus should be more on specificity or sensitivity of the model, the Fairness Compass guides AI practitioners towards the most suitable option for a given scenario. Formalizing this selection process and turning it into a straightforward procedure helps clear a hurdle for implementing Responsible Ai in the real world. Moreover, recording the reasoning behind the decisions can serve as documentation for internal purposes and as means of communication to increase transparency, fostering trust in the technology.
The Fairness Compass has been published as open source project on GitHub. It was nominated for the Gartner Eye on Innovation award and has been included in the AI Fairness Global Library by the World Economic Forum.
Sample application
To illustrate the concept, let's take a sample scenario in the human resources context. As sensitive subgroups, we consider men and women. The question to be answered is which definition of fairness would be most appropriate when it comes to assessing fairness in employee promotion decisions. Please note that this is just a fictional thought experiment, and depending on the context, other answers with different results may apply. The purpose of the Fairness Compass is to support well informed decision making based on the defined requirements for a given scenario.
In the animation above, the Fairness Compass is represented as a decision tree with three different types of nodes: The diamonds symbolize decision points; the white boxes stand for actions and the grey boxes with round corners are the fairness definitions. The arrows which connect the nodes represent the possible choices.
Let's kick-off the process. The first question is about existing policies which may influence the decision. Fairness objectives can go beyond equal treatment of different groups or similar individuals. If the target is to bridge prevailing inequalities by boosting underprivileged groups, affirmative actions or quotas can be valid measures. Such a goal may stem from law, regulation, or internal organizational guidelines. This approach rules out any possible causality between the sensitive attribute and the outcome. If the data tells a different story in terms of varying base rates across the subgroups, this is a strong commitment which leads to subordinating the algorithm's accuracy to the policy's overarching goal. For example, many universities aim to improve diversity by accepting more students from disadvantaged backgrounds. Such admission policies acknowledge an equally high academic potential of students from sensitive subgroups and considers their possibly lower level of education rather an injustice in society than a personal shortcoming.
For our sample scenario, we conclude that no such affirmative action policy is in place for promotion decisions. Therefore, we choose "No" and document the reasoning behind our choice. Now, we continue with the next question and repeat the procedure until we reach a leaf node which contains the recommended fairness definition for the defined use case.
Hence, following such a formalised process can significantly contribute to identifying and explaining the optimal fairness metric for a particular AI application.
So what
In practice, a lot of different definitions of fairness exist. Because some are mutually exclusive, it is necessary to settle for one of them. Making the choice is not trivial since the best decision always depends on the context of application and trade-offs are often unavoidable. Therefore, it is crucial to select the fairness objective of an AI application with great care and also communicate it to internal and external stakeholders. Transparency about the reasoning behind this decision is a key factor towards a sustainable implemention of fairer AI.
Many thanks to Antoine Pietri for his valuable support in writing this post. In the following article, I will outline how to actively mitigate biases in AI applications.
References
N. Mehrabi, F. Morstatter, et al. (2021). A Survey on Bias and Fairness in Machine Learning. ACM Computing Surveys (CSUR) 54, 6 (2021), 1–35.
J. Angwin, J. Larson, et al. (2016). Machine bias. Ethics of Data and Analytics, Auerbach Publications, 254–264.
W. Dieterich, C. Mendoza, et al. (2016). COMPAS risk scales: Demonstrating accuracy equity and predictive parity. Northpointe Inc.
S. Corbett-Davies & S. Goel (2018). The Measure and Mismeasure of Fairness: A Critical Review of Fair Machine Learning. arXiv:1808.00023.
P. Saleiro, B. Kuester, et al. (2018). Aequitas: A bias and fairness audit toolkit. arXiv:1811.05577.
K. Makhlouf, S. Zhioua, et al. (2021). On the Applicability of Machine Learning Fairness Notions. ACM SIGKDD Explorations Newsletter. 23, 1, 14–23.
B. Ruf & M. Detyniecki (2021). Towards the Right Kind of Fairness in AI. ECML/PKDD 2021 Industry Track.
