To Guarantee Impartial AI Decisions, Lady Justice Needs to Blink

Author:Murphy | View: 26490 | Time: 2025-03-23 19:49:32

Unwanted bias has been identified as major risk to the wider adoption of artificial intelligence (AI). In my previous article I have discussed different potential sources of this bias. But what if we simply ignored the sensitive attributes altogether? In the following text, I will explain why so-called "fairness through unawareness" is not the right solution.

Systematic, unequal treatment of individuals based on their membership of a sensitive group is considered discrimination. For example, when people face unequal or disadvantageous treatment simply because they are male or female, this is considered gender discrimination. When job applicants or employees are treated less favourably because of their age, we talk about age discrimination.

There is broad consensus in our society that it is unfair to make a distinction on the ground of a personal characteristic which is usually not a matter of choice. Therefore, most legal frameworks prohibit such actions. When it comes to non-discrimination in the EU, for example, the Convention for the Protection of Human Rights and Fundamental Freedoms defines the "Prohibition of discrimination" in Article 14. This principle is further contained in the Charter of Fundamental Rights of the European Union which states in Article 21 that "Any discrimination based on any ground such as sex, race, colour, ethnic or social origin, genetic features, language, religion or belief, political or any other opinion, membership of a national minority, property, birth, disability, age or sexual orientation shall be prohibited."

Photo by Tingey Injury Law Firm on Unsplash

Fairness through unawareness

Computers are not biased like humans often are, so it seems simple for a machine to remove discrimination. Take gender-based discrimination for instance: if the machine does not have any information regarding the gender, then there can be no more bias, right? Unfortunately, things are a little more complicated.

The principle of simply excluding any sensitive attributes as features from the data in order to obtain "fairness through unawareness" is known as "anti-classification" among legal scholars. In EU law, this is enforced on the level of data protection: The General Data Protection Regulation (GDPR) regulates the collection and use of personal data, including sensitive personal data. For many use cases, it strictly prohibits the storing and processing of a list of attributes which were classified as protected, i.e. sensitive.

Chatty proxies

For non-AI systems, when using conventional, deterministic algorithms with a manageable amount of data, the current approa

ch can provide a solution. However, it is important to point out that in the case of ill intention, anti-classification does not prevent discrimination per se, as the practice called "redlining" has proven in the past: Sometimes, non-sensitive attributes may be strongly linked to sensitive attributes. Consequently, they can serve as substitutes or proxies. For example, the non-sensitive attribute zip code might be correlated with the sensitive attribute race when many people from the same ethnic background live in the same neighbourhood. Hence, already in the context of non-AI systems, seemingly unsuspicious attributes can be misused to produce discriminatory decisions with the purpose of explicitly excluding a specific sensitive subgroup. Without actual knowledge of the sensitive attributes, such actions are hard to detect and to prevent.

Enter Big Data

When it comes to AI systems, the concept of removing sensitive attributes from data in order to prevent algorithms from being unfair has proven particularly insufficient: Such systems are usually backed by high-dimensional and strongly correlated datasets. This means that the decisions are based on hundreds or even thousands of attributes whose relevance is not obvious at first glance for the human eye. Further, some of those attributes usually contain strong links which again are difficult to spot for humans. Even after removing the sensitive attributes, such complex correlations in the data may continue to provide many unexpected links to protected information. In fact, heuristic methods exist to actively reconstruct missing sensitive attributes. For example, the Bayesian Improved Surname Geocoding (BISG) method attempts to predict the race given the surname and a geolocation. While the reliability of this method is generally disputed, it demonstrates that prohibiting to collect sensitive attributes does not prevent any possible misuse just by technical design.

Can you anonymise a resume?

But even without any bad intent to discriminate, there is a danger of hidden indirect discrimination which is very difficult to detect in the results. To illustrate the problem, imagine an AI system which analyses resumes in order to propose starting salaries for newly hired staff. Let's further assume that women were discriminated in the past because their salaries were systematically lower compared to those of their male colleagues. As explained above, historical bias of this kind cannot be overcome by excluding the sensitive attribute "gender" in the learning data since many links to non-sensitive attributes exist. For example, some sports are more popular among women or men. In languages with grammatical gender, the applicant's gender may be revealed through gender inflections of nouns, pronouns or adjectives. And yet more complex, in a country with compulsory military service exclusively for men, the entry age at university could provide a hint to the gender, too. Even when trying to additionally adjust for all of those identified correlations manually, it remains impossible to establish a sufficient degree of "unawareness" which could guarantee discrimination-free decisions: Based on seemingly unsuspicious proxy variables, a Machine Learning algorithm would be quick to recognise the old pattern and continue to allocate lower salaries to women.

Photo by Marjan Blan | @marjanblan on Unsplash

Active fairness

The risk to privacy protection when storing sensitive attributes is obvious. For example, confidential information could get leaked to an untrusted environment and misused by third parties for fraud. Accordingly, the motivation behind the current rules to address such concerns is understandable: Where no personal data is stored, none can be lost. When it comes to Big Data, however, this assumption does not hold anymore due to the high degree of correlations in the data. For AI applications, the current practice of trying to ignore the existence of sensitive subgroups by omitting sensitive attributes actually may bare greater risk than any privacy concerns related to the data collection. New technical security mechanisms are needed to protect the sensitive attributes from misuse, but allow their active use to make sensitive subgroups visible and account for them with the purpose of verifiable fair results. They are required, for AI stakeholders but also for regulators, to apply statistical measures in order to test for imbalanced results and detect any type of discrimination. It is only such "active fairness" that can ensure that the standards of fairness and non-discrimination in AI systems are respected.

So what

Ignoring the sensitive attributes to obtain impartial AI decisions is useless due to the many correlations and proxy variables in the data. Doing so is even counterproductive because it makes detecting unwanted biases very difficult. It is instead necessary to "see" the sensitive subgroups which ought to be protected to achieve AI fairness. This requires the active use of sensitive attributes.

Many thanks to Antoine Pietri for his valuable support in writing this post. In my next article, I discuss the challenging but essential task of defining fairness.