A Literature Review On AI fairness Best Practices

Introduction¶

Fairness in AI and ML has become a major concern in many fields, including healthcare, aviation, and hiring. These systems are increasingly used to make important decisions, and it’s important that they operate in a fair and unbiased way to ensure equal treatment and avoid harm to specific groups of people. While these technologies play vital roles in performance , efficiency and great innovations they also perpetuate biases and inequities, often embedded in their data, model types and algorithm structures. If we are to address these challenges we should not only solve them with technical solutions but also with sociotechnical solutions like integrating fairness in practitioners existing work flow and normalizing fairness in organizations.

Recent research has developed tools, frameworks and guidelines to promote fairness in AI. These tools are like softwares or platforms that help practitioners detect and mitigate biases in AI or ML systems, some examples of these tools are Aequitas developed by Center for Data science and Public Policy at the University of Chicago Saleiro et al. (2018) , IBM AI Fairness 360 Bellamy et al. (2019)(AIF360) , Fairlearn Bird et al. (2020) , Google indicators Google (n.d.) and Google (n.d.). While frameworks provide structured approaches and methodologies for incorporating fairness in AI deployment and development lifecycle, some of these frameworks are:

• Fairness for Socio-technical Process as examined by Richardson & Gilbert (2021) and Holstein et al. (2019).
• Co-Design Framework for Fairness as examined by Madaio et al. (2020) and Richardson et al. (2021).
• Fairness Principles into Practice Framework as highlighted by Beutel et al. (2019).

Guidelines offer high level principles and best practices for promoting fairness in AI , examples include Google AI principles (which are a set of principles written by Google that outlines its commitment to responsible AI development including fairness, safety, and accountability) and Microsoft Responsible AI Principles similar to Google’s principles, these guidelines outline Microsoft’s approach to responsible AI, emphasizing fairness, transparency, and human oversight.

These tools, frameworks, and guidelines represent a growing effort to operationalize fairness in AI. They provide practitioners with concrete resources and guidance for building more equitable and inclusive AI systems. However, it is important to know that simply using these tools,frameworks and guidelines does not guarantee fairness.For instance, a tool designed to detect bias in datasets may fail if practitioners do not fully understand how to interpret or act on the results, leading to continued biases in the final AI system. There is a gap in these advancements which is translating these advancements to actionable tools that can be adopted by industry practitioners and organizations.
Moreover, the rise of LLMs further amplifies these risks. Without clear, written, and empirically validated guidelines, LLMs may perpetuate and exacerbate biases, making it crucial to establish robust standards for fairness in automated decision making systems.

Together we will examine these works, because they provide a comprehensive exploration of fairness challenges, tools, and practitioner needs, offering valuable insights for bridging the gap between fairness research and its practical implementation.

This review is structured as follows:

What is Fairness ?¶

Fairness, as it pertains to AI is a very complex and contextual concept, over the years many researchers have had different definitions to fairness , making the decision of which fairness methodology to adopt and which corresponding metrics to quantify highly contextual. In reference to AI fairness encompassing ethical, social, and technical considerations.

Richardson & Gilbert (2021) defines fairness as solutions to combat algorithmic bias, which is often inclusive of top-tier solutions from explainability, transparency, interpretability, and accountability research. They emphasize the need for fairness to be embedded throughout the ML pipeline, from data collection to model evaluation. However, the interpretation and operationalization of fairness vary significantly across research and practice.

Selbst et al. (2019) defines fairness primarily as a social and legal property rather than a purely technical one. It argues that fairness and justice are attributes of social and legal systems, not merely of the technical tools embedded within them. In computer science, abstraction often involves treating systems as “black boxes” meaning a system where only the inputs and outputs are known or observed, while the internal workings remain hidden. This approach allows the focus to remain on how data is provided and received rather than on the underlying processes. For example, in machine learning, a neural network can be viewed as a black box, where an image is given as input, and the model outputs a classification (e.g., ‘cat’ or ‘dog’) without revealing how it made the decision. However, this approach abstracts away critical aspects of social dynamics and human institutions, which are integral to fairness. The paper identifies this as an abstraction error, where fairness metrics or guarantees are applied to isolated subsystems without considering the broader sociotechnical frame. Fairness in machine learning is often reduced to quantifiable properties, such as equality of outcomes (minimizing disparate impact) and equality of opportunity (equalizing FPR or FNR across groups). However, these mathematical formalisms of fairness are naturally limited because they often oversimplify fairness by reducing it to quantifiable metrics, while neglecting crucial aspects such as the processes used to achieve fairness (procedural), the specific social and historical context in which fairness is evaluated (contextual), and the fact that fairness itself is subject to debate and different perspectives (contestable). Mathematical definitions of fairness, such as disparate impact or equal opportunity, are often insufficient because they oversimplify nuanced social concepts and they struggle to reconcile conflicting fairness goals (e.g., balancing false positives vs. false negatives in different domains). The authors critique the tendency to prioritize fairness metrics that are mathematically simple, efficient, or easily transferable between systems, what they refer to as ‘technical elegance’ over the complex realities of fairness in the real world. They argue that fairness should be analyzed within a sociotechnical frame, which considers not just the algorithm but also the broader social systems, actors, and institutions that shape and are affected by it. A truly fair system must model and address human behaviors, social norms, and institutional dynamics, integrating them into the technical design.

Holstein et al. (2019) emphasizes that biases and unfairness are fundamentally socio-technical problems, and that technically focused research should improve fairness as a collaborative work between organization and practitioners. Richardson et al. (2021) emphasized that fairness cannot be defined in a single, universal way but should instead be tailored to specific goals and tasks. However, practitioners often struggle with key challenges, such as determining when to incorporate fairness into their workflow and how to define fairness in a meaningful way for their specific context. The authors highlight these difficulties, illustrating how the lack of clear guidance can make it hard for practitioners to apply fairness principles effectively. Meanwhile, Madaio et al. (2020) focus on fairness as a practical construct, proposing rubrics and tools that practitioners can use to evaluate and mitigate Bias in ML systems. While Ferrara (2023) defined fairness in AI as the absence of bias or discrimination in AI systems, which can be challenging to achieve due to the different types of bias that can arise in these systems.

Suresh & Guttag (2021) study addresses fairness based on different categories of harms such as allocative harms and representative harms.Their study defines allocative harms as when opportunities or resources are withheld from certain people or groups. For instance, a hiring algorithm may disproportionately filter out qualified candidates from underrepresented groups due to biases in the data it was trained on. Representational harms, on the other hand, arise when a system reinforces or amplifies stereotypes, misrepresents certain groups, or neglects to include them altogether. For example, an image classification model that performs poorly on images from certain geographic regions or ethnic groups may mislabel individuals, making them harder to find in searches, or even classify them inaccurately in offensive or harmful ways.

Representation bias is one source of representational harm. It occurs when the data used to train a model under-represents certain segments of the population, leading to a failure to generalize well for those groups. This bias can emerge in several ways:

1. Defining the Target Population: If the target population does not reflect the use population, the model may fail to perform adequately. For instance, data collected from Boston might not generalize to Indianapolis or even to Boston’s population 30 years later.
2. Underrepresented Groups: Even if the use population is accurately defined, minority groups within that population may not be sufficiently represented. For example, in a medical dataset targeting adults aged 18–40, pregnant individuals might constitute only 5% of the data, making the model less robust for that subgroup.
3. Sampling Limitations: Biases can also arise during data collection. For example, in healthcare datasets, there may be an over-representation of patients with severe or critical health conditions, such as those who require emergency treatment or hospitalization. These individuals are often considered higher priority cases, so the data may focus more on them than on those with mild or chronic conditions. As a result, the training data may become unbalanced, making it less reflective of the general population and potentially leading to less accurate predictions or recommendations for people with less severe health issues.

An illustrative example of representation bias is the ImageNet dataset, a widely-used collection of labeled images. Although it is intended to generalize to all natural images, 45% of its images were sourced from the United States, with very few from countries like India (2.1%) and China (1%). As a result, classifiers trained on ImageNet perform poorly on images from underrepresented regions, such as India and Pakistan, which leads to representational harms in recognizing objects or people. The distinction between allocative and representational harms underscores the importance of understanding fairness in terms of both outcomes and inclusion. By addressing these types of harms, practitioners can work toward more equitable AI systems that mitigate biases and their adverse impacts.

Despite all these different definitions from these researchers one thing is clear, fairness is not a fixed criterion that can be universally applied. Instead, it needs to adapt to the specific circumstances, contexts, and stakeholders involved in each situation.

Importance of fairness¶

The importance and use of AI is growing rapidly in society and its increasing influence in decision making in domains such as healthcare, education , hiring , aviation and criminal justice. It is important for these AI systems to be fair and unbiased, as unfairness and bias can perpetuate or even exacerbate societal inequities, causing harm to marginalized groups and undermining public trust in AI technologies.

Holstein et al. (2019) emphasize that fairness is essential to ensuring AI systems do not replicate biases embedded in historical data or human decision-making processes. For example, biased hiring algorithms may disadvantage candidates from underrepresented groups, while inequitable healthcare algorithms may lead to poorer outcomes for minority populations. Also Ferrara (2023) explores how biases in AI can stem from data, algorithms, and societal factors, such as historical inequalities or cultural stereotypes, and how these biases can amplify existing inequalities. For example, if an AI system is trained on biased data that reflects past discriminatory practices, it might continue to favor certain groups over others. The study highlights that, if these biases are not addressed, AI systems could perpetuate unfair treatment on marginalized groups, making societal disparities worse. This underscores the urgent need for fairness in AI to prevent these harmful impacts and promote more equitable outcomes.

Fairness is also crucial for ensuring AI systems are ethical and align with societal values. Richardson & Gilbert (2021) , highlight that fairness helps mitigate representational harms, such as misclassifying certain demographic groups, and allocate harms, where resources or opportunities are unfairly distributed. Addressing these harms is critical for creating AI systems that are not only efficient but also just.

Public trust is essential for the widespread adoption and acceptance of AI. If AI systems are perceived as unfair or biased, peoples confidence in these systems will decrease and it will hinder the advancements of the systems to go forward. Madaio et al. (2022) discusses the challenges that practitioners faced in assessing and ensuring fairness, including difficulties in defining fairness, managing different fairness objectives, and handling incomplete or biased data. The study emphasizes the need for transparent processes that help build trust between practitioners and stakeholders, as clear, consistent methods can ensure that AI systems are held to high ethical standards. While Madaio et al. (2020) focuses on organizational challenges such as the lack of clear roles and responsibilities for ensuring fairness or the difficulty in aligning diverse stakeholder interests, and suggests that integrating fairness considerations into the design process from the start can help foster greater trust and collaboration among stakeholders.

As AI and ML systems grow rapidly it is important to establish clear lines of accountability and responsibility among practitioners and stakeholders, ensuring that these systems are developed and deployed correctly. Balayn et al. (2023) and Beutel et al. (2019) examines the practical challenges of implementing fairness in real-world settings, highlighting the need for tools and practice that promote accountability.These papers think that a focus on fairness can prevent deployment of unfair systems to the society.

Richardson et al. (2021) evaluates fairness toolkit from a practitioner’s perspective, suggesting improvements that can facilitate the development of fairer AI systems.

Without fairness, AI systems risk increasing inequalities and losing public confidence. As a result, embedding fairness into AI development and deployment is not just a technical issue or solution but also socio-cultural one. It is essential for fostering trust, equity, and societal benefits.

Challenges And Barriers In Achieving Fairness¶

Achieving AI fairness is complex and filled with numerous challenges as described by numerous researchers. These challenges can be categorized into technical, organizational, and socio-cultural challenges.

Technical Challenges¶

Achieving fairness in AI systems presents complex sets of technical problems, these problems are often times confused with societal and ethical considerations.These challenges span through the entire ML lifecycle,from data collection and pre-processing to model training, evaluation and deployment.Below we are going to discuss some of these technical challenges as highlighted by the papers.

Data Bias¶

Bias in training data is a very pervasive problem Ferrara (2023) and Richardson & Gilbert (2021) discuss how historical biases reflected in data can lead to discriminatory outcomes. Ferrara (2023) highlighted that Data bias occurs when the data used to train ML models are unrepresentative or incomplete, leading to biased outputs. This can happen when the data is collected from biased sources or when the data are incomplete, missing important information, or contain errors. while Richardson & Gilbert (2021) on the other hand suggests that data bias is another form of data collection bias, the author explains that it arises from the insufficient representation of subgroups within the dataset. For example, if a dataset used for training an AI model on healthcare includes mostly data from young, healthy individuals, it may fail to accurately represent older or sicker patients. This misrepresentation can occur due to the selection of irrelevant attributes, such as only considering certain health conditions or age groups, or through incomplete data collection, such as excluding individuals with disabilities or those from different socioeconomic backgrounds. Holstein et al. (2019) highlights the difficulty of collecting unbiased data, especially in real-world scenarios where existing societal biases are deeply entrenched. For example, if a facial recognition system is trained primarily on images of lighter-skinned individuals, it may perform poorly on images of darker-skinned individuals, perpetuating existing biases.

Algorithmic Bias¶

Even with unbiased data, algorithms themselves can introduce biases. Ferrara (2023) defines algorithmic bias as biases that occurs when the algorithms used in ML models have inherent biases that are reflected in their outputs. This can happen when algorithms are based on biased assumptions or when they use biased criteria to make decisions. This study highlights some real world examples of algorithmic biases, such as:

• Facial recognition technology used by law enforcement agencies. A study by the National Institute of Standards and Technology (NIST) Grother et al. (2019) found that facial recognition technology was significantly less accurate for people with darker skin tones, leading to higher rates of false positives. This bias can have serious consequences, such as wrongful arrests or convictions.
• An algorithm that prioritizes age or gender, leading to unfair outcomes in hiring decisions.
• One well-known example is the COMPAS system used in the United States criminal justice system, which predicts the likelihood of a defendant re-offending. A study by ProPublica found that the system was biased against African-American defendants, as they were more likely to be labeled as high-risk even if they had no prior convictions.

Beutel et al. (2019) discusses the challenges of mitigating these biases in practice, particularly the difficulty of balancing competing goals, such as efficiency and fairness. For instance, an algorithm optimizing for efficiency might inadvertently discriminate against a particular demographic group if that group is underrepresented in the data. This challenge is evident in real world applications like loan approval algorithms, which, when prioritizing efficiency, may unintentionally disadvantage people from certain racial or socioeconomic backgrounds if they are not adequately represented in the training data.

Apart from the examples mentioned in these papers there are many other instances of algorithmic bias:

• Loan approval algorithms in financial institutions have been found to discriminate based on race, gender, and zip code. These algorithms may use proxy variables that indirectly encode racial or gender bias.
• Search engine and job recruitment platforms have been shown to display biased results. For example, Google searches for professional job names might return more images of men, while searches for certain service jobs might return more images of women or specific racial groups.

Organizational Challenges¶

These challenges often hinder the effective implementation of fairness principles in AI systems. They can stem from various factors, including:

Lack of Awareness and Expertise¶

Many organizations lack the awareness and expertise to address AI fairness effectively, which hinders their ability to implement fairness principles in practice. Madaio et al. (2020) reveals the challenges practitioners face in assessing fairness , some of the participants in his survey expressed their lack of knowledge in some domains of fairness and needed guidance, while Madaio et al. (2022) emphasizes the need for organizational checklists and guidelines to promote fairness. Without sufficient training and resources, organizations may struggle to implement fairness principles in practice Ferrara (2023) highlights that ensuring fairness in AI is a complex and ongoing challenge that requires a multi-disciplinary approach involving experts from fields such as computer science, law, ethics, and social science. By developing and implementing a range of approaches to ensure fairness, we can work towards creating AI systems that are unbiased, transparent, and accountable.

Conflicting Objectives:¶

Organizations often prioritize other objectives such as accuracy,reputation, or efficiency over fairness. Holstein et al. (2019) explores the needs of industry practitioners, highlighting the potential tension between fairness and other performance metrics. Similarly, in Madaio et al. (2020) survey some organizations prioritize accuracy and reputation of their organizations over fairness. For example, some participants from the survey shared, “I get paid to go fast. And I go as fast as I can without doing harm. We’re not allowed to spend three years developing a product. We will die. Our competitors are on a weekly cadence. Do I do a good thing or do I do the thing that ships the product?” Additionally, Deng et al. (2022) observes how practitioners sometimes use fairness toolkit superficially, suggesting that fairness may not always be a top priority. For example, a company might prioritize maximizing profits, even if it means deploying an AI system that exhibits some degree of bias.

Checkbox Culture¶

Balayn et al. (2023) warns against a “checkbox culture” where organizations focus on ticking off fairness requirements without genuinely addressing the underlying issues. This approach often leads to a false sense of security, as organizations may believe they are being fair when, in reality, they are only paying lip service to fairness. For example, an organization might implement a fairness checklist, ensuring that their AI models meet certain standards on paper, but they may not take the time to deeply examine how their models affect different groups in practice. This superficial approach can prevent real progress toward fairness and may even mask deeper biases in the system.

Societal challenges¶

Societal challenges related to AI fairness are broader and more complex than organizational challenges. One of the most pressing societal challenges in AI fairness is the difficulty of defining fairness itself.

Defining Fairness¶

Fairness is a complex concept that varies across contexts and stakeholders, and there is no single, universally agreed-upon definition. This variability makes it challenging to apply fairness consistently across different situations, as what is considered fair in one context may not be in another. For example, in hiring fairness might mean providing equal opportunities to all candidates, while in healthcare, it could involve tailoring treatment based on individual patient needs, ensuring that those with greater health challenges receive more resources.

Many practitioners struggle with understanding what fairness truly means, as they often don’t receive proper education on the topic. Richardson et al. (2021) and Richardson & Gilbert (2021) discusses how the concept of fairness is not clearly defined, which makes it difficult to establish objective standards for evaluating AI fairness. The lack of a clear definition and understanding of fairness can make it difficult for practitioners to apply and evaluate fairness in their work effectively.

Madaio et al. (2020) discusses the need of educating practitioners on what fairness is depending on their specific goals.Their study revealed that, many participants described anxiety or fear that they were missing important aspects of fairness related harms, but did not know how to assess their systems for potential risks. One participant from their study expressed this sentiment, saying: “I think that part of the problem is people don’t know what to ask. People want to do the right thing, but they don’t know what the right thing is.” Madaio et al. (2020) This highlights that while practitioners are willing to ensure their systems are fair and unbiased, they are often hindered by inadequate knowledge and support.

Selbst et al. (2019) discuss how efforts to define fairness in fair ML often rely on mathematical formulas. While these definitions can be translated into algorithms, they fail to capture the full complexity of fairness, which is deeply tied to procedures, social context, and political debate. They refer to this issue as The Formalism Trap, where reducing fairness to mathematical terms oversimplifies its refined and evolving nature. As Selbst et al. (2019) argue, fairness is not just an abstract outcome based metric but a concept deeply intertwined with social, cultural, and legal contexts. For example, Feldman et al. (2015) formalized the Equal Employment Opportunity Commission’s (EEOC) 80% rule into a measure called “disparate impact.” However, treating disparate impact as a simple numerical threshold misses the procedural steps the EEOC uses to assess whether the impact is justified by business necessity and whether there are less discriminatory alternatives. Limiting the question to a mathematical formulation gives rise to two distinct problems in practice:

1. Conflict between Definitions: Mathematical definitions often conflict, and there is no purely mathematical way to resolve these conflicts. For instance, debates about the COMPAS risk assessment tool have highlighted how different fairness metrics (e.g., equal error rates vs. equal accuracy across groups) can lead to incompatible conclusions about what is “fair.” The resolution of such conflicts requires normative judgments informed by the social context.

2. Inadequacy of Definitions: Mathematical definitions may not capture the core aspects of fairness and discrimination, which are often procedural, context-dependent, and politically contestable. Philosophers and legal scholars, such as Deborah Hellman and Issa Kohler-Hausmann, emphasize that fairness requires understanding situated cultural knowledge and societal norms, which cannot be fully encapsulated in mathematical models. In terms of:

   • Procedurality: Fairness in legal frameworks is often procedural rather than outcome based. For instance, firing an employee based on race or gender is illegal, but firing the same employee for other reasons is not even if the outcome is identical. The procedural nature of fairness also underpins laws like disparate impact, which go beyond thresholds to evaluate the necessity and alternatives of a decision.

   • Contextuality: Fairness depends on social and cultural context. What constitutes “discrimination” or “unfairness” often requires knowledge of societal stratification and its moral implications. For example, distinctions made in one context (e.g., hiring practices) may not be considered discriminatory in another (e.g., performance evaluations), and different attributes (e.g., race, gender, disability) require unique considerations.

   • Contestability: Fairness is a politically and culturally contested concept, subject to shifts in societal norms and laws. As new court cases, legislation, and advocacy movements redefine fairness, any static or universal definition risks becoming obsolete or exclusionary.

The process of formalizing fairness in computational systems, as highlighted in the Abebe et al. (2020), adds an additional layer of complexity. Formalization, which requires explicit inputs, goals, and assumptions, can illuminate hidden biases and prompt public debate about fairness. However, it also risks reducing fairness to rigid metrics that fail to account for its procedural, contextual, and politically contested nature. For example, as Abebe et al. (2020) note, formal models must specify what “good” outcomes mean and how they are prioritized, yet these choices often reflect practical constraints rather than broader societal goals. Similarly, the focus on individual predictions, rather than systemic evaluations, can obscure structural inequalities, reinforcing the concerns raised by Selbst et al. (2019) regarding the Formalism Trap. Addressing these challenges requires ensuring that formalized definitions align with societal values and incorporating inclusive, deliberative processes into the design of computational models.

By abstracting fairness into rigid mathematical definitions, fair ML approaches risk ignoring these critical dimensions. This not only undermines the societal relevance of fairness metrics but also imposes limitations on the ability of AI systems to adapt to evolving notions of equity and justice. Addressing the formalism trap requires integrating interdisciplinary perspectives and engaging with the procedural, contextual, and contestable nature of fairness.

Frameworks and Toolkits for Fairness¶

Fairness in AI requires a combination of frameworks and toolkit that address biases and inequities in ML and AI lifecycle. Frameworks provide conceptual guidance for incorporating fairness in the workflow while tools helps practitioners in detecting , evaluating and mitigating biases in AI systems. Together, these two concepts can help achieve fairness by aligning technical and socio-cultural solutions.

Fairness Frameworks¶

Frameworks provide conceptual guidance for incorporating fairness in the workflow,helping practitioners understand and apply fairness principles effectively. These frameworks are important because they offer structured approaches to address fairness in complex systems, ensuring that fairness is considered at every stage of development. Below are some of the frameworks discussed in the literature:

Fairness for socio-technical process¶

Richardson & Gilbert (2021), emphasizes the need to incorporate fairness throughout the entire AI/ML lifecycle. This framework categorizes bias into pre-existing, technical, and emergent types, ensuring that practitioners evaluate fairness at every stage, from data collection to deployment. Holstein et al. (2019) highlights that while much of their discussion focuses on needs for tools, they emphasized that biases and unfairness are fundamentally socio-technical problems, and that technically focused research efforts must go hand-in-hand with efforts to improve organizational processes, policies, and education. However, its reliance on fairness metrics poses challenges for practitioners unfamiliar with statistical concepts there are too many metrics for measuring unfairness with too few differences to delineate them. Furthermore, major trade offs exist between fairness metrics and making the choice of metrics depend on the specific situation. It is concluded that many of these metrics are complex and require expert input, which can lead to the introduction of hidden biases.

Co-Design Framework for Fairness¶

Madaio et al. (2020) and Richardson et al. (2021), which focuses on participatory methods to develop fairness practices. By involving technical and non-technical stakeholders, this framework ensures that fairness guidelines align with organizational goals and societal values. Madaio et al. (2020) also mentioned that one implication of their findings however, is that human centered requirements (such as, but not limited to, engagement with direct stakeholders) should be considered on an equal footing with technical feasibility .While effective in fostering collaboration, it is resource intensive and may be impractical for smaller organizations. Some of their participants voiced how it could be difficult to integrate in their organizations For example, soliciting input and concerns from diverse stakeholders and monitoring fairness criteria after deployment. Although many participants agreed that soliciting stakeholder feedback was important.

Fairness Principles into Practice Framework¶

Beutel et al. (2019) framework focuses on translating high level fairness principles into actionable steps for AI system development.The aim was to improve group fairness, specifically minimizing disparities in FPR between sensitive groups ensuring fairness and product health.Equality of Opportunity was the focus, with emphasis on FPR. A generalized version conditional equality, was introduced to make evaluation decisions explicit and account for varying group difficulties. However, it poses some challenges, as implementation may be resource intensive for smaller organizations.

Practitioner-Centered Framework for Fairness Assessment¶

Madaio et al. (2022) examines a practitioner focused framework aimed at understanding practitioner need and simplifying fairness assessment, making it easier to integrate fairness into existing workflows. The study highlights the need for accessible tools to measure and mitigate bias and encourages collaboration across teams to ensure fairness goals align with organizational values. It also provides guidance for practitioners on auditing datasets and models for biases. However, while the study focuses primarily on detecting fairness issues, it pays less attention to strategies for mitigating these biases.

Ethical AI Development Framework¶

Ferrara (2023) highlights an AI development framework that emphasizing fairness as a core value in AI development. Ferrara (2023) emphasizes that ethical values such as fairness, transparency, accountability and explainability should be integrated in the AI workflow. And also encourages organizational policies and governance structures to ensure adherence to fairness principles. If an AI system is biased and produces discriminatory outcomes, the responsibility lies not only with the system itself but also with those who created and deployed it.

Tools for Fairness in AI¶

Tools helps practitioners in detecting , evaluating and mitigating biases in AI systems. These tools support practitioners by offering metrics, visualizations, and algorithms to address fairness concerns at various stages of the AI lifecycle, from data preprocessing to post-deployment monitoring. By operationalizing fairness principles, these tools bridge the gap between theoretical frameworks and real world applications, enabling organizations to create more equitable and trustworthy AI systems. Here we look at key tools identified in the literature, highlighting their features, strengths, and limitations.

Google Toolkit¶

Fairness indicator and the What-If toolkit (WIT) are toolkit created by Google Google (n.d.) and Google (n.d.). Richardson et al. (2021) and Richardson & Gilbert (2021) describes these tools as an interactive widget, where users can provide their model(s), their performance and fairness metrics of choice, and the attributes on which slicing and evaluating will take place. Users also have the choice of doing model comparisons, intersectional analysis, and performance testing across thresholds.Fairness Indicators allow users to compare at most two models and allows users complete control of the attribute slices they choose to focus on. As an interactive widget, users have the opportunity to manipulate the metrics that they see, the thresholds that they can compare, and the slices that they’d like to focus on. While Richardson & Gilbert (2021) describes WIT as a tool that includes a features overview, a performance chart, and a data point editor. The features overview provides visualizations to depict the distribution of each feature with some summary statistics. The performance chart provides simulated results depicting the outcome of select fairness transformations, including an option to customize a fairness solution. Lastly, the data point editor provides custom visualizations of data points from the data and allows users to compare counterfactual points, make modifications to points to see how the results change, and plot partial dependence plots to determine model’s sensitivity to a feature. Richardson & Gilbert (2021) highlighted that unlike other toolkits, Google’s toolkits require that users provide a model before they can use the toolkits. In a separate, Richardson et al. (2021) found participants raised concerns about the toolkit’s complexity. Some felt overwhelmed by the amount of information and statistical details, while others noted that external tables could not be extracted for the Fairness Indicators widget. Participants who used this tool expressed a need for more resources to help them achieve their goals. Although the toolkit offers good visualizations, there is still room for improvement to better meet practitioners’ expectations.

Uchicago Aequitas¶

Aequitas Saleiro et al. (2018) is an open-source bias audit toolkit developed by the Center for Data Science and Public Policy at The University of Chicago that can be used to audit the predictions of ML based risk learning tools to understand different types of biases and make informed decisions about developing and deploying such system Richardson et al. (2021) and Richardson & Gilbert (2021) describes it as a bias detecting tool. This tool can be access through three different platforms: the command line, a python package, or through their web interface. It evaluates and compares models, calculates group biases, and highlights disparities. Aequitas also produces plots that record these fairness disparities and group metrics. Aequitas provides visualizations with color-coded groupings, allowing users to analyze disparities based on color categories. However, Richardson & Gilbert (2021) highlights that Aequitas does not offer fairness mitigation techniques. In a separate study, Richardson et al. (2021) found that some participants felt overwhelmed by the amount of information presented. While Aequitas allows users to choose which fairness metrics to display, the large number of available metrics can make the process overwhelming. Additionally, users must rerun scripts to update their visualizations, adding to the complexity.

IBM’s AI Fairness 360 Bellamy et al. (2019)¶

AIF360 (AI Fairness 360) is an open-source toolkit designed to help detect, understand, and mitigate algorithmic bias in machine learning models. It includes over 71 bias detection metrics and 9 bias mitigation algorithms, making it one of the most comprehensive fairness tools available. Richardson & Gilbert (2021) notes that while the toolkit does not have built in visualization functions, it provides guidance on integrating its features with the Local Interpretable Model-Agnostic Explanations (LIME) toolkit. In Deng et al. (2022) survey participants appreciated that AIF360 offers a broad view of state-of-the-art machine learning fairness techniques and includes a dedicated reference section for ease of use. However, participants also faced challenges in applying AIF360 to models beyond binary classification. One participant noted, ‘It was hard to apply AIF360 to many of our models which are not binary classification; e.g., for regression models, there is not much guidance on if or how the toolkit should be used.’ Additionally, some found it difficult to explain the toolkit’s concepts to stakeholders or individuals with little knowledge of fairness tools, making adoption more challenging.

Fair-learn¶

Fairlearn is an open source project to help practitioners assess and improve fairness of AI systems. The associated Python library, fairlearn, supports evaluation of a model’s output across affected populations and includes several algorithms for mitigating fairness issues Bird et al. (2020). Deng et al. (2022) highlights that the goal of Fairlearn is to create a vibrant community and resource center that provides not only code, but also resources such as domain specific guides for when to use different fairness metrics and bias mitigation algorithms. Deng et al. (2022) also highlights and encourage other fairness toolkits experts to emulate fairlearn and AIf360 for that community meeting which fairlearn hold on Discord while AIF 360 holds their on slack, where practitioner can join give their feedbacks.Some of the participants were pleased that Fairlearn and Scikit-learn had some resemblance in terms of API classes and functions,so it was easy for them to incorporate in their workflow and easy transitioning from one tool to another. However, in Deng et al. (2022) survey some of the participants voiced out their challenges such as fairlearn not being able to evaluate other models like regression but was made to evaluate classification problems. While other voiced that Fairlearn had not clearly listed out supported fairness metrics and mitigation algorithms and also organized codebase to start compared to AIF360. This calls for more explainability in how these tools are built and a need for extra resources on these tools creations because users don’t only want to use these tools but they also want to know how they were made.

LinkedIn’s Fairness Toolkit(LiFT)¶

The LinkedInFairnessToolkit (LiFT) Vasudevan & Kenthapadi (2020) is a Scala/Spark library that enables the measurement of fairness and the mitigation of bias in large-scale machine learning workflows. Richardson & Gilbert (2021) highlights that they provide an assortment of metrics, segregated based on whether they are for the data or the model outputs. Richardson & Gilbert (2021) suggests that they could be similar to AI Fairness 360, they do not produce their own visualizations. Unlike previous toolkits, they are built to be applied in Scala/Spark programs.

OxonFair¶

OxonFair Delaney et al. (2024) is an open source fairness toolkit designed to address biases across multiple domains, including tabular data, NLP, and computer vision. It distinguishes itself from other tools through its validation based fairness enforcement, which reduces overfitting by using validation data during optimization. This feature ensures better generalization to unseen data, a challenge often encountered in fairness interventions. Another key strength of OxonFair is its ability to jointly optimize fairness and performance, minimizing the trade-offs typically associated with improving fairness in AI systems. For example, while some tools like Fairlearn and AIF360 focus on specific fairness metrics, OxonFair offers a more flexible and expressive approach, supporting custom fairness measures based on metrics like TPR and FPR. In comparative evaluations, OxonFair outperformed existing tools on challenging domains like NLP and computer vision, where biases are harder to address. For instance, it was shown to reduce disparities in classification performance on datasets such as Jigsaw NLP and CelebA (computer vision) more effectively than baseline methods. This makes OxonFair particularly valuable for practitioners working in diverse applications and seeking robust solutions for fairness.

Below is a comparative table of all the tools mentioned

Synthesis of Frameworks and Tools¶

Frameworks and tools work hand in hand in addressing both the conceptual and practical dimensions of achieving equitable outcomes in AI and ML systems.

Both emphasize detecting pre-existing, technical, and emergent biases during the development process Richardson & Gilbert (2021) and Ferrara (2023) emphasizes on this in their research.

Frameworks advocate for fairness considerations at every stage of the AI lifecycle, while tools like AI Fairness 360, Aequitas, Google fairness indicator and Fairlearn provide functionalities to address biases at specific stages (e.g., data preprocessing or model evaluation)as examined by Richardson & Gilbert (2021), Richardson et al. (2021), Deng et al. (2022) and Balayn et al. (2023).

Frameworks, such as the co-design approach, stress involving diverse stakeholders in defining fairness goals, while checklist-based tools operationalize these collaborations. Madaio et al. (2020) and Holstein et al. (2019).

Notably, the distinction between these categories can be confusing, as some resources may embody characteristics of two or all three. For instance, a fairness toolkit may include both practical software tools and methodological frameworks, or a set of principles might inspire both guidelines and framework development. Below is a table that gives some of the traits and examples of tool, frameworks and guidelines:

Practitioner needs and feedback¶

This section highlights the needs practitioners want from the organizations, tools and fairness experts to help them understand and incorporate fairness in their AI or ML task.We will discuss the needs and feedbacks highlighted by the papers.

Need for Practical Guidance and knowledge¶

Practitioners often express the a need for clear, actionable guidance on how to incorporate fairness considerations into their work. They seek practical tools and frameworks that can be readily integrated into their existing workflows.

In Richardson et al. (2021) some participants expressed concerns about their lack of knowledge on fairness, as well as the limited information provided by the fairness toolkits they used. This paper emphasizes that institutions should be actively involved in the fairness process. They should incentivize fairness analysis, remain knowledgeable on fairness research, and provide infrastructure that supports organizational fairness guidelines. Furthermore, they should equip practitioners with a diverse set of fairness resources to support learning and the practical application of fair AI. This includes providing practitioners with all the necessary tools, as well as full access to fairness experts and domain-specific guides on fairness research.

Madaio et al. (2020) study shares similar views as Richardson et al. (2021), in Madaio et al. (2020)
survey his participants described anxiety or fear that they were missing important aspects of fairness related harms, but did not know how to assess their systems for potential risks. As one participant told mentioned “I think that part of the problem is people don’t know what to ask. People want to do the right thing, but they don’t know what the right thing is”.This paper believe that checklists could empower individual to advocates or address fairness issues, or at least to raise concerns, without the social costs inherent to adhoc processes. So there is a need for educating these practitioners about fairness starting from defining fairness based on their specific task or team goals, the type of tools to choose when doing a fairness process , what to look out for.

While Deng et al. (2022) and Richardson & Gilbert (2021) suggests that the majority of the work lies on the fairness experts not necessarily the organization. Deng et al. (2022) suggests that practitioners need adequate support to learn ML fairness from toolkits.Future fairness toolkits could include instructions and educational materials on creating and reviewing Data-sheet, and similar documentations like “Dataset Nutrition Labels” and Model Cards to better inform practitioners data explorations. Deng et al. (2022) also recommends that fairness toolkits can be created with designated pages or interactive modules that introduce ML fairness concepts, procedures, and best practices. Toolkits might proactively direct practitioners to these pages both when they first begin using a toolkit, and at critical points throughout their use of a toolkit. Richardson & Gilbert (2021) suggests that these fairness experts should help define and translate fairness into procedures that can be well and easily understood by practitioners. This implies that fairness experts should add more resources to their tools, not only creating tools that do the work but creating tools that can help these practitioners understand each step of what they are doing, starting by defining fairness based on their goals or the tool used, the importance of the architecture of the tool on their project, what to expect at each stage, how to tailor the tool to best suite their goal or task, all these things are very much important let practitioner be part of the tool. There is a need for educational resources tailored for industry professionals, bridging the gap between theoretical fairness models and practical applications.

Context-Specific Considerations¶

Fairness is not a one-size-fits-all concept. Practitioners emphasize the need for approaches that are sensitive to the specific context and potential harms of the AI system being develop. In Madaio et al. (2022) study practitioners recognized that existing performance metrics often fail to capture fairness-related harms. They needed to develop or adapt metrics tailored to the specific application domain and potential disparities the system might cause (e.g., fraud detection, text prediction).Metrics must go beyond traditional measures like precision or recall to address outcomes that could indicate fairness-related harms, such as demographic disparities or disproportionate impacts on marginalized groups. To address this, some participants described how their teams were developing new performance metrics specifically for assessing the fairness of their AI systems, saying, for example, “We need to start quantifying and optimizing towards not just success, but output quality and how catastrophic failures are. So we invented new metrics around quality”. This team therefore wanted to use this new metric for their disaggregated evaluation. However, other teams that created new performance metrics had difficulty agreeing on whether they should use these new metrics for their disaggregated evaluations or whether they should use the metrics that they typically used to assess performance.

While Richardson et al. (2021) study also emphasizes that practitioners must align fairness definitions with the specific objectives, stakeholders, and potential impacts of their task.Without a clear understanding of what fairness means in a given context, users struggle to choose from too many metrics and tools available. This lack of clarity leads to confusion and sometimes inefficient or incorrect tool usage.Some participants expressed uncertainty about how to define fairness due to many different definitions of fairness available. Others found the mathematical definitions provided by some of the tools confusing, as they lacked clear resources to help them understand. So Richardson et al. (2021) recommends that fairness experts should reduce the amount of workload the put in their tools make it less and easy to understand, because not all users are data scientists but have the will to learn. Make it easy for people with little or no knowledge to navigate their way through.

A well defined fairness objective allows practitioners to identify which tools and metrics are most appropriate for their scenario. For example, some tools might be better suited for addressing group disparities, while others may focus on individual fairness or long term impacts.

Integration with Existing Processes¶

Many practitioners report difficulty integrating fairness toolkits into existing workflows, citing issues like lack of interpretability, insufficient documentation, and mismatch with organizational needs. In studies like Richardson et al. (2021), Deng et al. (2022),Holstein et al. (2019), Madaio et al. (2020), Madaio et al. (2022), Ferrara (2023), and Balayn et al. (2023), participants emphasized on the need of integrating or align AI fairness in their existing workflow. Richardson et al. (2021) thinks that by embedding fairness considerations in workflows, it helps practitioners make informed decisions by proactively identifying potential biases and assessing trade offs. This study notes that fairness tools integrated into workflows can streamline evaluation processes by aligning with practitioners’ operational needs.

Integrating fairness into existing workflows helps navigate tensions between diverse stakeholders, particularly when selecting performance metrics. Many teams face challenges balancing business priorities, such as customer satisfaction or compliance, with fairness considerations that impact marginalized groups. For example, Madaio et al. (2022) highlight a case study of a fraud detection system where conflicting stakeholder goals caused significant disagreements about metric priorities. By embedding fairness into workflows, teams can ensure that these trade-offs are approached collaboratively and transparently, involving direct stakeholders where possible. This fosters trust by making fairness related concerns an integral part of the decision making process, promoting accountability and inclusively.

Embedding fairness into existing processes ensures that AI development aligns with organizational goals and values. Madaio et al. (2020) emphasize the importance of collaborative discussions when incorporating fairness into workflows, particularly through tools like co-designed checklists. These practices help organizations address diverse stakeholder priorities and align fairness considerations with business objectives, ethical principles, and regulatory requirements. By fostering collaboration across teams, organizations can ensure that their technical workflows reflect their broader mission, enabling the creation of systems that are both effective and equitable. However, they also highlighted the challenges they might face if these checklists are incorporated into their exciting workflow. For example, soliciting stakeholder’s ideas on inputs and monitoring fairness would be hard due to lack of fairness resources.

Incorporating fairness into workflows helps mitigate algorithmic harms by identifying and addressing biases early in the development lifecycle. Balayn et al. (2023) highlights how workflows designed with fairness considerations enable practitioners to flag potential risks and prevent the deployment of systems that could cause harm. Many participants in the study mentioned that simply discussing fairness with colleagues and stakeholders helped them learn new ways to identify and mitigate algorithmic harms. By embedding fairness practices into existing processes, organizations can proactively address algorithmic harms, fostering both ethical AI development and user trust.

Fairness considerations should be integrated throughout the entire AI development lifecycle, from data collection to deployment and monitoring. Practitioners need tools and processes that support this integration.

Stakeholder Engagement¶

Madaio et al. (2022) study participants found it difficult in identifying stakeholders and domain experts that are informed about demographic group in their teams. For example a participant said “I think that that’s where we do need to bridge the people who are experts on this and know the processes we should be going through before we [...] decide on implementations with our opinions of what is important and who is harmed. For gender non-binary [...] We need to ensure we have the right people in the room who are experts on these harms and/or can provide authentic perspectives from lived experiences [...] I think the same could be said about speakers of underrepresented dialects as well”. This challenge is exemplified by the case of a healthcare company that developed an AI system to recommend treatment plans. They primarily consulted with doctors and technical staff but failed to engage experts on gender identity or the experiences of non-binary patients. As a result, the system inadvertently made biased recommendations that didn’t fully consider the needs of non-binary patients, leading to concerns about fairness and equity in treatment.

Engaging with diverse stakeholders, including those potentially impacted by the AI system, is crucial for identifying and addressing fairness concerns. Practitioners need guidance and support for effective stakeholder engagement.

Organizational Support and Incentives¶

Practitioners often face a lack of support from their organizations, which is crucial for creating fair AI models. Many organizations don’t provide good incentives for practitioners working on fairness. For example in Holstein et al. (2019) participants noted that they are not rewarded within their organizations for their efforts around fairness.This is very crucial for the organization for their efforts around fairness. This lack of recognition is a problem because when practitioners feel appreciated, it encourages them to do more research, improve their work, and feel more comfortable voicing their concerns. Organizations should take fairness very seriously by making sure fairness is an integral part of the organization’s culture. This means ensuring everyone knows their role in achieving fairness and taking responsibility when things go wrong. By creating a culture where fairness is prioritized, organizations can help practitioners feel supported and accountable. As Richardson & Gilbert (2021) points out the organization’s support is essential for making fairness a real focus. Madaio et al. (2022) research note that practitioners often face challenges in defining and applying fairness. By investing in training programs and encouraging cross-disciplinary learning, organizations can empower their teams with the knowledge and tools to effectively address fairness related issues. This helps foster a culture where fairness is seen as an integral part of an employee’s role rather than an added burden. Ferrara (2023) suggests that it is the responsibility of organizations and government to ensure that AI systems are designed and used in a fair and transparent manner. If an AI system is biased and produces discriminatory outcomes, the responsibility lies not only with the system itself but also with those who created and deployed it. As such, it is crucial that organizations establish ethical guidelines and regulatory frameworks that hold those responsible for the development and use of AI systems accountable for any discriminatory outcomes.

Practitioners need to be comfortable in expressing their mind or thoughts about fairness in the organization , some of the participants in Madaio et al. (2020) and holstein2019improving said they felt fairness was a personal concept for everyone, like the burden was left on them to handle and they feared to voice their opinions as they may not algin with their organization or superior’s. So organizations need to step in and mitigate this problem by supporting them to voice out their thoughts, and understand that the burden is not only on them but on everyone. The prioritization of fairness by organizations is important for their practitioners, their global image, and the consumers of their products and services. Organizations can prioritize fairness by operationalizing it, creating an explicit infrastructure to support it, and ensuring protection for employees interested in implementing it (Richardson & Gilbert (2021)).

Therefore, creating a culture of fairness within organizations requires buy in from leadership and appropriate incentives for practitioners to prioritize fairness, until then fairness will keep on raising issues in many organizations.

Choice Of Data And Processing¶

Many practitioners in several studies have voiced their thoughts on having difficulties in determining what type of data to use to achieve a fair model. Ferrara (2023) highlights that one of the major cause of unfair or biased system is because of wrong choice of data. They emphasize that the best way to mitigate this is using the Group fairness and Individual fairness approach . Where Group fairness is ensures that different groups are treated equally or proportionally in AI systems. This can be further subdivided into approaches like demographic parity, disparate mistreatment, and equal opportunity. These can be achieved through various techniques such as re-sampling, pre-processing, or post-processing of the data used to train the AI model. For example, if an AI model is trained on data that are biased toward a particular group, re-sampling techniques can be used to create a balanced dataset,ensuring each group is equally represented. On the other hand, individual fairness focuses on ensuring that AI systems are fair to individuals, regardless of their group membership. This approach aims to prevent the AI system from making decisions that negatively affects certain individuals. Techniques like counterfactual fairness and causal fairness are used to achieve individual fairness. For example, counterfactual fairness aims to ensure that the AI model would have made the same decision for an individual, regardless of race or gender. However, these approaches are without limitations. For example, group fairness approaches may result in unequal treatment of individuals within a group, while individual fairness approaches may not address systemic biases that affect entire groups.

While Holstein et al. (2019) study the participants faced difficulties in identifying relevant subpopulations. Teams struggle to anticipate which subpopulations should be considered, as these can vary significantly by context and application. Additionally,limited pre-deployment detection of biases and fairness issues often go undetected until after deployment, when customer complaints or negative media coverage the problems to light. Teams also lacked adequate tools to detect biases in underrepresented groups before deployment, even with user studies. Furthermore Domain-specific challenges exist, as attributes like ethnicity and gender may not sufficient to define subpopulations. To solve these issues, its important to recognize that fairness related subpopulations depend on the context and domain (For example, distinguishing between native and non-native speakers in writing evaluation). Teams should also focus on providing tools and processes to anticipate fairness issues before deployment, rather than relying on external complaints or media coverage. Improving pre-deployment bias detection through better user studies and subpopulation analysis can help address this gap. Madaio et al. (2022) highlights the challenges faced by participants when collecting datasets for disaggregated evaluations, especially when trying to balance organizational privacy requirements with the need for demographic data. One participant expressed this challenge by saying, “So I guess I’m just having trouble getting over the hurdle that I don’t think we have a real approved data collection method (for data that lets us evaluate fairness) at all”. For this team, as well as others, the importance of protecting personally identifiable information restricted their access to demographic data, a tension identified across multiple domains in prior research. Teams face difficulties due to the absence of established methods or expertise in collecting demographic data for fairness evaluations. To address this,teams are exploring methods to collect demographic data while adhering to privacy requirements. For instance, they are considering “side channel data collection methods”, though these methods are not perfect. Some teams are considering using internal company data or conducting voluntary surveys to gather demographic information, although this is not ideal due to representativeness concerns while others teams are encouraged to consider domain specific approaches and make efforts to understand the broader demographic spectrum related to their applications.

Comparative Analysis¶

The rapid development of AI and ML in various domains has sparked the need to ensure fairness in the development and deployment of these systems. However fairness in AI is without challenges,including biases in data, inadequacies in fairness toolkits, and the socio-technical complexities of operationalizing fairness principles. This section conducts a comparative analysis of nine key research papers that addresses these challenges, offering insights into existing methodologies, gaps, and best practices. The analysis aims to synthesize findings to inform the creation of a robust rubric or checklist for AI fairness.

Frame and Toolkits¶

This section examines existing frameworks and toolkits designed to support fairness in AI development. The study of Richardson et al. (2021) aims at improving ML toolkits to best fit users use, based on their research they found that existing toolkits though helpful were not often used to help practitioners meet their goals. These are far from practitioners expectations. They point out that fairness toolkits have a great influence on practitioners deploying their models. They emphasize the need of organizations supporting their practitioners with adequate resources to learn about fairness, fairness experts should create multipurpose tools that can work on a large variety of tasks not only specific tasks, while also emphasizing on the need for practitioners to choose tools that best suit their needs and communicate with fairness experts giving their feedback for better improvement. They ended up by providing a rubric for evaluating fairness toolkits, helping practitioners choose appropriate tools for their needs.

Despite these great assessment the rubric doesn’t point out how to help these practitioners choose the right datasets for their models. Several researches have highlighted that the main source of biases stems from the type of data used to train these AI or ML models. As Ferrara (2023) states that the main causes of bias and unfairness arises from unbalanced data and a way of solving it is resampling or pre-processing the data to better the output of the Ai model.

Recommendations/ checklist of best practices¶

In this section we are going give recommendations on how to achieve fairness in AI and ML systems and also create a checklist that guides practitioners in choosing the right data, right tool, metric how to identify and mitigate biases in their models.

Recommendations¶

The recommendations provided in this review are critical for translating theoretical insights into actionable practices that foster fairness in AI systems. Building on the challenges and findings identified, these recommendations aim to guide organizations, practitioners, and policymakers in developing and deploying more equitable AI technologies. By addressing issues such as data bias, practitioner engagement, and toolkit usability, these suggestions offer a future to enhance accountability, inclusivity, and transparency. The practical implementation of these recommendations has the potential to mitigate algorithmic harms and build trust in AI systems across diverse applications.

Data Collection and Processing¶

To ensure fairness in AI systems, prioritizing representative datasets is important. This involves collecting data that accurately reflects diverse subpopulations, especially those historically underrepresented or marginalized. Emphasizing privacy preserving techniques is equally important, striking a balance between ethical data use and compliance with privacy regulations. By adopting these practices, organizations can address biases in training data and create systems that perform equitably across different demographic groups, thereby promoting inclusivity and reducing potential algorithmic harms. To make sure that data is well balanced to train a model, the data can be resampled, preprocessing or postprocessing.

Priority of Action¶

Practitioner Training and Awareness¶

Promoting practitioner training and awareness is essential for embedding fairness principles in AI development. Educational initiatives should focus on equipping practitioners with a deep understanding of fairness frameworks, ethical considerations, and their real world implications. Workshops, training modules, and ongoing professional development can help practitioners recognize biases, interpret fairness metrics accurately, and make informed decisions. By ensuring the training of practitioners, practitioners can come to understand the concept of fairness, define fairness based on their task , know how to implement fairness, and be able to take full responsibility for anything that goes wrong. By fostering awareness and building expertise, organizations can empower teams to prioritize fairness throughout the AI lifecycle, ensuring equitable and socially responsible outcomes.

Priority of Action¶

Tools Design and Usability¶

Fairness tools must be designed to support practitioners in understanding and mitigating biases without fostering over-reliance on the tools. These tools should have the following improvements :

1. Created in such a way that aligns with the existing workflow of the practitioners.
2. They should be easy to understand, have clear explanation of fairness metric and their applicability.
3. They should be able to work on different types of models, not just a one size fits all tool.
4. These tools should be created in a way that practitioners can give their feedbacks on them, communicate with fairness experts so they can ask for help were need be.
5. Tools should be able to detect and mitigate bias and unfairness throughout the life cycle of the model.
6. Tools should include mechanisms for testing tools across diverse contexts (real world scenarios and user settings) to ensure that they work effectively across different demographics, industries, cultural settings, or application domains.

Priority Actions¶

Stakeholder Engagement¶

Effective stakeholder engagement is pivotal for ensuring AI systems meet diverse societal needs. A co-design approach should be employed, involving input from a wide range of stakeholders, including marginalized communities, domain experts, and end-users. By fostering inclusive collaboration, organizations can identify and address potential biases, ensure representational fairness, and align AI systems with ethical and social priorities. Regular feedback loops and transparent communication can strengthen trust and improve the alignment of AI outputs with real-world requirements.

Priority Action¶

While these recommendations provide a roadmap for achieving fairness in AI systems, it is imperative for practitioners to recognize their critical role in this process. These guidelines are ineffective without their active engagement. Practitioners must not only implement these strategies but also continuously reflect on their practices, providing valuable feedback on fairness tools to both experts and organizations and adhere to organizational innovations designed to foster fairness. Their involvement is key to refining tools, addressing unforeseen challenges, and ensuring that fairness becomes an integral part of AI development and deployment.

Checklist of Best Practices¶

This section introduces a comprehensive checklist derived from an in-depth literature review of nine foundational research papers on AI fairness. The goal is to provide practitioners, researchers, and organizations with actionable guidelines to identify, evaluate, and implement best practices for fairness throughout the AI development lifecycle. By synthesizing challenges, tools, and methodologies discussed in these studies, the checklist offers a structured framework to address issues such as bias mitigation, stakeholder involvement, and fairness evaluations. It serves as a practical tool for fostering accountability and equity in AI systems.

First we provide a high level checklist and then a scoring rubric.

Pre-processing phase¶

1. Model Overview :
   • Clearly articulate the purpose, scope, and intended application of the AI model.
   • Specify the problem it aims to solve and the context in which it will operate.
2. Stakeholder Identification and Fairness Definition:
   • Identify a diverse set of stakeholders, including domain experts, affected communities, and interdisciplinary contributors.
   • Ensure these stakeholders reflect the demographic, cultural, and contextual diversity relevant to the task.
   • Collaboratively define fairness in the context of the specific AI model and its application.
   • Establish clear, task specific fairness goals that are measurable and aligned with stakeholder and organization priorities.
3. Data Collection and Preprocessing:

3.1. Assess the representativeness of datasets

• Are datasets reflective of the intended user population?
• Are marginalized subgroups sufficiently represented?

3.2. Evaluate privacy vs fairness trade-offs.

• Have privacy-preserving methods been employed without compromising demographic diversity?

3.3. Address demographic data gaps.

• Is there a strategy to identify and fill blind spots? strategies like
  • Assessing where demographic data is missing or underrepresented by comparing available data against relevant subpopulations for the AI task.
  • Using surveys, focus groups, or external data sources to gather more diverse datasets. Where privacy concerns arise, privacy-preserving methods like synthetic data generation or differential privacy can help.

4. Bias Detection and Mitigation:
   • Have fairness metrics been applied to specific subgroups?
   • Are bias mitigation techniques(Preprocessing,post-processing and Fair Representation Learning) Implemented?
   • Are mitigation strategies aligned with context-specific fairness goals?
   • Is there an ongoing feedback mechanism to detect new biases?
5. Fairness Metric Selection:
   • Have appropriate fairness metrics been chosen, considering the specific context and potential harms of the AI system?
   • Are tools used to complement domain expertise rather than replace critical judgment?
   • Are decisions about fairness criteria transparent and actionable?

Development Phase¶

1. Model Selection and Training:

   • Mitigation Techniques: Evaluate whether fairness mitigation techniques have been incorporated during model selection and training. Consider approaches like bias correction, fairness constraints, or adversarial de-biasing.
   • Transparency and Explainability: Assess if the model’s decision making process is transparent and explainable. Ensuring clarity in how decisions are made helps practitioners understand and identify potential biases, contributing to more accountable AI systems.

2. Evaluation and Testing:

   • Disaggregated Evaluation: Verify if the model’s performance has been evaluated across various demographic groups. This process helps identify any disparities in accuracy, ensuring that fairness is maintained across diverse populations.
   • Bias Auditing: Conduct regular bias audits to monitor the model’s fairness over time. These audits ensure the model continues to meet fairness objectives and adapt to evolving data or societal needs.

   Rubric for Evaluating Checklist¶

   Criterion	Excellent (4)	Good (3)	Needs Improvement (2)	Unsatisfactory (1)	Weight
   Model Overview	Clear purpose, scope, and context defined with specific objectives.	Well-defined, but could be more specific.	Vague or incomplete definition of purpose and scope.	No clear definition of purpose or scope.	10%
   Stakeholder Identification and Fairness Definition	Diverse stakeholders identified with specific fairness goals set.	Most stakeholders identified, goals mostly clear.	Limited stakeholder involvement and fairness goals.	No stakeholder involvement or unclear fairness goals.	15%
   Data Collection and Processing	Comprehensive, representative datasets with privacy considerations without compromising fairness.	Datasets generally representative, some gaps,privacy concerns considered but some trade-offs.	Some missing or unbalanced datasets that is significant gaps in representation of key subgroups and trade-offs not sufficiently addressed or balanced.	Data collection not reflective of population or privacy concerns ignored.	30%
   Bias Detection and Mitigation	Comprehensive bias mitigation strategies (Preprocessing, Post-processing, Fair Representation Learning) implemented.	Some bias mitigation strategies used with minor gaps.	Only basic bias mitigation approaches used.	No bias mitigation strategies used.	15%
   Fairness Metric Selection	Clear, context-specific metrics chosen with transparent, actionable decisions.	Good selection of metrics with reasonable transparency.	Metrics chosen without full alignment with context.	No clear fairness metrics or criteria selection.	10%
   Model Selection and Training	Fairness mitigation techniques incorporated from the start.	Some fairness techniques applied, but not comprehensive.	Techniques implemented too late in the process.	No fairness techniques applied during training.	10%
   Evaluation and Testing	Thorough disaggregated evaluation and regular bias auditing.	Adequate evaluation with some gaps in auditing.	Evaluation done for limited groups; auditing irregular.	No evaluation or auditing conducted.	10%

Limitations¶

While this literature review provides a comprehensive analysis of AI fairness practices across various domains, it is essential to acknowledge certain limitations that could impact the findings and recommendations:

1. This paper addresses the problem using a limited set of papers, which, while diverse, may not fully capture the range of perspectives and advancements in AI fairness. Emerging research or alternative viewpoints might be underrepresented. Therefore, future work will aim to expand the review by incorporating additional papers to provide a more comprehensive analysis..

2. The review does not delve deeply into the specific challenges and solutions for certain domains or use cases, such as the application of AI in human resource management, healthcare and other industries. Additional research may be needed to explore fairness in more specialized contexts and provide guidance tailored to the needs of practitioners in those fields.

3. As AI technologies and societal expectations evolve, some insights or recommendations from the reviewed papers may become outdated, limiting their long-term applicability. Researchers may need to keep on updating the recommendations and checklists to make sure it meets with the current state of AI

Conclusions¶

This literature review examines key practices, challenges, and strategies for achieving fairness in AI systems, drawing insights from foundational research. It underscores the importance of integrating fairness considerations throughout the AI lifecycle, from data collection to deployment, emphasizing stakeholder engagement, bias mitigation, and the use of fairness-aware tools.

While advancements in fairness research provide valuable frameworks and solutions, the effectiveness of these efforts emphasizes on practitioners’ understanding, continuous feedback, and commitment. Moving forward, fostering collaboration among technologists, ethicists, and policymakers will be essential to building AI systems that are equitable and trustworthy.