A Literature Review On AI fairness Best Practices

Introduction¶

Fairness in AI and ML systems play larger roles in critical decisions such as healthcare, employment, education, and criminal justice fairness has become urgent. Fairness in AI is not just technical it is a moral and societal necessity. It determines if algorithms support justice, equality, and human rights. Biased AI can have serious effects, like denying candidates jobs, misdiagnosing patients, or causing racial disparities in policing and lending. Besides direct harm, unfair systems erode trust and reduce AI adoption. They also create ethical, reputational, and legal risks. Fairness in AI means preventing systematic bias or discrimination. Outcomes should be fair across all social and demographic groups. Achieving fairness remains complex. Biases can come from imbalanced data, flawed algorithms, or from the organizational, economic, and societal forces that shape development and use. Solutions need more than technical fixes. They require ethical reflection, transparency, and stakeholder involvement throughout the AI lifecycle.

In recent years, researchers and institutions have developed a range of tools and frameworks to operationalize fairness in practice. Tools include software or platforms that help practitioners detect and mitigate biases in AI or ML systems, some examples of these tools are Aequitas (University of Chicago; Saleiro et al. (2018)), AI Fairness 360 (IBM; Bellamy et al. (2018)), Fairlearn (Bird et al. (2020)), and Google Fairness Indicators (Google (n.d.); Google (n.d.)) provide practitioners with methods to detect, quantify, and mitigate bias in datasets and models. While frameworks provide structured approaches and methodologies for incorporating fairness in AI deployment and development lifecycle, some of these frameworks are: Fairness for Socio-technical Process as examined by Richardson & Gilbert (2021) and Holstein et al. (2019), Co-Design Framework for Fairness as examined by Madaio et al. (2020) and Richardson et al. (2021), Fairness Principles into Practice Framework as highlighted by Beutel et al. (2019) and Decission Tree Framework as examine by Smith et al. (2023). These tools and frameworks represent a growing effort to operationalize fairness in AI. They provide practitioners with concrete resources and guidance for building more equitable and inclusive AI systems. However, it is important to know that simply using these tools and frameworks does not guarantee fairness. For instance, a tool designed to detect bias in datasets may fail if practitioners do not fully understand how to interpret or act on the results, leading to continued biases in the final AI system. There is a gap in these advancements which is translating these advancements to actionable tools that can be adopted by industry practitioners and organizations. Moreover, the rise of LLMs further amplifies these risks. Without clear, written, and empirically validated frameworks, LLMs may perpetuate and exacerbate biases, making it crucial to establish robust standards for fairness in automated decision making systems. Together, these efforts mark real progress toward fair, responsible AI. Yet turning these advances into everyday practice is still difficult. Many organizations struggle to apply fairness principles. Limited expertise, conflicting goals, and unclear guidance are barriers. Even with tools, practitioners may lack support in choosing metrics, interpreting results, or integrating fairness into existing workflows. Rapid adoption of large language models creates new risks. Their opacity and scale can increase historical biases unless guided by strong, validated standards.

This literature review explores how fairness in AI and ML is conceptualized and implemented, highlighting technical, organizational, and societal challenges. Drawing from over forty peer-reviewed papers published between 2018 and 2025, the review examines automated decision-making systems in hiring, lending, healthcare, and law enforcement, covering structured data. The goal of this review is to connect theory and practice showing how fairness is defined, where implementation breaks down, and which tools and frameworks help bridge the gap between ethical ideals and everyday AI development. This analysis is guided by four central research questions:

By comparing technical toolkits, co-design frameworks, and policy initiatives, this review highlights how fairness is shaped by the interaction of technical design choices, organizational structures, and societal values. It identifies recurring barriers, such as unclear accountability, limited practitioner guidance, and minimal stakeholder participation, which undermine consistent adoption of fairness practices.

This review synthesizes prior research and proposes a practical fairness checklist and rubric to help practitioners and organizations embed fairness throughout the AI lifecycle.

What is Fairness ?¶

Fairness, as it pertains to AI is a very complex and contextual concept, over the years many researchers have had different definitions to fairness, making the decision of which fairness methodology to adopt and which corresponding metrics to quantify highly contextual. How fairness is defined and measured often depends on the specific goals, tasks, and social situations involved. Fairness brings together ethical, social, and technical issues, and there is no one-size-fits-all approach. Fairness in AI can be measured in several ways. Group fairness aims for equal outcomes across groups and includes metrics like demographic parity (making sure different demographic groups receive similar results) and equalized odds (ensuring error rates are similar for each group). Individual fairness focuses on treating similar people similarly, such as making sure two applicants with similar qualifications are given the same chance. Causal or counterfactual fairness tries to remove unfair influences from decisions for example, checking if a decision would have been different for a person had they belonged to another group. In industry settings, fairness is often viewed as a practical and context-dependent goal rather than a strict mathematical property. For example, in recommender systems like YouTube or Netflix, fairness involves ensuring algorithms do not favor or harm specific groups, such as users, advertisers, or content creators. Practitioners tend to balance fairness among multiple stakeholders and define it in ways that fit their organizational and product contexts rather than following fixed academic definitions (Yan et al. (2025)). These different metrics can sometimes conflict, so teams have to make careful, transparent choices about which to prioritize (Mehrabi et al. (2021)). But fairness is not just about numbers, it’s also about how we collect data, design models, create tools, and make decisions within organizations (de Troya et al. (2025)). Instead of one universal meaning, fairness in AI is shaped by many factors and changes depending on the situation. It needs to be considered at every step, from how data is collected to how models are evaluated and used (Richardson & Gilbert (2021)). Some scholars argue that fairness is fundamentally a social and legal property, not just a technical one (Selbst et al. (2019)). For example, abstraction in computer science often treats systems as “black boxes,” focusing on inputs and outputs while disregarding the underlying processes. In machine learning, a neural network might classify images (e.g., as ‘cat’ or ‘dog’) without revealing how it makes these decisions. Similarly, fairness is frequently reduced to quantifiable metrics such as equality of outcomes (e.g., demographic parity or minimizing disparate impact) or equality of opportunity (e.g., equalizing FPR or FNR rates across groups). While these metrics are mathematically convenient, they can oversimplify nuanced social concepts and neglect procedural, contextual, and contestable aspects of fairness. Relying exclusively on such metrics risks ignoring the broader sociotechnical systems in which AI operates, as well as the values, histories, and institutions that shape what fairness means in practice. Given these challenges, researchers argue that we should look at fairness through both technical and social lenses, considering how people, institutions, and social systems interact with AI (Selbst et al. (2019)).

Bias and unfairness in AI come from both technical and social sources, so fixing them means combining technical solutions with changes in how organizations work (Holstein et al. (2019)). Attempts to define fairness universally tend to fall short; instead, fairness must be tailored to the goals, tasks, and contexts at hand. Practitioners often struggle with key challenges, such as determining when to incorporate fairness into their workflow and how to define fairness in a meaningful way for their specific context. This contextualization is reflected in practical tools and rubrics developed to help practitioners identify and mitigate bias in machine learning systems(Madaio et al. (2020)), as well as in definitions that focus on the challenging goal of eliminating bias and discrimination from AI (Ferrara (2023)). A key dimension of fairness is the recognition of different types of harm, allocative harms (where resources or opportunities are denied to certain groups) and representational harms (where systems reinforce stereotypes, misrepresent, or neglect certain groups) (Suresh & Guttag (2021)). For instance, a hiring algorithm may disproportionately filter out qualified candidates from underrepresented groups due to biases in training data (allocative harm), while an image classification model may mislabel individuals from certain regions or ethnicities, making them harder to find in searches or classifying them inaccurately in harmful ways (representational harm). These examples illustrate why context-sensitive definitions and interventions are necessary for addressing fairness in practice. Representation bias is one source of representational harm. It occurs when the data used to train a model under-represents certain segments of the population, leading to a failure to generalize well for those groups. This bias can emerge in several ways:

1. Defining the Target Population: If the target population does not reflect the use population, the model may fail to perform adequately. For instance, data collected from Boston might not generalize to Indianapolis or even to Boston’s population 30 years later.

2. Underrepresented Groups: Even if the use population is accurately defined, minority groups within that population may not be sufficiently represented. For example, in a medical dataset targeting adults aged 18–40, pregnant individuals might constitute only 5% of the data, making the model less robust for that subgroup.

3. Sampling Limitations: Biases can also arise during data collection. For example, in healthcare datasets, there may be an over-representation of patients with severe or critical health conditions, such as those who require emergency treatment or hospitalization. These individuals are often considered higher priority cases, so the data may focus more on them than on those with mild or chronic conditions. As a result, the training data may become unbalanced, making it less reflective of the general population and potentially leading to less accurate predictions or recommendations for people with less severe health issues. An illustrative example of representation bias is the ImageNet dataset, a widely-used collection of labeled images. Although it is intended to generalize to all natural images, 45% of its images were sourced from the United States, with very few from countries like India (2.1%) and China (1%). As a result, classifiers trained on ImageNet perform poorly on images from underrepresented regions, such as India and Pakistan, which leads to representational harms in recognizing objects or people. The distinction between allocative and representational harms underscores the importance of understanding fairness in terms of both outcomes and inclusion. By addressing these types of harms, practitioners can work toward more equitable AI systems that mitigate biases and their adverse impacts.

Despite all these different definitions from these researchers one thing is clear, fairness is not a fixed criterion that can be universally applied. Instead, it needs to adapt to the specific circumstances, contexts, and stakeholders involved in each situation.

Importance of fairness¶

The importance and use of AI is growing rapidly in society and its increasing influence in decision making in domains such as healthcare, education , hiring , aviation and criminal justice. It is important for these AI systems to be fair and unbiased, as unfairness and bias can perpetuate or even exacerbate societal inequities, causing harm to marginalized groups and undermining public trust in AI technologies. Bias in AI arises from data, algorithms, or social factors such as stereotypes, so without careful attention, AI can reproduce and amplify the unfairness already present in society (Holstein et al. (2019); Ferrara (2023)). For instance, biased hiring algorithms may disadvantage underrepresented groups; unfair healthcare AI can worsen outcomes for minorities; and government systems that ignore real-world context can strip away human judgment and legal protections, leading to rigid, unfair automation (de Troya et al. (2025)). In the Public Employment Service (PES) case, a lack of fairness checks or complaint processes led to discrimination and violations of the law that went unnoticed. Ensuring fairness is not just about avoiding harm. It is about making AI ethical and aligned with social values, preventing both representational harms (such as misclassifying groups) and allocative harms (such as unfairly distributing resources) (Richardson & Gilbert (2021)). However, achieving fairness is challenging. People will not trust or use AI if it is seen as unfair, but fairness is difficult to define, measure, and enforce (Madaio et al. (2022)). Organizations struggle to assign responsibility and balance different views, but involving fairness from the start can help build trust among all those affected (Madaio et al. (2020)). Practical challenges like a lack of good tools make fairness hard to achieve, but focusing on fairness can stop unfair systems from being released (Balayn et al. (2023); Beutel et al. (2019)). Improvements to fairness toolkits can help developers build better systems (Richardson et al. (2021)).

The risks go beyond harming users. Unfair AI can cost organizations money, cause legal trouble, and damage reputations (Pant et al. (2024)). when organizations release unfair or biased AI systems, the consequences can be serious and far-reaching. Unfair outcomes can lead to financial losses, such as product recalls, lawsuits, or the cost of rebuilding damaged systems. Companies may also face legal problems if their AI models violate anti-discrimination or data protection laws. Beyond money and regulation, unfair systems can cause reputation damage, leading to a loss of user trust and negative public attention. For example, if an AI tool used in hiring or lending is found to disadvantage certain groups, it can harm the company’s image and reduce confidence in its products. These outcomes show that fairness is not just an ethical issue, it directly affects a company’s long-term success and public credibility. Even audits meant to ensure fairness can fail if they are incomplete or misleading, giving a false sense of fairness and allowing bias to persist. For example, (Gerchick et al. (2025)) studied 44 public audits under New York City’s Local Law 144, and found that more than half contained incomplete or misleading data, with many excluding smaller demographic groups or misusing fairness metrics. Some reports used inconsistent impact ratio calculations, while others left out protected groups entirely. These weak audits give a false sense of fairness, allowing biased systems to continue operating unchecked. This shows that fairness cannot depend on weak or symbolic audits without transparency, independence, and enforcement, such processes may actually hide unfairness instead of fixing it.

Without fairness, AI systems risk increasing inequalities and losing public confidence. As a result, embedding fairness into AI development and deployment is not just a technical issue or solution but also socio-cultural one. Building fair AI is essential for both ethical reasons and long-term success, for individuals, organizations, and society as a whole.

Challenges And Barriers In Achieving Fairness¶

Achieving fairness in AI is difficult because it involves technical, organizational, and societal challenges that are all connected. Technical challenges are about how we design and test algorithms and data. Organizational challenges include company policies, team practices, and available resources. Societal challenges come from broader social structures and biases that shape technology and its use. These categories influence each other: for example, what a company values can shape technical decisions, while societal attitudes can affect both company practices and the data collected. Understanding these links helps us see why fairness problems are so tough to solve.

Technical Challenges¶

Technical challenges can be divided into data and algorithmic bias. Data bias refers to issues in the collection, representation, and quality of data, while algorithmic bias originates from model design, optimization objectives, and the way algorithms process information. Understanding the distinction between these two sources of bias is critical: data bias usually reflects pre-existing societal inequalities and poor sampling, whereas algorithmic bias can emerge even with perfect data due to modeling decisions, optimization trade-offs, or unintended interactions within the system. Many practitioners, as shown by Cinca et al. (2025), treat bias as only a data issue, focusing on imbalanced datasets or missing demographic representation. While many recognize data collection and labeling bias, they are less aware of other sources such as algorithmic bias,bias introduced by algorithmic design or optimization goals and societal bias, which comes from broader structural inequalities. Practitioners tend to believe fairness can be achieved by improving or diversifying datasets, overlooking how bias can also stem from modeling choices or deployment contexts. This narrow focus leads to solutions that fix surface problems but ignore deeper causes of unfairness. The evidence suggests that we need to look beyond data and consider how algorithms and social context interact to shape outcomes.

Data Bias and Missing Demographic Data.¶

Historical bias in data is one of the most common sources of unfairness. Ferrara (2023) and Richardson & Gilbert (2021) show that if the data used to train AI is incomplete or only represents certain groups, the outcomes will be unfair. For example, if a dataset used for training an AI model on healthcare includes mostly data from young, healthy individuals, it may fail to accurately represent older or sicker patients. This misrepresentation can occur due to the selection of irrelevant attributes, such as only considering certain health conditions or age groups, or through incomplete data collection, such as excluding individuals with disabilities or those from different socioeconomic backgrounds. Holstein et al. (2019) points out that collecting unbiased data is very difficult because real-world data reflects existing social biases. For example, if a facial recognition system is trained primarily on images of lighter-skinned individuals, it may perform poorly on images of darker-skinned individuals, perpetuating existing biases.

Without clear demographic data, it’s almost impossible to spot or fix unfairness. Andrus et al. (2021) shows that while gender and age data are commonly accessible, information on race is rarely available outside specific regulated contexts such as employment or finance. Legal restrictions, privacy concerns, and organizational hesitation often prevent teams from collecting or using these sensitive attributes. As a result, practitioners frequently depend on proxy data. For example, inferring race from names or location which can be inaccurate and introduce new forms of bias. This can be described as a “fairness in the dark” problem, where organizations claim fairness without truly understanding how their systems affect different groups. Communication gaps between teams, limited resources, and fear of legal consequences make fairness auditing even more difficult. Without demographic visibility, it becomes nearly impossible to identify or fix underlying disparities. The solution, according to the authors, is for technologists, policymakers, and ethicists to work together to develop rules that allow teams to use demographic data responsibly while protecting privacy.

Algorithmic Bias¶

Algorithmic bias can arise even when the data appear unbiased. As defined by Ferrara (2023), such bias occurs when machine learning algorithms produce outputs that reflect inherent distortions stemming from their underlying assumptions or decision criteria. Rather than stemming solely from data, these biases often arise from how algorithms process information and optimize for specific outcomes. For example, the National Institute of Standards and Technology (NIST) Grother et al. (2019) found that facial recognition technology was significantly less accurate for people with darker skin tones, resulting in higher false-positive rates. This disparity persists even when datasets include diverse images, suggesting that the issue lies partly in how algorithms encode facial features, handle lighting conditions, and set decision thresholds optimized for overall accuracy rather than subgroup fairness. Similarly, in automated hiring systems, biases can emerge when models prioritize features such as education history, career gaps, or age attributes that may correlate with gender or socioeconomic background. These patterns are often amplified by optimization objectives that reward “efficiency” or “fit” based on historically biased hiring practices. Another well-known example is the COMPAS risk assessment system used in the United States’ criminal justice system, which predicts a defendant’s likelihood of reoffending. A study by ProPublica found that the system was biased against African-American defendants, as they were more likely to be labeled as high-risk even if they had no prior convictions. This bias is not solely due to data representation but stems from the selection of features such as prior arrests, employment status, and residential location, which reflect structural inequities in policing and socioeconomic opportunity. Collectively, these cases demonstrate that algorithmic bias is a sociotechnical issue, rooted in model design, feature selection, and evaluation practices as much as in the underlying data itself.

Despite the growing number of fairness interventions, achieving technical fairness remains challenging because bias arises at multiple points in the machine learning pipeline not only in data but also in model design and deployment. As Beutel et al. (2019) notes, practitioners often face trade-offs between efficiency and fairness, where optimization for accuracy or speed can unintentionally disadvantage underrepresented groups. This challenge is evident in real world applications like, loan approval algorithms in financial institutions have been found to discriminate based on race, gender, and zip code. These algorithms may use proxy variables that indirectly encode racial or gender bias. Another example is search engine and job recruitment platforms google search for professional job names might return more images of men, while searches for certain service jobs might return more images of women or specific racial groups. Similarly, Cinca et al. (2025) shows that algorithmic bias can also emerge from the design and modeling choices made by practitioners themselves: choices in feature selection, hyperparameter tuning, or performance metrics may amplify disparities even when data are balanced. Fairness cannot be achieved through isolated technical fixes, such as data cleaning or bias detection tools, but requires deeper reflection on design goals and evaluation practices.

There is broad agreement that technical fairness cannot be achieved by a single intervention, such as cleaning data or applying bias-detection tools. As Ferrara (2023) and Richardson & Gilbert (2021) stress, data quality is necessary but insufficient algorithmic fairness also depends on how systems are conceptualized and integrated into broader social and organizational contexts. Meanwhile, Cinca et al. (2025) and Beutel et al. (2019) reveal that algorithmic design choices and optimization goals can introduce new forms of bias even with balanced datasets. Building on this, Holstein et al. (2019) and Andrus et al. (2021) demonstrate that fairness is further constrained by real-world data limitations, including missing demographic information and legal or institutional barriers to data access. Taken together, these studies illustrate that technical fairness must account for the entire ML pipeline data, model design, and deployment context while acknowledging the societal and organizational factors that shape how these systems operate. Addressing technical bias, therefore, requires not just better datasets or algorithms, but coordinated efforts across data governance, model interpretability, and ethical oversight.

Organizational Challenges¶

Beyond technical limitations, many barriers to fairness arise from organizational structures, priorities, and cultures. A recurring issue across studies is that fairness work is often treated as optional or secondary, rather than as a shared institutional responsibility.

Lack of Awareness and Expertise¶

A major challenge is the lack of awareness and expertise within organizations. Many practitioners lack training in fairness concepts or struggle to translate ethical goals into technical actions. Studies by Madaio et al. (2020) and Madaio et al. (2022) show that practitioners often need clearer guidance and structured resources, such as checklists and organizational frameworks, to assess fairness effectively. Addressing fairness in AI is a complex, ongoing process requiring a multi-disciplinary approach involving computer science, law, ethics, and social science experts (Pant et al. (2024) and Ferrara (2023)). This knowledge gap makes it difficult for practitioners to identify subtle biases or understand how technical design choices affect different social groups. Without sufficient training and resources, organizations may struggle to implement fairness principles in practice. By developing and applying a range of approaches, we can work toward AI systems that are more unbiased, transparent, and accountable.

Conflicting Objectives:¶

Ensuring fairness in AI is rarely a straightforward issue; instead, it unfolds in a landscape of competing priorities and organizational realities. For many organizations goals like accuracy, efficiency, or reputation often take precedence over fairness. As Holstein et al. (2019) and Madaio et al. (2020) reveal, practitioners face pressure to deliver products quickly and maintain competitiveness, which can sideline fairness concerns. One practitioner put it plainly: “I get paid to go fast... We’re not allowed to spend three years developing a product. We will die. Our competitors are on a weekly cadence. Do I do a good thing, or do I do the thing that ships the product?” This practical tension means that, at times, fairness toolkits are used only superficially Deng et al. (2022), and bias may be tolerated if it helps meet business targets. In such environments, fairness can become an afterthought, rather than an integral part of AI development. Even when fairness is on the agenda, technical barriers complicate progress. Many toolkits such as AIF360 and Fairlearn offer a wide range of fairness metrics but little guidance on choosing or interpreting them, as Lee & Singh (2021) notes. This leaves practitioners, especially those without deep academic backgrounds, confused by overlapping definitions and unfamiliar terminology. For example, the same underlying concept might be labeled as “false-negative-rate difference” in one toolkit and “equal-opportunity difference” in another, with no explanation that they refer to the same thing. Without deep academic knowledge, practitioners cannot confidently navigate this definitional diversity, making toolkit selection error‑prone and undermining the promised plug‑and‑play workflow integration. Most toolkits focus mainly on group fairness and rarely support individual or causal fairness, leaving important fairness types unaddressed. To be truly helpful fairness toolkits must offer clearer guidance, easier interfaces, and better alignment with real-world workflows.

The technical confusion is only part of the picture. As Wong et al. (2023) argues, most AI ethics toolkits frame fairness as a technical problem that individual engineers can fix, rather than as a shared responsibility across teams. This limits collaboration between technical and non-technical staff and leaves little space for input from policymakers, designers, or affected communities. Although many toolkits encourage stakeholder engagement, they provide no clear methods for involving those stakeholders or addressing internal power dynamics. As a result, fairness decisions often remain in the hands of small technical groups, and ethical discussions are disconnected from wider organizational goals. To move forward, organizations need to create toolkits and processes that promote collaboration, collective accountability, and cross-functional dialogue, ensuring that fairness becomes part of the broader culture rather than just a technical task. Finally, even the best tools will not succeed unless practitioners see their value and can easily fit them into their routines, as Voria et al. (2024) highlights. Practitioners are more likely to use fairness tools if they clearly improve their work and fit naturally into their normal development workflows. Clear documentation, organizational support, and seamless integration are key to making fairness tools part of regular practice. Addressing fairness, is not just about better technology it is about tackling technical, organizational, and behavioral barriers together. Only by integrating these perspectives can we make fairness a practical reality in AI development.

Checkbox Culture¶

A growing concern in the field is the rise of a “checkbox culture” in which organizations focus on following fairness rules rather than addressing real ethical or social problems. Rather than prompting meaningful engagement, fairness checklists often become tools for superficial compliance, providing a false sense of security and masking deeper biases (Balayn et al. (2023)). For instance, an organization might use a checklist to demonstrate that its AI models meet specific standards, but fail to examine their actual impact across different groups. This superficial compliance can impede real progress and conceal systemic problems. This pattern is reinforced when fairness tools are used as generic, context-free resources, leading teams to complete checklists for compliance rather than thoughtful action (Madaio et al. (2024)). Without adaptation or ownership, these tools become administrative tasks instead of opportunities to build a genuine understanding of fairness. As a result, organizations neglect the crucial work of contextualizing and putting fairness into action, reducing the positive impact such tools could have. For organizations to foster genuine progress, they must move beyond procedural compliance and actively confront the underlying complexities of fairness in context.

Societal challenges¶

Societal challenges in AI fairness go beyond what happens inside organizations they involve complex questions about what fairness means for different people and groups in society. One of the most pressing societal challenges in AI fairness is the difficulty of defining fairness itself.

Defining Fairness: A Complex and Contextual Challenge¶

Fairness means different things depending on who you ask and the situation. There is no single way to define fairness that works everywhere. In hiring, fairness might mean everyone gets an equal chance; in healthcare, it could mean giving more help to those who need it most. Because what is considered fair can change depending on the context, it is hard to create AI systems that are fair for everyone. Many practitioners struggle to understand fairness because they are not taught what it really means (Richardson et al. (2021); Richardson & Gilbert (2021)). Without clear standards, it is hard to judge if an AI system is fair or not. Fairness tools and metrics often do not fit real-world problems, and many existing fairness tools are difficult to use or cannot be applied to large, complex systems. Different people in the same company like engineers, designers, and researchers also interpret fairness in different ways, which makes collaboration difficult Deng et al. (2023). The lack of a clear definition and understanding of fairness can make it difficult for practitioners to apply and evaluate fairness in their work effectively. Practitioners often feel anxious because they do not know what questions to ask about fairness or how to find hidden problems. For example, one participant said: “I think that part of the problem is people don’t know what to ask. People want to do the right thing, but they don’t know what the right thing is.” Madaio et al. (2020). This shows that even when people want to do the right thing, they often lack the knowledge or support to make sure their AI systems are fair.

Selbst et al. (2019) warns that trying to define fairness in fair ML often rely on mathematical formulas. While these definitions can be translated into algorithms, they fail to capture the full complexity of fairness, which is deeply tied to procedures, social context, and political debate. Fairness is not just about numbers it involves social, cultural, and legal issues too. This called the Formalism Trap, where reducing fairness to mathematical terms oversimplifies its refined and evolving nature. For example, Feldman et al. (2015) formalized the Equal Employment Opportunity Commission’s (EEOC) 80% rule into a measure called “disparate impact.” However, treating disparate impact as a simple numerical threshold misses the procedural steps the EEOC uses to assess whether the impact is justified by business necessity and whether there are less discriminatory alternatives. Limiting the question to a mathematical formulation gives rise to two distinct problems in practice:

1. Conflict between Definitions: Mathematical definitions often conflict, and there is no purely mathematical way to resolve these conflicts. For instance, debates about the COMPAS risk assessment tool have highlighted how different fairness metrics (e.g., equal error rates vs. equal accuracy across groups) can lead to incompatible conclusions about what is “fair.” The resolution of such conflicts requires normative judgments informed by the social context.

2. Inadequacy of Definitions: Mathematical definitions often miss important parts of fairness, like whether a decision is made through a fair process or if it fits with cultural norms. Legal and philosophical experts say fairness is about more than numbers; it is about how decisions are made and the society they happen in.

Practitioners see fairness in two main ways (Pant et al. (2024)). First, fairness means no bias—AI should not treat people differently because of gender, race, or age. For example, a fair hiring algorithm does not prefer men over women with the same skills. Second, fairness is about positive qualities such as transparency, explainability, and reliability. For example, a credit scoring system that clearly explains its decisions is seen as fairer than one that keeps users in the dark. So, fairness is both about avoiding harm and building trust. Making fairness work in AI systems adds more challenges (Abebe et al. (2020)). Creating formal rules can help reveal hidden bias and start conversations about fairness, but it can also make fairness too rigid and miss important context. For example, formal models have to decide which outcomes matter most, but these choices are often practical rather than ideal. This focus on rules and metrics can hide big-picture inequalities, as Selbst et al. (2019) also points out. To really address fairness, we need to connect formal rules with real social values and include more voices in the design process. When fairness is reduced to just formulas, we risk missing what really matters to people. AI systems need to reflect evolving ideas of fairness and justice, so it is important to draw on insights from diverse fields and to include real-world context. Despite these challenges, practitioners have found creative ways to work on fairness. Deng et al. (2023) describes two main kinds of efforts: “bridging” and “piggybacking.” Bridging means building connections between teams that usually work separately such as engineers, UX researchers, and policy specialists, so that they can share ideas and understand fairness from multiple perspectives. Practitioners do this by organizing cross-team meetings, writing internal guides, or creating shared documents about fairness best practices. Piggybacking means attaching fairness work to existing company processes that already have management support. For example, fairness discussions might be added to privacy reviews, compliance checks, or accessibility evaluations. This approach helps practitioners promote fairness even when there is no formal system or dedicated resources for it. These informal strategies allow fairness work to continue within the constraints of real industry environments.

Frameworks and Toolkits for Fairness¶

Achieving fairness in AI depends on combining frameworks that guide ethical thinking with practical tools to identify and tackle bias throughout the AI and machine learning lifecycle. Frameworks offer structured ways to include fairness at every stage, while tools make it easier for practitioners to spot and address problems in their systems. Both are needed to balance technical fixes with broader social and organizational changes, helping ensure fairness is genuinely woven into how AI is built and used.

Fairness Frameworks¶

Frameworks provide conceptual guidance for incorporating fairness in the workflow,helping practitioners understand and apply fairness principles effectively. These frameworks are important because they offer structured approaches to address fairness in complex systems, ensuring that fairness is considered at every stage of development. Below are some of the frameworks discussed in the literature:

Fairness for socio-technical process¶

Fairness in socio-technical processes requires a holistic approach that integrates technical and organizational considerations. Evidence from Richardson & Gilbert (2021) demonstrates that addressing fairness must occur throughout the AI/ML lifecycle, with attention to multiple types of bias and ongoing evaluation from data collection through deployment. Meanwhile, Holstein et al. (2019) provides evidence that tackling unfairness is not solely a technical challenge, but also a socio-technical one, necessitating improvements in organizational processes, policies, and education alongside technical solutions. The literature collectively shows that while fairness metrics are essential tools, their complexity, abundance, and the trade-offs involved (as discussed in both sources) create practical barriers for practitioners, particularly those without advanced statistical expertise. The choice and interpretation of these metrics are context-dependent, and the risk of hidden biases remains if expert input is not carefully managed. Thus, the evidence points to the need for a comprehensive, interdisciplinary approach to fairness that is sensitive to both technical and human factors.

Co-Design Framework for Fairness¶

Co-design frameworks for fairness are most effective when they integrate diverse perspectives and treat human-centered requirements as equal to technical feasibility (Madaio et al. (2020); Richardson et al. (2021)). Bringing together technical and non-technical stakeholders aligns fairness guidelines with both organizational goals and societal values, but implementation can be resource-intensive and challenging for smaller organizations. The importance of ongoing engagement and feedback collection from diverse participants emerges clearly from participant feedback, with sustained input seen as essential for achieving fairness, even as it remains difficult in practice. Further, effective participatory frameworks must account for the diversity of stakeholder knowledge spanning technical, domain-specific, and experiential expertise as well as the particular contexts in which stakeholders interact with AI systems (Suresh et al. (2021)). Recognizing different types of knowledge and practical needs, from high-level concerns like trust and fairness to concrete interpretability requirements, strengthens co-design practices. Grounding fairness and interpretability in the real-world experiences and needs of those most affected makes participatory frameworks more inclusive and responsive to the contexts in which AI systems operate.

Fairness Principles into Practice Framework¶

Beutel et al. (2019) framework focuses on translating high level fairness principles into actionable steps for AI system development. The aim was to improve group fairness, specifically minimizing disparities in FPR between sensitive groups ensuring fairness and product health. Equality of Opportunity was the focus, with emphasis on FPR. A generalized version conditional equality, was introduced to make evaluation decisions explicit and account for varying group difficulties. However, it poses some challenges, as implementation may be resource intensive for smaller organizations.

Practitioner-Centered Framework for Fairness Assessment¶

Madaio et al. (2022) examines a practitioner focused framework aimed at understanding practitioner need and simplifying fairness assessment, making it easier to integrate fairness into existing workflows. The study highlights the need for accessible tools to measure and mitigate bias and encourages collaboration across teams to ensure fairness goals align with organizational values. It also provides guidance for practitioners on auditing datasets and models for biases. However, while the study focuses primarily on detecting fairness issues, it pays less attention to strategies for mitigating these biases.

Ethical AI Development Framework¶

Current research emphasizes that fairness, transparency, accountability, and explainability must be foundational to ethical AI development not only as abstract principles but as practices integrated throughout the AI lifecycle. For example, Ferrara (2023) argue that responsibility for bias and discrimination in AI systems extends beyond the algorithms themselves to the organizations and people who design, deploy, and oversee them. This perspective situates ethical responsibility within broader institutional and social contexts, demanding robust governance and policy structures. Advancing the discussion, Alvarez et al. (2024) propose a comprehensive policy framework that links legal and technical approaches to fairness, as exemplified by the EU’s NoBIAS project. The analysis highlights the necessity of integrating existing legal mandates such as GDPR and the forthcoming AI Act with effective bias management throughout the technical pipeline. However, Alvarez et al. caution against the overreliance on technical debiasing, arguing that clearly defined social harms and multidisciplinary governance (drawing on law, philosophy, sociology, and causal reasoning) are required for trustworthy AI. The recommendations adopting thorough documentation, recognizing bias at every development stage, and embedding participatory design reinforce the notion that ethical AI depends on both policy infrastructure and active inclusion of diverse perspectives.

Young (2025) significantly enriches the ethical AI conversation by shifting the locus of fairness to those most affected by AI insisting that ethical development cannot be achieved solely through top-down governance or technical solutions. Many participatory AI projects only consult communities after key design choices have already been made, leaving little room for meaningful influence. Instead, real fairness requires that impacted communities become co-designers and decision-makers, shaping the goals, risks, and evaluation criteria of AI systems. This approach reframes fairness as not only a technical or legal goal but also a social and political process rooted in power, inclusion, and lived experience and calls for long-term partnerships with marginalized groups, embedding their perspectives into governance structures and resource allocation. Taken as a whole, the contributions illustrate that ethical AI emerges from a dynamic interplay among institutional responsibility, comprehensive policy frameworks, and authentic participatory practices. Each scholarly perspective adds essential layers of accountability, inclusivity, and guidance, reinforcing the imperative that AI systems remain answerable to the very individuals and communities they impact most.

Decission Tree Framework¶

Smith et al. (2023) introduces a decision-tree framework that helps practitioners connect high-level fairness goals with measurable metrics in recommender systems. The framework organizes fairness considerations around three main decision points: who the fairness concern applies to (e.g., consumers or providers), at what level fairness is being assessed (e.g., individual or group), and what kind of fairness objective is being targeted (e.g., exposure, ranking position, or calibration). Through this structure, the framework encourages practitioners to reason systematically about fairness rather than choosing metrics arbitrarily. For example, if a team wants to ensure that artists from underrepresented groups receive comparable visibility on a music recommendation platform, the framework directs them toward provider-side exposure metrics such as “equal opportunity of exposure.” In contrast, if the goal is to ensure users from different regions receive equally relevant recommendations, it points to consumer-side metrics such as calibration or proportional accuracy. Practitioners in the study found the framework useful for clarifying the scope of fairness concerns and identifying suitable metrics; however, they also noted that using it effectively required prior knowledge of fairness terminology and concepts. Some participants expressed uncertainty when deciding between overlapping fairness categories, such as representational versus allocative fairness, or between individual and group metrics. The authors interpret these reactions as evidence that while the framework offers a structured pathway for metric selection, it must be complemented with organizational support, shared language, and educational resources to help practitioners navigate complex fairness trade-offs. Overall, the framework translates fairness from an abstract ethical ideal into a practical decision-support tool, guiding teams in choosing context-appropriate metrics while exposing the reasoning behind those choices.

Tools for Fairness in AI¶

Tools helps practitioners in detecting , evaluating and mitigating biases in AI systems. These tools support practitioners by offering metrics, visualizations, and algorithms to address fairness concerns at various stages of the AI lifecycle, from data preprocessing to post-deployment monitoring. By operationalizing fairness principles, these tools bridge the gap between theoretical frameworks and real world applications, enabling organizations to create more equitable and trustworthy AI systems. Here we look at key tools identified in the literature, highlighting their features, strengths, and limitations.

Google Toolkit¶

Fairness indicator and the What-If toolkit (WIT) are two distinct toolkits created by Google Google (n.d.) and Google (n.d.). Fairness Indicators is primarily designed for high-level model fairness evaluation across subgroups and metrics (Richardson et al. (2021); Richardson & Gilbert (2021)). It enables users to compare up to two models and focus on specific attribute slices, supporting intersectional analysis and performance testing across thresholds. Its interactive interface allows users to adjust metrics and slices, but evidence suggests some users feel overwhelmed by the complexity and amount of statistical information presented (Richardson et al. (2021)). The inability to extract external tables and a need for more resources to guide users are notable limitations, even as the tool is praised for its visualizations (Richardson et al. (2021)). In contrast, the What-If Tool specializes in instance-level exploration and individual fairness analysis (Lee & Singh (2021)). WIT stands out for its unique ability to let users inspect model behavior on a per-instance basis, compare counterfactual points, and run sensitivity analyses, making it especially valuable for understanding how models treat similar data points. Its rich, interactive visualizations facilitate communication with non-technical stakeholders and support exploratory analysis (Lee & Singh (2021)). However, WIT requires users to upload or connect datasets, which can raise data privacy concerns, and assumes familiarity with TensorFlow workflows due to its browser-based interface. It is less suitable for batch automation and enterprise scale deployment (Lee & Singh (2021)). In sum, the literature highlights that while both tools excel at interactive, visualization driven fairness analysis, Fairness Indicators is better suited for aggregate fairness assessment, whereas WIT provides deeper insights into individual fairness.

Uchicago Aequitas¶

Aequitas Aequitas is a robust open-source toolkit designed to audit the predictions of machine learning-based risk assessment tools, with a particular focus on identifying and communicating different types of biases. The tool is recognized for its accessibility through multiple platforms and its ability to evaluate models, calculate group biases, and visualize disparities using color-coded groupings Saleiro et al. (2018). Importantly, several studies highlight both strengths and limitations: Aequitas is praised for facilitating step-by-step fairness audits and aiding non-technical stakeholders in understanding fairness issues Lee & Singh (2021), and for offering transparency and high-level reporting that can inform responsible decision-making Richardson & Gilbert (2021). However, the literature also points to challenges such as the overwhelming volume of metrics which may hinder usability for some participants Richardson et al. (2021), the absence of built-in fairness mitigation techniques Richardson & Gilbert (2021), and concerns about privacy related to report storage in the web version Lee & Singh (2021). Additionally, the tool’s primary focus on group-level fairness limits its flexibility for individual or causal fairness analyses. Collectively, these studies indicate that while Aequitas is valuable for bias detection, high-level reporting, and communication, it is less suited for granular diagnostics or algorithmic mitigation efforts.

IBM’s AI Fairness 360 Bellamy et al. (2018)¶

AIF360 is an open-source toolkit designed to detect, understand, and mitigate algorithmic bias in machine learning models. It offers a comprehensive suite of over 70 bias-detection metrics and 9 bias-mitigation algorithms, supporting both Python and R for primarily classification tasks, though its support for regression is limited. Although AIF360 lacks built-in visualization functions, it provides guidance for integration with tools like LIME, enhancing interpretability (Richardson & Gilbert (2021)). Users have highlighted the toolkit’s broad coverage of fairness techniques and its extensive documentation as strengths that improve usability. Nevertheless, significant challenges remain: difficulties in applying the toolkit to non-binary classification models, limited guidance for regression use cases, and issues communicating complex fairness concepts to non-experts have all been reported (Deng et al. (2022)). Further critique identifies usability issues, such as inconsistent terminology in documentation and a steep learning curve that restricts accessibility for practitioners, despite the toolkit’s technical depth. As with many fairness libraries, AIF360 predominantly emphasizes group fairness, offering little practical support for individual fairness metrics (Lee & Singh (2021)).

Fair-learn¶

Fairlearn is an open-source project designed to help practitioners assess and improve the fairness of AI systems by offering tools for evaluating model outputs across different populations and providing several algorithms for mitigating fairness issues (Bird et al. (2020)). The platform stands out for its usability, as it integrates smoothly with scikit-learn workflows and offers a dashboard for interactive exploration of fairness–accuracy trade-offs, which practitioners have found highly accessible (Lee & Singh (2021)). Fairlearn also fosters an active community and resource hub, not only providing code but also offering domain-specific guides and channels for sharing feedback and best practices, as evidenced by its Discord discussions (Deng et al. (2022)). Despite these strengths, limitations remain: Fairlearn is primarily oriented toward classification rather than regression, relies mainly on group-level metrics such as demographic parity and equal opportunity, and offers limited guidance for selecting fairness metrics tailored to real-world contexts (Deng et al. (2022); Lee & Singh (2021)). Participants in recent studies have identified the need for clearer documentation, more transparency about supported metrics and algorithms, and expanded educational resources to help non-experts interpret fairness results and effectively use the toolkit (Deng et al. (2022)). Overall, while Fairlearn is among the most practitioner-friendly fairness toolkits, its impact would be strengthened by addressing these gaps in guidance and documentation.

LinkedIn’s Fairness Toolkit(LiFT)¶

The LinkedInFairnessToolkit (LiFT) is a Scala/Spark library designed for measuring fairness and mitigating bias in large-scale machine learning workflows, offering a comprehensive suite of metrics applicable to both data and model outputs (Vasudevan & Kenthapadi (2020)). Evidence from Richardson & Gilbert (2021) demonstrates the toolkit’s versatility in metric selection, while its integration in Scala/Spark environments distinguishes it from other fairness toolkits. Richardson & Gilbert (2021) suggests that they could be similar to AI Fairness 360; they do not produce their own visualizations. Unlike previous toolkits, they are built to be applied in Scala/Spark programs.

OxonFair¶

OxonFair Delaney et al. (2024) is an open source fairness toolkit designed to address biases across multiple domains, including tabular data, NLP, and computer vision. It distinguishes itself from other tools through its validation based fairness enforcement, which reduces overfitting by using validation data during optimization. This feature ensures better generalization to unseen data, a challenge often encountered in fairness interventions. Another key strength of OxonFair is its ability to jointly optimize fairness and performance, minimizing the trade-offs typically associated with improving fairness in AI systems. For example, while some tools like Fairlearn and AIF360 focus on specific fairness metrics, OxonFair offers a more flexible and expressive approach, supporting custom fairness measures based on metrics like TPR and FPR. In comparative evaluations, OxonFair outperformed existing tools on challenging domains like NLP and computer vision, where biases are harder to address. For instance, it was shown to reduce disparities in classification performance on datasets such as Jigsaw NLP and CelebA (computer vision) more effectively than baseline methods. This makes OxonFair particularly valuable for practitioners working in diverse applications and seeking robust solutions for fairness.

Scikit-fairness / scikit-lego¶

scikit-fairness Vincent (n.d.) (often referred to as scikit-lego) is an open-source Python library that extends scikit-learn with additional machine learning tools, including fairness and bias-mitigation utilities. It’s designed to help practitioners integrate fairness checks directly into existing ML workflows without switching ecosystems. Lee & Singh (2021) study recognizes scikit-fairness for its tight integration with scikit-learn, which makes it convenient for ML engineers to adopt within existing Python workflows. It supports a smaller set of group-fairness metrics (demographic parity, equal opportunity, equalized odds) and a few pre-processing methods, like information filtering and sampling. The main limitation is its narrow task coverage: it supports binary classification only, with little to no functionality for regression or multi-class protected attributes. The authors suggest that this limits its use in complex, real-world scenarios. Still, they note its potential for wide adoption because it aligns directly with the ML tools developers already use daily.

While the toolkits discussed above, such as AIF360, Fairlearn, What-If, Aequitas, and Scikit-Fairness, show real progress in helping practitioners address fairness, they also share similar weaknesses. They mostly focus on technical solutions, like measuring bias or adjusting models, and give limited guidance on how to include different stakeholders or handle social challenges. Wong et al. (2023) takes a broader view by studying many AI ethics toolkits together to understand how they shape the actual work of fairness and ethics. Their study shows that most toolkits turn fairness into a set of technical tasks for individual engineers, even though they often claim that fairness needs teamwork and diverse perspectives. This creates a gap between what the toolkits say and what they help people do.

Below is a comparative table of all the tools mentioned

Comparative Analysis of Fairness Frameworks and Toolkits¶

Across the literature, fairness frameworks and toolkits share the goal of embedding ethical principles throughout the AI lifecycle, but they approach the challenge from different angles, each with distinct strengths and limitations. Conceptual frameworks such as (Richardson & Gilbert (2021) and Holstein et al. (2019)) emphasize that fairness must be supported not only by technical fixes but also by policy, education, and organizational change. Participatory frameworks, such as those of Madaio et al. (2020) and Richardson et al. (2021), extend this vision by involving a broader range of stakeholders, ensuring that fairness reflects the lived experiences and values of those affected by AI systems. Expanding this participatory vision, Suresh et al. (2021) propose mapping stakeholder expertise across technical, domain, and experiential contexts, while Young (2025) advances this agenda by arguing that fairness must begin with and center the voices of the most affected communities those who experience algorithmic harms firsthand. Such inclusive approaches reframe fairness as a question of power, representation, and accountability rather than a purely technical challenge. Practical frameworks, such as those by Beutel et al. (2019) and Smith et al. (2023), focus on translating abstract fairness ideals into actionable steps or decision pathways, helping teams choose suitable metrics and evaluation strategies. However, their complexity and technical language often demand organizational support to ensure consistent use. Ethical and policy-oriented frameworks, such as those of Ferrara (2023) and Alvarez et al. (2024), emphasize that fairness also depends on transparency, documentation, and legal compliance, and call for stronger governance structures and interdisciplinary collaboration. Complementing these conceptual advances, toolkits such as AIF360, Fairlearn, Aequitas, and Google’s Fairness Indicators operationalize fairness detection and mitigation. Studies by Lee & Singh (2021), Deng et al. (2022) and Balayn et al. (2023). find that while these tools provide valuable metrics, they often lack guidance on metric selection and interpretation, leading to inconsistent or superficial adoption. These shortcomings align with Wong et al. (2023) and Voria et al. (2024), who highlight that fairness tools are frequently misaligned with real-world workflows and are used as one-time compliance checks rather than ongoing practices.

Together, these works show that frameworks offer ethical and conceptual grounding, while toolkits translate these ideas into measurable actions. Yet both face shared limitations: insufficient contextual guidance, limited usability, and underdeveloped mechanisms for stakeholder inclusion. The most effective fairness approaches integrate these elements—combining rigorous technical tools with participatory frameworks and strong organizational accountability—to ensure fairness becomes a continuous, collaborative, and power-aware practice. Below is a table that gives some of the traits and examples of tool, frameworks and guidelines:

Practitioner needs and feedback¶

This section highlights the needs practitioners want from the organizations, tools and fairness experts to help them understand and incorporate fairness in their AI or ML task. We will discuss the needs and feedbacks highlighted by the papers.

Need for Practical Guidance and knowledge¶

Practitioners frequently express a need for clear, actionable guidance to address fairness in their work. Practical tools and accessible frameworks remain a known gap, with many indicating that simply absorbing fairness principles from toolkits is insufficient. Organizations should take a more active role investing in resources and infrastructure that embed fairness as a core value by incentivizing fairness analysis, providing education, and ensuring access to domain experts and effective tools. As Richardson et al. (2021) notes, without these supports, practitioners may feel underprepared and unsure. The responsibility for advancing fairness is institutional and should be supported by pathways for learning and implementation. Multiple studies echo this: for instance, Madaio et al. (2020) highlights practitioner anxiety when lacking methods to identify or mitigate fairness-related harms. Structured resources, such as checklists, empower practitioners to flag concerns and take action, reducing the risk of overlooking fairness issues when guidance is lacking. These tools help create a culture where fairness is routinely addressed, not an afterthought. Education remains critical: practitioners need adaptive guidance that aligns with their goals, contexts, and project challenges. Defining fairness is best when collaborative and contextual, supported by resources that connect theory to actionable steps. Some work, such as Deng et al. (2022) and Richardson & Gilbert (2021), emphasizes fairness experts translating theory to practice, reinforcing the need for organizational support so practitioners can develop expertise. Toolkits should facilitate fairness evaluation while providing educational pathways such as interactive modules, data documentation guides, and procedures to help practitioners confidently apply fairness principles.

Fairness toolkits must guide practitioners step by step, from defining fairness goals to understanding the implications of tools. Real-world educational resources are critical to bridge theory and practice. When organizations invest in these supports, practitioners can make fairness a routine, integrated part of their work.

Context-Specific Considerations¶

Fairness is not a one-size-fits-all concept. Practitioners emphasize the need for approaches that are sensitive to the specific context and potential harms of the AI system being developed. This underscores the importance of choosing evaluation methods that account for the details of each unique application. Existing performance metrics often fall short at capturing fairness-related harms in AI systems. It’s not enough to rely only on traditional measures like precision or recall. Instead, metrics must fit the application and address issues such as demographic disparities and the risk of disproportionate impacts on marginalized groups. Practitioners in Madaio et al. (2022) described how their teams developed new performance metrics specifically for assessing fairness, noting, “We need to start quantifying and optimizing towards not just success, but output quality and how catastrophic failures are. So we invented new metrics around quality.” Some teams wanted to use these new metrics for disaggregated evaluation, while others had difficulty agreeing on whether to adopt them or to stick with their usual performance metrics. This ongoing debate illustrates the challenge of embedding fairness meaningfully, rather than treating it as a checkbox exercise. Aligning fairness definitions with a project’s specific objectives and context is critical. However, this process is rarely straightforward. When fairness lacks a clear, context-driven definition, practitioners can get overwhelmed by the number of available metrics and tools. This can lead to confusion, mistakes, or even paralysis by analysis. The abundance of mathematical definitions and the lack of clear resources add to this complexity. I believe fairness tools and frameworks should be accessible. This empowers users who may not be data scientists but want to engage with the topic. A clear fairness objective is foundational. It guides the selection of tools and metrics that align with the project’s real-world impacts. Some tools are best for group disparities, others for individual fairness or long-term effects. What matters is a thoughtful, context-aware application, as evidenced by Richardson et al. (2021).

Integration with Existing Processes¶

Many practitioners struggle to integrate fairness toolkits into workflows, citing interpretability issues, poor documentation, and misalignment with organizational needs. Recent research (Richardson et al. (2021); Deng et al. (2022); Holstein et al. (2019); Madaio et al. (2020); Madaio et al. (2022); Ferrara (2023); Balayn et al. (2023)) shows integrating fairness into workflows is essential. Practitioners find value in embedding fairness into routine processes, making it fundamental to system design and evaluation rather than an afterthought. This practical necessity helps organizations address real-world constraints and competing priorities. Embedding fairness into routine processes enables proactive identification of bias and more transparent navigation of stakeholder tensions, especially when balancing business goals with marginalized users’ needs. For example, Madaio et al. (2022) describes fraud detection teams using open fairness discussions to collaboratively negotiate metrics, like with co-designed checklists (Madaio et al. (2020)). However, workflow tools alone are insufficient—resourcing and buy-in remain challenges. Embedding fairness in workflows enables organizations to catch and mitigate algorithmic harms before deployment, as Balayn et al. (2023) highlights. These interventions establish user trust and ensure systems serve their communities. Practitioners want practical, well-documented, and flexible fairness toolkits, with plain-language explanations and real-world examples, as in Lee & Singh (2021). Effective integration depends on meeting practitioners’ daily needs. Fairness should be integrated throughout the AI lifecycle, from data collection to deployment and monitoring. Practitioners need tools and processes that ensure fairness remains a consistent, practical part of building and maintaining AI systems.

Stakeholder Engagement¶

Achieving fairness in AI systems requires meaningful and systematic stakeholder engagement, yet current practices often fall short of this. Studies consistently demonstrate that practitioners struggle to identify and involve the full range of stakeholders, particularly those with lived experience or expertise on demographic groups most affected by AI decisions (Madaio et al. (2022)). This lack of engagement can result in blind spots—such as in the case of a healthcare AI system, where failing to consult gender identity experts led to biased treatment recommendations for non-binary patients. Evidence suggests that bridging these gaps is not just a matter of ethics but strongly shapes system outcomes and perceived fairness. This challenge is further complicated by the fact that fairness is not a singular concept—trade-offs between the interests of different stakeholder groups (such as consumers and content providers) are common and rarely confronted directly (Smith et al. (2023)). Fairness cannot be achieved through technical solutions alone; instead, it requires ongoing cross-functional collaboration among technical teams, policymakers, and impacted stakeholders. Without clear processes for defining stakeholder roles and goals, and mechanisms for sustained coordination, fairness initiatives tend to remain fragmented and ineffective. To move beyond superficial inclusion, researchers recommend structured frameworks that map out the knowledge, perspectives, and needs of diverse stakeholders throughout the lifecycle of AI systems (Suresh et al. (2021)). This includes distinguishing between different types of expertise (formal, instrumental, personal) and ensuring that those directly impacted by AI—such as patients, data subjects, or marginalized communities are meaningfully involved in design and evaluation. Such frameworks enable teams to clarify stakeholder goals, address power imbalances, and design AI systems that better reflect the values and concerns of those most affected. Engaging with diverse stakeholders, including those potentially impacted by the AI system, is crucial for identifying and addressing fairness concerns. Practitioners need guidance and support to engage stakeholders effectively.

Organizational Support and Incentives¶

Organizational support and incentives are essential for fostering fairness in AI development. Studies show that when fairness work isn’t recognized or rewarded, practitioners are less likely to prioritize it, feel less empowered to raise concerns, and are less able to drive systemic change (e.g., Holstein et al. (2019); Richardson & Gilbert (2021)). Cultivating an environment where fairness is valued and incentivized enables practitioners to address bias more effectively. To truly embed fairness, organizations must clearly define roles and create accountability, ensuring fairness is a shared priority. When fairness is upheld at all organizational levels, practitioners feel supported and can advance equitable outcomes. Treating fairness as a collective responsibility leads to lasting change.Practitioners often lack clear guidance on implementing fairness, which underscores the need for organizational investment in training and collaboration. Elevating fairness as a shared, fundamental responsibility empowers teams to address issues more effectively. Research highlights that both organizations and governments are responsible for fairness and transparency in AI, with accountability for outcomes extending to all involved (Ferrara (2023)). Ethical guidelines and regulatory frameworks support practitioners and clarify shared responsibility. Organizations must encourage open dialogue and create environments where perspectives on fairness are valued, making fairness a collective obligation (Madaio et al. (2020); Holstein et al. (2019)). To prioritize fairness, organizations should build infrastructure and policy safeguards for employees who advocate for fair AI practices. Embed fairness through systematic, explicit operational processes and protections throughout all organizational levels (Richardson & Gilbert (2021)). This benefits both internal culture and public trust. One key barrier is practitioners’ reluctance to collect demographic data due to privacy concerns. Organizations should implement clearly communicated, ethical processes for demographic data collection—ensuring transparency and robust privacy safeguards—while enabling necessary analysis to identify and mitigate bias (Deng et al. (2022)). To foster a culture of fairness, leaders must prioritize setting clear expectations and offering specific incentives for practitioners. Otherwise, organizations will continue to confront ongoing challenges in achieving fairness in AI.

Choice Of Data and Processing¶

Achieving fairness defined as equitable treatment and outcomes for individuals or groups—in AI models is consistently hampered by challenges in data selection, with practitioners across multiple studies identifying uncertainty about data type choice as a major barrier. Fairness challenges emerge throughout the AI lifecycle, from data preprocessing to evaluation. There is a widespread lack of clear, actionable guidance for practitioners on how to address them. Across the literature, evidence shows that data imbalance or poor selection is a recurring root cause of unfair or biased systems (Ferrara (2023); Ferrara et al. (2023)). To address this, both group and individual fairness frameworks are recommended. Group fairness approaches such as demographic parity, disparate mistreatment, and equal opportunity seek to ensure proportional treatment across groups. These can be operationalized through techniques such as resampling or adjustments at various modeling stages. Evidence indicates these strategies can correct for data that would otherwise reinforce existing biases. Individual fairness, defined as the principle that similar individuals should be treated similarly, conversely emphasizes equitable treatment at the level of specific individuals. It utilizes methods such as counterfactual and causal fairness (which evaluate decisions by comparing outcomes for individuals or the causes driving those outcomes) to reduce the risk of unfair discrimination. Yet, synthesis of the literature shows that both approaches come with trade-offs. Group fairness, which seeks equal outcomes for defined groups, may overlook individual disparities, while individual fairness often fails to address broader, systemic inequity. Practitioners also face persistent uncertainty in identifying relevant subgroups for fairness analysis, a task that is highly context-dependent (Holstein et al. (2019)). Notably, studies highlight that pre-deployment bias detection remains limited, as fairness concerns frequently surface only after deployment often in response to external complaints or negative publicity. Furthermore, existing detection tools are frequently inadequate for surfacing biases in underrepresented subgroups, and domain specific complexities such as the limitations of using broad categories like ethnicity or gender further complicate subgroup analysis. Data collection for fairness evaluation presents further obstacles. Balancing privacy with the need for demographic information is a key challenge (Madaio et al. (2022)). Multiple studies document how privacy requirements routinely limit the collection of demographic data essential for disaggregated evaluations. This leads to ad hoc methods such as side channel data collection, internal company datasets, or voluntary surveys. These workarounds introduce new challenges related to representativeness and reliability. This situation reinforces the need for domain tailored approaches and ongoing efforts to broaden understanding of demographics in fairness assessments.

Recommendations/ checklist of best practices¶

In this section we are going give recommendations on how to achieve fairness in AI and ML systems and also create a checklist that guides practitioners in choosing the right data, right tool, metric how to identify and mitigate biases in their models.

Recommendations¶

The recommendations provided in this review are critical for translating theoretical insights into actionable practices that foster fairness in AI systems. Building on the challenges and findings identified, these recommendations aim to guide organizations, practitioners, and policymakers in developing and deploying more equitable AI technologies. By addressing issues such as data bias, practitioner engagement, and toolkit usability, these suggestions offer a future to enhance accountability, inclusivity, and transparency. The practical implementation of these recommendations has the potential to mitigate algorithmic harms and build trust in AI systems across diverse applications.

Data Collection and Processing¶

To ensure fairness in AI systems, prioritizing representative datasets is important. This involves collecting data that accurately reflects diverse subpopulations, especially those historically underrepresented or marginalized. Emphasizing privacy preserving techniques is equally important, striking a balance between ethical data use and compliance with privacy regulations. By adopting these practices, organizations can address biases in training data and create systems that perform equitably across different demographic groups, thereby promoting inclusivity and reducing potential algorithmic harms. To make sure that data is well balanced to train a model, the data can be resampled, preprocessing or postprocessing.

Priority of Action¶

Practitioner Training and Awareness¶

Promoting practitioner training and awareness is essential for embedding fairness principles in AI development. Educational initiatives should focus on equipping practitioners with a deep understanding of fairness frameworks, ethical considerations, and their real world implications. Workshops, training modules, and ongoing professional development can help practitioners recognize biases, interpret fairness metrics accurately, and make informed decisions. By ensuring the training of practitioners, practitioners can come to understand the concept of fairness, define fairness based on their task , know how to implement fairness, and be able to take full responsibility for anything that goes wrong. By fostering awareness and building expertise, organizations can empower teams to prioritize fairness throughout the AI lifecycle, ensuring equitable and socially responsible outcomes.

Priority of Action¶

Tools Design and Usability¶

Fairness tools must be designed to support practitioners in understanding and mitigating biases without fostering over-reliance on the tools. These tools should have the following improvements :

1. Created in such a way that aligns with the existing workflow of the practitioners.

2. They should be easy to understand, have clear explanation of fairness metric and their applicability.

3. They should be able to work on different types of models, not just a one size fits all tool.

4. These tools should be created in a way that practitioners can give their feedbacks on them, communicate with fairness experts so they can ask for help were need be.

5. Tools should be able to detect and mitigate bias and unfairness throughout the life cycle of the model.

6. Tools should include mechanisms for testing tools across diverse contexts (real world scenarios and user settings) to ensure that they work effectively across different demographics, industries, cultural settings, or application domains.

Priority Actions¶

Stakeholder Engagement¶

Effective stakeholder engagement is pivotal for ensuring AI systems meet diverse societal needs. A co-design approach should be employed, involving input from a wide range of stakeholders, including marginalized communities, domain experts, and end-users. By fostering inclusive collaboration, organizations can identify and address potential biases, ensure representational fairness, and align AI systems with ethical and social priorities. Regular feedback loops and transparent communication can strengthen trust and improve the alignment of AI outputs with real-world requirements.

Priority Action¶

While these recommendations provide a roadmap for achieving fairness in AI systems, it is imperative for practitioners to recognize their critical role in this process. These guidelines are ineffective without their active engagement. Practitioners must not only implement these strategies but also continuously reflect on their practices, providing valuable feedback on fairness tools to both experts and organizations and adhere to organizational innovations designed to foster fairness. Their involvement is key to refining tools, addressing unforeseen challenges, and ensuring that fairness becomes an integral part of AI development and deployment.

Checklist of Best Practices¶

This section introduces a comprehensive checklist derived from an in-depth literature review of nine foundational research papers on AI fairness. The goal is to provide practitioners, researchers, and organizations with actionable guidelines to identify, evaluate, and implement best practices for fairness throughout the AI development lifecycle. By synthesizing challenges, tools, and methodologies discussed in these studies, the checklist offers a structured framework to address issues such as bias mitigation, stakeholder involvement, and fairness evaluations. It serves as a practical tool for fostering accountability and equity in AI systems.

First we provide a high level checklist and then a scoring rubric.

Pre-processing phase¶

1. Model Overview :

   • Clearly articulate the purpose, scope, and intended application of the AI model.

   • Specify the problem it aims to solve and the context in which it will operate.

2. Stakeholder Identification and Fairness Definition:

   • Identify a diverse set of stakeholders, including domain experts, affected communities, and interdisciplinary contributors.

   • Ensure these stakeholders reflect the demographic, cultural, and contextual diversity relevant to the task.

   • Collaboratively define fairness in the context of the specific AI model and its application.

   • Establish clear, task specific fairness goals that are measurable and aligned with stakeholder and organization priorities.

3. Data Collection and Preprocessing:

3.1. Assess the representativeness of datasets

• Are datasets reflective of the intended user population?

• Are marginalized subgroups sufficiently represented?

3.2. Evaluate privacy vs fairness trade-offs.

• Have privacy-preserving methods been employed without compromising demographic diversity?

3.3. Address demographic data gaps.

• Is there a strategy to identify and fill blind spots? strategies like

  • Assessing where demographic data is missing or underrepresented by comparing available data against relevant subpopulations for the AI task.

  • Using surveys, focus groups, or external data sources to gather more diverse datasets. Where privacy concerns arise, privacy-preserving methods like synthetic data generation or differential privacy can help.

4. Bias Detection and Mitigation:

   • Have fairness metrics been applied to specific subgroups?

   • Are bias mitigation techniques(Preprocessing,post-processing and Fair Representation Learning) Implemented?

   • Are mitigation strategies aligned with context-specific fairness goals?

   • Is there an ongoing feedback mechanism to detect new biases?

5. Fairness Metric Selection:

   • Have appropriate fairness metrics been chosen, considering the specific context and potential harms of the AI system?

   • Are tools used to complement domain expertise rather than replace critical judgment?

   • Are decisions about fairness criteria transparent and actionable?

Development Phase¶

1. Model Selection and Training:

   • Mitigation Techniques: Evaluate whether fairness mitigation techniques have been incorporated during model selection and training. Consider approaches like bias correction, fairness constraints, or adversarial de-biasing.

   • Transparency and Explainability: Assess if the model’s decision making process is transparent and explainable. Ensuring clarity in how decisions are made helps practitioners understand and identify potential biases, contributing to more accountable AI systems.

2. Evaluation and Testing:

   • Disaggregated Evaluation: Verify if the model’s performance has been evaluated across various demographic groups. This process helps identify any disparities in accuracy, ensuring that fairness is maintained across diverse populations.

   • Bias Auditing: Conduct regular bias audits to monitor the model’s fairness over time. These audits ensure the model continues to meet fairness objectives and adapt to evolving data or societal needs.

   Rubric for Evaluating Checklist¶

   Criterion	Excellent (4)	Good (3)	Needs Improvement (2)	Unsatisfactory (1)	Weight
   Model Overview	Clear purpose, scope, and context defined with specific objectives.	Well-defined, but could be more specific.	Vague or incomplete definition of purpose and scope.	No clear definition of purpose or scope.	10%
   Stakeholder Identification and Fairness Definition	Diverse stakeholders identified with specific fairness goals set.	Most stakeholders identified, goals mostly clear.	Limited stakeholder involvement and fairness goals.	No stakeholder involvement or unclear fairness goals.	15%
   Data Collection and Processing	Comprehensive, representative datasets with privacy considerations without compromising fairness.	Datasets generally representative, some gaps,privacy concerns considered but some trade-offs.	Some missing or unbalanced datasets that is significant gaps in representation of key subgroups and trade-offs not sufficiently addressed or balanced.	Data collection not reflective of population or privacy concerns ignored.	30%
   Bias Detection and Mitigation	Comprehensive bias mitigation strategies (Preprocessing, Post-processing, Fair Representation Learning) implemented.	Some bias mitigation strategies used with minor gaps.	Only basic bias mitigation approaches used.	No bias mitigation strategies used.	15%
   Fairness Metric Selection	Clear, context-specific metrics chosen with transparent, actionable decisions.	Good selection of metrics with reasonable transparency.	Metrics chosen without full alignment with context.	No clear fairness metrics or criteria selection.	10%
   Model Selection and Training	Fairness mitigation techniques incorporated from the start.	Some fairness techniques applied, but not comprehensive.	Techniques implemented too late in the process.	No fairness techniques applied during training.	10%
   Evaluation and Testing	Thorough disaggregated evaluation and regular bias auditing.	Adequate evaluation with some gaps in auditing.	Evaluation done for limited groups; auditing irregular.	No evaluation or auditing conducted.	10%

Limitations and Future Work.¶

While this literature review provides a comprehensive analysis of AI fairness practices across various domains, it is essential to acknowledge certain limitations that could impact the findings and recommendations:

1. This paper addresses the problem using a limited set of papers, which, while diverse, may not fully capture the range of perspectives and advancements in AI fairness. Emerging research or alternative viewpoints might be underrepresented. Therefore, future work will aim to expand the review by incorporating additional papers to provide a more comprehensive analysis.

2. The review does not delve deeply into the specific challenges and solutions for certain domains or use cases, such as the application of AI in human resource management, healthcare and other industries. Additional research may be needed to explore fairness in more specialized contexts and provide guidance tailored to the needs of practitioners in those fields.

3. Evaluate the impact of our checklist and rubric in the real world by conducting surveys with practitioners and stakeholders.

4. Develop our checklists and rubrics as accessible software tools that can be integrated directly into practitioners’ existing workflows (for example, inside Jupyter notebooks, VS Code, GitHub Actions, or ML pipelines, giving real-time guidance, warnings, or automated fairness checks as models are built and deployed.).

5. As AI technologies and societal expectations evolve, some insights or recommendations from the reviewed papers may become outdated, limiting their long-term applicability. Researchers may need to keep on updating the recommendations and checklists to make sure it meets with the current state of AI

Conclusions¶

This literature review examines key practices, challenges, and strategies for achieving fairness in AI systems, drawing insights from foundational research. It underscores the importance of integrating fairness considerations throughout the AI lifecycle, from data collection to deployment, emphasizing stakeholder engagement, bias mitigation, and the use of fairness-aware tools. While advancements in fairness research provide valuable frameworks and solutions, the effectiveness of these efforts emphasizes on practitioners’ understanding, continuous feedback, and commitment. Moving forward, fostering collaboration among technologists, ethicists, and policymakers will be essential to building AI systems that are equitable and trustworthy.