AI Ethics & Responsible AI New
Bias in AI, fairness, transparency, regulatory landscape
What is AI bias and why does it matter?
AI bias occurs when an algorithm produces systematically prejudiced results due to flawed assumptions in the machine learning process or unrepresentative training data. It matters because biased AI can cause real-world harm—denying people loans, jobs, or healthcare—and reinforces existing social inequalities at scale. [Source: NIST]
How does training data cause AI bias?
Training data causes AI bias when historical datasets reflect past discrimination, lack demographic diversity, or contain labeling errors. A model trained on such data learns and perpetuates those patterns—for example, a hiring model trained on historically male-dominated résumé data will systematically rank women lower. [Source: ACM]
What is algorithmic fairness?
Algorithmic fairness is the principle that an AI system's decisions should not produce discriminatory outcomes across demographic groups such as race, gender, or age. There is no single agreed definition—researchers have identified over 20 distinct mathematical fairness criteria that can be mutually incompatible. [Source: ACM FAccT]
What are the main types of AI fairness metrics?
The three most commonly used AI fairness metrics are demographic parity (equal positive prediction rates across groups), equalized odds (equal true and false positive rates), and individual fairness (similar individuals receive similar predictions). Each captures a different aspect of fairness and can conflict with the others. [Source: NIST]
How do you detect bias in an AI model?
Detecting AI bias requires disaggregating model performance metrics—accuracy, false positive rate, false negative rate—across demographic subgroups, and running statistical tests for disparate impact. Tools such as IBM's AI Fairness 360 and Google's What-If Tool provide open-source frameworks for systematic bias auditing. [Source: IBM Research]
How can AI bias be mitigated or reduced?
AI bias mitigation operates at three stages: pre-processing (rebalancing or relabeling training data), in-processing (adding fairness constraints during model training), and post-processing (adjusting decision thresholds per group). No single technique eliminates bias entirely; NIST recommends combining technical fixes with organizational governance. [Source: NIST AI RMF]
What is AI transparency and why is it required?
AI transparency means that the inputs, logic, and outputs of an AI system are understandable and auditable by relevant stakeholders—developers, regulators, and affected individuals. It is required because opaque 'black-box' systems make it impossible to identify errors, bias, or misuse, and several regulations now mandate it. [Source: EU AI Act]
What is explainable AI (XAI)?
Explainable AI (XAI) refers to methods and techniques that make the outputs of AI models understandable to humans. DARPA defines it as AI that can explain its rationale, characterize its strengths and weaknesses, and convey an understanding of how it will behave in the future—enabling meaningful human oversight. [Source: DARPA]
What are LIME and SHAP in AI explainability?
LIME (Local Interpretable Model-agnostic Explanations) and SHAP (SHapley Additive exPlanations) are the two most widely used post-hoc explainability techniques. LIME approximates a complex model locally with a simpler one; SHAP uses game-theoretic Shapley values to assign each feature a contribution score to a specific prediction. [Source: NeurIPS / Proceedings of Machine Learning Research]
What does the EU AI Act require from AI developers?
The EU AI Act, which entered into force on 1 August 2024, classifies AI systems by risk level. High-risk systems—covering areas like hiring, credit, and biometric identification—must meet requirements for transparency, human oversight, accuracy, and robustness, with conformity assessments before market placement. Prohibited uses include real-time biometric surveillance. [Source: European Parliament]
What are the main AI regulations in the United States?
The U.S. lacks a single federal AI law. Key instruments include President Biden's 2023 Executive Order on AI (since partially revised under the Trump administration), the NIST AI Risk Management Framework, and sector-specific rules from the FTC, EEOC, and CFPB covering algorithmic discrimination in credit, employment, and consumer markets. [Source: NIST / White House]
What is the NIST AI Risk Management Framework?
The NIST AI Risk Management Framework (AI RMF 1.0), published in January 2023, is a voluntary framework that helps organizations design, develop, deploy, and evaluate AI systems to manage risk. It is structured around four core functions: Govern, Map, Measure, and Manage, and applies across sectors and AI use cases. [Source: NIST]
What is responsible AI?
Responsible AI is a governance framework and set of practices ensuring AI systems are developed and deployed in ways that are fair, transparent, accountable, safe, and aligned with human values. Microsoft, Google, and the OECD each publish principles, but core pillars consistently include fairness, reliability, privacy, and human oversight. [Source: OECD]
What are the OECD AI Principles?
The OECD AI Principles, adopted in May 2019 and updated in 2024, are the first intergovernmental standards on AI adopted by governments. The five principles are: inclusive growth and well-being; human-centred values; transparency and explainability; robustness and safety; and accountability. They have been endorsed by G20 nations. [Source: OECD]
What is AI governance and who is responsible for it?
AI governance is the set of policies, processes, and organizational structures that guide how AI systems are created, deployed, and monitored to ensure accountability and alignment with legal and ethical standards. Responsibility is shared: developers, deployers, governments, and affected individuals all have roles defined by frameworks such as the EU AI Act and NIST AI RMF. [Source: NIST]
What is an AI impact assessment?
An AI impact assessment (AIA) is a structured process for identifying and evaluating the potential harms an AI system could cause to individuals, groups, or society before and during deployment. The EU AI Act mandates fundamental rights impact assessments for high-risk AI systems, and Canada's Directive on Automated Decision-Making requires similar assessments for government use. [Source: EU / Government of Canada]
Why does facial recognition AI raise concerns about bias?
Multiple peer-reviewed studies, including NIST's FRVT evaluation, have found that commercial facial recognition algorithms exhibit significantly higher false positive and false negative rates for darker-skinned individuals and women. In real-world use, these errors have led to wrongful arrests, making facial recognition one of the most documented sources of consequential AI bias. [Source: NIST]
Do people have a legal right to an explanation of AI decisions?
Under the EU's General Data Protection Regulation (GDPR), individuals have the right not to be subject to solely automated decisions with significant effects, and can request meaningful information about the logic involved. The EU AI Act reinforces this for high-risk systems. No equivalent federal right exists in the U.S., though some state laws apply. [Source: European Commission / GDPR]
What is AI hallucination and why is it an ethical concern?
AI hallucination refers to large language models generating confident but factually incorrect or fabricated outputs. It is an ethical concern because hallucinations can cause direct harm when AI is used in high-stakes domains—medical advice, legal research, journalism—and raise accountability questions about who is responsible when false AI-generated information causes damage. [Source: NIST]
What AI ethics principles do major technology companies publish?
Major tech companies have published AI ethics frameworks: Google's Responsible AI Practices identify seven principles including safety and fairness; Microsoft's Responsible AI Standard covers six principles; IBM's AI Ethics Board oversees five pillars including explainability. While influential, these are self-regulatory—critics note they lack independent enforcement mechanisms. [Source: IEEE / Google / Microsoft]