Syllabus

Session 1: What went wrong with this case? - Algorithmic accountability in practice

Mandatory readings

• The case study.
• Ada Lovelace Institute, AI Now Institute and Open Government Partnership. (2021). Executive Summary. Algorithmic Accountability for the Public Sector..
• Green, B. Z. (2019). Chapter 6: The Innovative City: The Relationship between Technical and Nontechnical Change in City Government. In The Smart Enough City: Putting Technology in Its Place to Reclaim Our Urban Future. MIT Press.
• Hao, K. (2020). The UK exam debacle reminds us that algorithms can’t fix broken systems. MIT Technology Review.
• Solomon, D. and Holmes, O. (2026). AI cheating, leaked papers and marking errors: how exam protests went global. The Guardian.
• Toh, A. (2014). The Algorithms Too Few People Are Talking About. Human Rights Watch.

Session 2: Avoiding technical pitfalls - errors, choices, biases, discrimination, (injustice?)

Mandatory readings

Three case studies on fraud detection in social services:

• The Childcare Benefits scandal (The Netherlands): Adelmant, V., van Veen, C. (2022). Hollow rights victories? Dutch struggles against digital injustice. Open Global Rights.
• The Robodebt scandal (Australia): Karp, P. and Henrique-Gomes, L. (2023). Explainer: What is robodebt? Six things to watch for in the royal commission’s final report today. The Guardian.
• Amsterdam’s “fair risk score” expriment: Guo, E., Geiger, G., Braun, J.-C. (2025). Inside Amsterdam’s high-stakes experiment to create fair welfare AI. MIT Technology Review.

Readings:

• Leufer, D. (2020). Myth: AI can be objective or unbiased. AI Myths.
• O’Neil, C. (2016). Introduction and Chapter 1: “Bomb parts: What is a model?”. In Weapons of Math Destruction: How Big Data Increases Inequality and Threatens Democracy. Crown Publishing Group.
• Yong, E. (2018). A Popular Algorithm Is No Better at Predicting Crimes Than Random People. The Atlantic.

Optional readings

• Bennett, S. H. (2020, 20 August). On A-Levels, Ofqual and Algorithms. Sophie Bennett’s blog.
• Corbett-Davies, S., Pierson, E., Feller, A., Goel, S. (2016, October 17). A computer program used for bail and sentencing decisions was labeled biased against blacks. It’s actually not that clear. The Washington Post.
• Dressel, J. and Farid, H. (2018). The accuracy, fairness, and limits of predicting recidivism. Science Advances.
• D’Ignazio, C. and Klein, L. (2020). 2. Collect, Analyze, Imagine, Teach. In Data Feminism. MIT Press.
• Gosciak, J., Boyce, L., Wang, A., and Koenecke, A. (2026). Scrutinizing Index-Based Risk Assessments: A Case Study in NYC Decision-making for Heat Emergency Management. Proceedings of the 2026 ACM Conference on Fairness, Accountability, and Transparency. Association for Computing Machinery, New York, NY, USA, 6659–6701.
• Narayan, A. (2018). Tutorial: 21 Definitions of Fairness and their politics. Proceedings of the 2018 ACM Conference on Fairness, Accountability, and Transparency.
• Reply All Podcast. (2018). Episodes 127 and 128: The Crime Machine. Gimlet Media. - For an example of how metrics can impact policies.
• Suresh, H. and Guttag, J. (2020). A Framework for Understanding Sources of Harm throughout the Machine Learning Life Cycle. Proceedings of the 2020 ACM Conference on Fairness, Accountability, and Transparency.
• Wachter, S., Mittelstadt, B., Russell, C. (2021). Bias Preservation in Machine Learning: The Legality of Fairness Metrics Under EU Non-Discrimination Law. West Virginia Law Review, Vol. 123, No 3.
• Wang, A., Kapoor, S., Barocas, S., Narayanan, A. (2023). Against Predictive Optimization: On the Legitimacy of Decision-Making Algorithms that Optimize Predictive Accuracy.
• Ziosi, M. and Pruss, D. (2024). Evidence of What, for Whom? The Socially Contested Role of Algorithmic Bias in a Predictive Policing Tool. FAccT ‘24: Proceedings of the 2024 ACM Conference on Fairness, Accountability, and Transparency.
• Documentation on the A-Level algorithm: Ofqual (2020). Executive summary, Student-level equalities analyses for GCSE and A level, Summer 2020. pp. 5-8; Ofqual. (2020, 15 April). Equality Impact Assessment.

Policies, Tools, Standards, and Good Practices

Concrete resources

• ACPR - Banque de France’s resource on algorithmic equity in the financial sector (in French). The City of Amsterdam’s Fairness Handbook, written for its public servants.
• The EU General-Purpose AI (GPAI) Code of Practice’s Model Documentation Form for general-purpose AI models (scroll down at the bottom of the page).
• The “tool specification” section of the United Kingdom’s Algorithmic Transparency Recording Standard.
• The European Data Protection Board’s guide on AI-Complex Algorithms and effective Data Protection Supervision: Bias evaluation (written by Dr. Kris Shrishak).
• The European Data Protection Board’s Checklist for AI Auditing (written by Dr. Gemma Galdon Clavell).
• Balayn, A., Gürses, S. (2021). Beyond Debiasing: Regulating AI and its inequalities. EDRi.
• Mittelstadt, B. (2024). How to use the Artificial Intelligence Act to investigate AI bias and discrimination: A guide for equality bodies. Equinet, European Network of Equality Bodies.

Standards and Laws

• International standards: IEEE Standard for Algorithmic Bias Considerations (7003-2024), ISO/IEC TR 24027:2021 - Bias in AI systems and AI aided decision making. - Note: you have to buy those, but this is so you have an idea of the current state of standards.
• NIST’s documentation on bias.
• Article 10 of the EU AI Act.

Session 3: Assessing the impacts

Mandatory readings

Three case studies on access to social benefits:

• Tafakul (Jordan) - see the summary in Toh, A. (2023). Automated Neglect: How The World Bank’s Push to Allocate Cash Assistance Using Algorithms Threatens Rights. Human Rights Watch.
• Nutrition welfare programs (India) - see Rizwan, H. India’s Facial Recognition Drive On Hungry Children Is Erasing Them and Tapasya, Sambhav, K., Joshi, D. (2024). How an algorithm denied food to thousands of poor in India’s Telangana. Al Jazeera.
• SHA’s means testing tool (Kenya) - see Mukami, P., Kirigia, J., Geiger, G., Statius, T., Lepapa, N. (2026). Hiding Behind AI: How SHA Was Used to Load Health System Costs Onto Poorest. Africa Uncensored.

Readings:

• Ada Lovelace Institute. (2020). Examining the Black Box: Tools for assessing algorithmic systems.
• Costanza-Chock, S., Raji, I. D., Buolamwini, J. (2022). Who Audits the Auditors? Recommendations from a field scan of the algorithmic auditing ecosystem. FAccT ‘22: Proceedings of the 2022 ACM Conference on Fairness, Accountability, and Transparency.
• Groves, L., Metcalf, J., Kennedy, A., Vecchione, B., Strait, A. (2024). Auditing work: Exploring the New York City algorithmic bias audit regime. In Proceedings of the Association for Computing Machinery. Association for Computing Machinery.
• Riley, S. (2024). Overriding (in)justice: Pretrial risk assessment administration on the frontlines. FAccT ‘24: Proceedings of the 2024 ACM Conference on Fairness, Accountability, and Transparency.

Optional readings

• Ada Lovelace Institute. (2025). Going Pro? Considerations for the emerging field of AI assurance. - For an overview of the current state of the field of “AI assurance”.
• Brandusescu, A., Sieber, R. E. (2025). Design versus reality: assessing the results and compliance of algorithmic impact assessments. Digital Society.
• Braun, J.-C., Constantaras, E., Haung, H., Geiger, G., Mehrotra, D., Howden, D. (2023). Suspicion Machines Methodology: A detailed explainer on what we did and how we did it. Lighthouse Reports. - For a methodology deep-dive on how data journalists audited a risk-assessment algorithm for bias (here, in the Dutch context).
• Constantaras, E., Geiger, G., Braun, J.-C., Mehrotra, D., Aung, H. (2023). Inside the Suspicion Machine. Wired.
• Elish, M. C. (2020). Sepsis Watch in Practice: The labor of disruption and repair in healthcare. Data & Society: Points. - For an account of why assessments are in context are important.
• Gender Shades. How well do IBM, Microsoft, and Face++ AI services guess the gender of a face?. - For an example of one the first third-party technical audits.
• Marda, V., and Narayan, S. (2020). Data in New Delhi’s Predictive Policing System. Proceedings of the 2020 ACM Conference on Fairness, Accountability, and Transparency. - For an example of what it is possible to do even without access to algorithms (here, in the Indian context).
• Varon, J. and Peña, P. (2022). Not My A.I.: Towards Critical Feminist Frameworks To Resist Oppressive A.I. Systems. Carr Center for Human Rights Policy, Harvard Kennedy School, Harvard University. - For a non-institutional vision of impacts.

Policies, Tools, Standards, and Good Practices

Examples of impact assessments

• UNESCO’s Ethical Impact Assessment.
• Canada’s Algorithmic Impact Assessment. See for instance the impact assessment for the “advanced analytics triage of overseas temporary resident visa applications”.
• New Zealand’s Algorithmic Charter (for context, see this presentation page).
• Australia’s National Framework for the Assurance of Artificial Intelligence in Government.
• The Catalonia Region’s Fundamental Rights Impact Assessment, with examples.

On environmental impacts

• Sustainable AI Coalition (2025). Standardization for AI Environmental Sustainability: Towards a coordinated global approach. - For an example of (institutional)

Examples of institutional audits

• Dutch Data Protection Authority - Department for the Coordination of Algorithmic Oversight (DCA). (2024). AI & Algorithmic Risks Report Netherlands - for an example of how national authorities can practice “algorithmic oversight”.
• Supreme Audit Institutions of Finland, Germany, the Netherlands, Norway and the UK, Auditing machine learning algorithms: A white paper for public auditors (2023) - for an example of how Courts of Audits can tackle the issue.

Examples of explainers/primers to familiarize public servants with issues and impacts

• Chen, B. (2025). Policy Brief: Dispelling Myths of AI and Efficiency. Data & Society. - for an example of a policy brief, US-oriented.
• Dent, A. (2024). Automating public services: a careful approach. Promising Trouble. - for another example of a policy brief, UK-oriented. coordination efforts around environmental impact assessments of AI.

An example of impact assessment in context

• Ada Lovelace Institute. (2024). Critical Analytics? Learning from the early adoption of data analytics for local authority service delivery. - For an example of second-party auditing in context.

An example in the law

• Article 27 of the EU AI Act and CEDPO. (2025). Fundamental Rights Impact Assessments: What are they? How do they work?. CEDPO AI and Data Working Group Micro-Insights Series.

Session 4: Building Accountability through “AI governance” (transparency, appeals, oversight)

Mandatory readings

Case studies on law enforcement and fight against domestic violence:

• VioGén (Spain). - see Satariano, A. and Toll Pifarré, R. (2025). Spain Overhauls Domestic Violence System after Criticism. The New York Times.
• Riscanvi (Catalonia) - see See Arandia, P., Ley, M., Sisqués, S., Martín., Ortega, M., Mateo, M., Luengo, J. (2024). Riscanvi: Un algoritmo define el futuro de los presos en Cataluña: ahora sabemo cómo funciona.
• Alerta Niñez (Chile) - see Risk Scoring Children in Chile, a recap by Paz Peña for the Center for Human Rights and Global Justice (NYU Law).

Readings:

• Ananny M, Crawford K. Seeing without knowing: Limitations of the transparency ideal and its application to algorithmic accountability. New Media & Society.
• Green, B., Kak, A. (2021). The False Comfort of Human Oversight as an Antidote to A.I. Harm. Slate.
• Jansen, F., Cath, C. (2021). Just Do It: on the limits of governance through AI registers. In AI Snake Oil, Pseudoscience and Hype, edited by Frederike Kaltheuner. Meat Space Press.
• Kolkman, D. (2020). F**ck the algorithm? What the world can learn from the UK A-level grading algorithm fiasco. LSE Impact Blog.
• Rodelli, C., Chander, S. (2025). One Year On, EU AI Act Collides with New Political Reality. Tech Policy Press.

Optional readings

• Défenseur des droits. (2024). Algorithms, AI systems and public services: what rights do users have? Critical considerations and recommendations.
• Feathers, T. (2023). It takes a small miracle to learn basic facts about government algorithms. The Markup.
• Parmar, T. (2025). Government Documents Show Police Disabling AI Oversight Tools. MotherJones.
• Pénicaud, S. (2025). Making Algorithm Registers Work for Meaningful Transparency. IA Ciudadana.
• Pi, Y., Li, W., and Singh, J. (2026). Understanding the Role of Algorithm Registers in AI Governance Through Comparative Analysis of China and the UK. In Proceedings of the 2026 ACM Conference on Fairness, Accountability, and Transparency (FAccT ‘26). Association for Computing Machinery, New York, NY, USA, 3367–3391.
• Wright, L., Muenster, R. M., Vecchione, B., Qu, T., Cai, P. (S.), Smith, A., Comm 2450 Student Investigators, Metcalf, J., & Matias, J. N. (2024). Null compliance: NYC Local Law 144 and the challenges of algorithm accountability. FAccT ‘24: Proceedings of the 2024 ACM Conference on Fairness, Accountability, and Transparency.
• Yew, R., Marino, B., Venkatasubramanian, S. (2025). Red Teaming AI Policy: A Taxonomy of Avoision and the EU AI Act. FAccT ‘25: Proceedings of the 2025 ACM Conference on Fairness, Accountability, and Transparency.

Policies, Tools, Standards, and Good Practices

Registers

• The Algorithm Register of the Dutch government.
• Chile’s register.
• The UK’s Algorithmic Transparency Recording Standard Hub.
• You can also look through the government AI registers listed in the Civic Tech Field Guide.

Redress

There are few examples of good appeal and redress. This report by Doteveryone, although focused on online services, is a good source of inspiration, especially in its recommendations.

• Massey, J., Pope, R. Kitcher, H., Miller, C. (2020). Better Redress for the Digital Age. Doteveryone.

Session 5: Leveraging participation and procurement

Mandatory readings

Case studies on AI and algorithms in healthcare:

• Robinson, D. G. (2022). The Kidney Transplant Algorithm’s Surprising Lessons for Ethical AI. Slate.
• Bernardo, Á., Álvarez del Vayo, M., Torrecillas, C., Maqueda, A., Laursen, L. (2025). Mole or cancer? The algorithm that gets one in three melanomas wrong and erases patients with dark skin. Civio.

Readings:

• Attard-Frost, B. (2023). AI Countergovernance. Midnight Sun.
• Costanza-Chock, S. (2020). Design Practices: “Nothing about Us without Us.”. In Design Justice.
• Hu, W. and Singh, R. (2024). Enrolling Citizens: A Primer on Archetypes of Democratic Engagement with AI. Data & Society.
• Kuehnert, B, Johnson, N., Dotan, R., and Heidari, H. (2026). Disclosure or Marketing? Analyzing the Efficacy of Vendor Self-reports for Vetting Public-sector AI. In Proceedings of the 2026 ACM Conference on Fairness, Accountability, and Transparency (FAccT ‘26). Association for Computing Machinery, New York, NY, USA, 687–713.
• Robinson, D. G. (2022). “Chapter 2: Democracy on the Drawing Board” and Conclusion. Voices in the Code. Russell Sage Foundation.
• Sloane, M., Moss, E., Awomolo, O., Forlano, L. (2022). Participation is not a Design Fix for Machine Learning. EAAMO ‘22: Proceedings of the 2nd ACM Conference on Equity and Access in Algorithms, Mechanisms, and Optimization.

Optional readings

• Carollo, M., Tanen, B. (2023). How a Group of Health Executives Transformed the Liver Transplant System. The Markup.
• Cardullo, P., Kitchin, Rob. (2019). Being a ‘citizen’ in the smart city: up and down the scaffold of smart citizen participation in Dublin, Ireland. GeoJournal.
• Johnson, N., Silva, E., Leon, H., Eslami, M., Schwanke, B., Dotan, R., Heidari, H. (2025). Legacy Procurement Practices Shape How U.S. Cities Govern AI: Understanding Government Employees’ Practices, Challenges, and Needs. FAccT ‘25: Proceedings of the 2025 ACM Conference on Fairness, Accountability, and Transparency.
• Office for Statistics Regulation Authority. (2021). Ensuring statistical models command public confidence: Learning lessons from the approach to developing models for awarding grades in the UK in 2020, Executive summary.
• Ofqual. (2020). Analysis of Consulation Responses: Exceptional arrangements for exam grading and assessment in 2020.
• Wylie, B. (2018, 13 August). Searching for the Smart City’s Democratic Future. Centre for International Governance Innovation.
• Young, M. (2025). Gear Shift: Driving Change in Public Sector Technology through Community Input.
• Robin Pocornie’s case against Proctorio (The Netherlands) - see the Racism and Technology Center’s dossier on the case.

Policies, Tools, Standards, & Good Practices

Literacy

• AI and discrimination course in France, by the Council of Europe and the French Equality Body (Défenseur des Droits).
• UNESCO’s Global AI Training.
• Data and Society’s new AI Civics program.

Frameworks

• ECNL’s Framework for Meaningful Engagement: Human Rights Impact Assessments of AI.
• Connected by Data’s Toolkit: Worker voice in public sector procurement of digital and AI systems in Wales. - For an example of worker involvement.
• Data Justice Lab. (2021). Advancing civic participation in algorithmic decision-making: A guidebook for the public sector.
• Gilman, M. (2023). Democratizing AI: Principles for Meaningful Public Participation. Data & Society.

Concrete Examples

• Connected by Data’s People Panel on AI. - For a concrete example of participatory general principles.
• Long Beach’s LB Co-Lab program. - For a concrete example of permanent participation.
• Ofqual. (2020). Analysis of Consultation Responses: Exceptional arrangements for exam grading and assessment in 2020. - As a potential counterexample.

Procurement practices

• ChileCompra’s Standard Bidding Terms for algorithms and artificial intelligence with ethical requirements.
• EU model contractual AI clauses to pilot in procurements of AI.
• Pakzad, R., and Conti-Cook, C. (2025). Key Considerations When Procuring AI in the Public Sector. Taraaz & The Collaborative Research Center for Resilience (CRCR).
• The resources of the GovAI Coalition for local governments (US context).
• UK’s Guidelines for AI Procurement. (2021)
• IEEE Standard for the Procurement of Artificial Intelligence and Automated Decision Systems.

Session 6: Taking down a system and managing the aftermath - Conclusion

Mandatory readings

Case studies about generative AI in the public sector:

• Reporting by 404 Media on a judge in Indiana (USA) using generative AI in court transcripts.
• Research on AI detectors being biased against non-native English writers.
• Reporting by The Guardian on the UK Home Office using generative AI to wrongly reject an asylum’s claim
• Bruff, O. and Groves, L. (2026). Scribe and Prejudice? Exploring the use of AI transcription tools in social care. Ada Lovelace Institute.

Readings:

• Ehsan, U., Singh, R., Metcalf, J., & Riedl, M. (2022). The algorithmic imprint. FAccT ‘22: Proceedings of the 2022 ACM Conference on Fairness, Accountability, and Transparency.
• Foxglove. (2020, 17 August). We put a stop to the A Level grading algorithm!.
• Leufer, D. (2020). Myth: AI has agency: headline rephraser tool. AI Myths.
• Poole, S. (2020, September 3). Steven Poole’s word of the day: ‘Mutant algorithm’: boring B-movie or another excuse from Boris Johnson?. The Guardian.
• Redden, J. (2022). Government’s use of automated decision-making systems reflects systemic issues of injustice and inequality. The Conversation.

Optional readings

• Green, B. Z. (2019). Chapter 2: The Livable City: The Limits and Dangers of New Technology. In The Smart Enough City: Putting Technology in Its Place to Reclaim Our Urban Future. MIT Press.
• Johnson, N., Moharana, S., Harrington, C., Andalibi, N., Heidari, H., Eslami, M. (2025). The Fall of an Algorithm: Characterizing the Dynamics Toward Abandonment. FAccT ‘25: Proceedings of the 2025 ACM Conference on Fairness, Accountability, and Transparency.
• Lulamae, J. (2022). People are still angry about the UK’s 2020 grading algorithm experiment. AlgorithmWatch’s Automated Society.
• Ofqual. (2021). Decisions on how GCSE, AS and A- level grades will be determined in summer 2021.

Examples of resistance

France

• Amnesty International. (2024). France: CNAF State Council Complaint. (and an article covering it in Wired: Meaker, M. (2024). Algorithms Policed Welfare Systems for Years. Now They’re Under Fire for Bias).

The Netherlands

• Vervloesem, K. (2020, April 6). How Dutch activists got an invasive fraud detection algorithm banned. AlgorithmWatch.
• Adelmant, V., van Veen, C. (2022). Hollow rights victories? Dutch struggles against digital injustice. Open Global Rights.

The US

• The Stop LAPD Spying Coalition.
• Surveillance Resistance Lab. MyCity, INC. A Case Against “CompStat Urbanism”.

The UK

• Public Law Project’s Tracking Automated Government register.

Global, not limited to public sector: The AI Resist List.