AI Audit Lesson Plan

AI Audit Lesson Plan [80 minute class period] Lesson Plan AI Audit

The purpose of this lesson plan is to equip students with the skills and background knowledge to perform an AI audit. It accomplishes this by walking through the process of an AI audit on a fictional AI. 

This lesson plan is informed by

“Algorithmic Accountability” by Nicholas Diakopoulos

and

“Auditing Algorithms” by Christian Sandvig et al.

Play Survival of the Best Fit [10 min] followed by discussion [5 min]

  • What did you notice in the game?  Did anything surprise you? What areas would you immediately want to investigate about this AI?
  • Is there anything from this game that you think is applicable to real-world AI? 

Connect to the real world. This is a fictional AI, but stuff like this really happens! These are just a few examples [5 min]

  • Amazon resume screening  
  • Predictive Policing 
  • ProPublica Risk Assessment 
  • Healthcare 
  • Facial Recognition 

We know about these AI fails because of journalistic AI audits. Given that companies don’t share their training data and algorithms, one of the most useful tools thus far is the AI audits. The goal of this lesson is to equip you with the skills to do it yourself. [5 min] 

How to perform an AI Audit [50 min]  

We’re going to practice an AI audit in three steps: 1) Identify an AI to audit, 2) Sample the AI, and 3) Find the story. We will run through these steps with the fictional AI. 

Identify an AI to Audit.

  • We’ve already done that in this case, but how can we tell that this is an interesting audit? (It has real-world consequences that are newsworthy and important, it hasn’t been audited before, etc.)
  • How would you go about finding an AI to audit in a real AI audit? (think about systems you interact with, gatekeeping metrics like credit scores, think about any time some complex aspect of human life is quantified, software that prioritizes, classifies, or filters). 
  • How would you tell an AI wouldn’t be interesting to audit? (While almost any AI could be potentially interesting, AI that has very low stakes is harder to make a case for, so are things that have been extensively studied. There is a lot of work on YouTube’s algorithm but little on TikTok’s.)

Sample the AI.

  • Imagine we’re external to the company in the game. This company is selling their hiring AI to your company. What information would you want before you would make the decision to purchase their AI service? 
    • Be sure to cover 1) the data you’d need to make sure it actually does the job they say it does—maybe the rates of people it recommended hiring getting let go within 2 years, 2) the data you’d need to tell if it is biased against protected groups—maybe the hiring rates of different demographics before and after a different company implemented the algorithm, and anything else students think of.
    • This is a kind of “blue sky” audit. You will most likely not have access to the information you’d need to truly audit the AI, but it’s an important step to clarify what your goals are and what you’d like to check for. It’s also a good indication of the kind of data AI companies should make public so that they can be audited. A blue sky exercise can also help students figure out alternate ways they could get at similar information. 
  • What makes sampling an AI hard in the real world?
    • Blackbox nature of AI. Internal system inaccessible—only inputs and outputs.
    • Additionally, many times you don’t even have access to the inputs, only outputs.
    • Many AIs are tailored to you. It’s hard to get a representative sample.
    • It’s difficult to show causation. Correlation is easier. 

Find the story.

  • What would your story be for our example AI?  
  • Prompts to help in coming up with a story: 
    • Has the algorithm made a bad decision or broken an expectation for how we think it should be operating? 
    • How does it affect individuals and or communities
    • Are there any false positives or false negatives? 
    • What is driving that break in expectations?
      • This last question is harder because it requires knowing more about the AI’s inner workings, but if you can dig into it, you really have a story.

Get into groups and search for AIs to audit. Come together and discuss what you found and ways to audit. [10 minutes] 

Questions and create individual plans for sampling an AI [10 minutes] 

Student AI audit examples available here.