How to Make AI Audits Societal, Not Just Technical

Lessons from the Data Nutrition Project’s 2023 convening

By Kasia Chmielinski, Sarah Newman, and Matt Taylor

Increased reliance on AI and ML systems brings with it increased scrutiny into how they work. Yet many auditing proposals are largely technical, lacking social, qualitative, domain-specific context, and thus potentially scoring differently in technical reviews than these models will perform in the wild. There is growing awareness that algorithmic harms are sociotechnical in nature. In other words, their societal context (and actual versus …