Assessing ML Test Scores: Surveys, Interviews, and Documentation Analysis

Maria Vechtomova discusses the methodology employed to evaluate ML test scores. Maria explains that they utilized existing papers focused on organizational practices, but decided to delve deeper into product-level aspects. They developed a set of 70 questions, covering topics like version control, code tracking, model artifacts, and data used for deployments. The evaluation process involved conducting interviews, scheduling calls, and sharing a SharePoint document for participants to complete. Their approach combined surveys, interviews, and documentation analysis to comprehensively assess ML …

analysis code control data documentation interviews methodology practices product questions set surveys test topics tracking version control