Human-in-the-Loop AI for Large-Scale Institutional Assessment Review: An Assessment Practice Case Study

Katherine Perez, Jennifer Doherty-Restrepo, & Eilyn Sanabria   |    Volume 21 Issue 3  |   

Higher education assessment offices are increasingly expected to provide timely, consistent, and actionable feedback across large volumes of reports. This descriptive assessment-practice case study documents Florida International University’s human-in-the-loop, custom GPT-supported process for first-tier assessment report review. The workflow combines rubric formalization, structured prompting, historical-report calibration, standardized evidence tables, and required human verification. During 2023–2025, two reviewers processed 1,226 reports from October to November; prior human-only first-tier review typically extended through February. Reviewers estimated that a typical review decreased from approximately 30–60 minutes to approximately 15 minutes, although time-on-task was not systematically measured. Accordingly, this case presents early feasibility and implementation observations rather than validated evidence of reliability, effectiveness, or usability. It offers practical guidance for assessment offices seeking to use generative AI to increase review capacity while preserving rubric fidelity, governance, and professional judgment.

Click here to download the full article

« Back to Archive