Traditional and Innovative Standard-setting Methods for Written, Oral, and Longitudinal assessments

As acknowledged by the Standards for Educational and Psychological Testing (AERA/APA/NCME, 2014), standard setting, a process of establishing a performance standard on an examination/assessment, embodies value judgments as well as technical and empirical considerations. In the high-stakes credentialing and certification context, standard setting is essentially policy-making that defines whether individuals possess the knowledge, skills, and abilities for safe and effective practice. While many factors are taken into the consideration, standard setting is often informed by a recommendation from a panel of experts.

This session will focus on psychometric methods of working with an expert panel and examination/assessment materials to come up with a standard recommendation, including methods for traditional, point-in-time multiple-choice examinations (including the Angoff and bookmark), methods for oral examinations or other performance assessments (including the fair average, synthetic candidate, and predicted average approaches based on the Many-facet Rasch model) and two innovative methods for longitudinal assessments (measurement decision theory and a revised Hofstee method).

In addition to the detailed description of the standard-recommending procedures, presenters will share experiences and empirical consideration of the pros and cons of each method. The session will appeal to psychometricians, test developers, and subject matter experts (SMEs) that participate in standard-setting exercises.

Thank you Excelsoft for sponsoring the Test Development and Administration track.