Skip to main content
2023 I.C.E. Exchange Main banner
Back

Test Development and Administration

A Look Under the Hood: Psychometric Methods Used in Longitudinal Assessments for Maintenance of Certification

Thursday, October 26, 2023
10:30 AM – 11:30 AM MT
Location: Colorado Hall E
Join us as we “pop the hood” on psychometric methods applied to Maintenance of Certification exams!

The American Board of Medical Specialties (ABMS) announced revised “Standards for Continuing Certification” which will be implemented in 2024. To meet these new standards, member boards have designed unique longitudinal assessments. This panel of four psychometricians (representing four member boards) will look at their revised Maintenance of Certification longitudinal examinations through a psychometric lens. Each participant will answer questions about how they supported their Board in evaluating and scoring their assessments.

The panel will discuss standard setting methodologies, diplomate feedback, data forensic measures to detect collusion, reliability indices, participation patterns, and lessons learned as they aim to balance the coinciding formative and summative components of these assessments. The panel will also discuss approaches for measuring item performance when multiple attempts are allowed.

Thank you Excelsoft for sponsoring the Test Development and Administration track.

Learning Objectives:

  • Upon completion, participants will be able to understand important psychometric principles for evaluating and scoring longitudinal assessments.
  • Upon completion, participants will be able to describe various ways that pass points are determined for longitudinal assessments.
  • Upon completion, participants will be able to describe how psychometricians have handled the complications of longitudinal assessments that measure both core specialty knowledge and subspecialty knowledge.