Skip to main content
2023 I.C.E. Exchange Main banner
Back

Test Development and Administration

Not the Same Ol’ Certification Exam: Applying Innovations to the Development of Situational Judgment Tests for the Coaching Profession

Wednesday, October 25, 2023
1:15 PM – 2:15 PM MT
Location: Colorado Hall D

    Main Contact/Presenter(s)

  • Andrea Sinclair, PhD

    Manager, Validity Investigations for Education and the Workplace
    Human Resources Research Organization, United States

    • Presenter(s)

  • Genevieve Ainslie, PhD

    Senior Scientist
    Human Resources Research Organization, United States

  • Carrie Abner

    Vice President, Credentials and Standards
    International Coaching Federation, United States

This session brings together test development experts and their partner organization, faced with updating an outdated and overexposed multiple-choice, global certification of exam.

Presenters will take attendees on a comprehensive journey of conducting a practice analysis of the coaching profession, revising a competency model, and developing and launching a new certification exam—a situational judgment test (SJT)—for the complex profession of coaching. The presenters will discuss how they applied innovative approaches to established methodologies, including the critical incident technique and the scoring of an SJT.

Hear results from field test analyses, and unanticipated challenges and decisions implemented to address these challenges.

Thank you Excelsoft for sponsoring the Test Development and Administration track.

Learning Objectives:

  • Incorporate international subject matter experts intentionally and strategically in all phases of test development (e.g., critical incident development, practice analysis, item review, bias and sensitivity review) to ensure global representation on an international credentialing exam.
  • Apply an innovative twist to the critical incident technique to yield rich, nuanced content for developing an authentic, scenario-based Situational Judgment Test (SJT) for credentialing in a complex profession.
  • Understand how to develop an innovative SJT scoring model to elicit more information with fewer test items by laying the foundation to successfully use a “pick best” and “pick worst” scoring model.
  • List strategies for overcoming challenges associated with transitioning to a new exam, including strategies for recruiting subject matter experts, incentivizing participation in the field test, and managing communications during the roll out of a new certification exam.