Advancements in Job Analysis and Content Outline Development

Job analysis is one of the most foundational and important elements of any certification program. This session highlights recent research and methodological changes that two organizations have implemented to improve their respective job analysis processes.

Alpine Testing Solutions worked with a large-scale licensure program to update their analysis of practice, with a primary goal of maximizing the breadth of stakeholder input. Learn about the study, which consisted of multiple phases and included both qualitative and quantitative methods. Innovative design elements including reviews of position descriptions, electronic bulletin boards, ethnographies, web-based interviews, and social intercepts were combined with traditional elements like large-scale surveys to obtain supporting data from more than 24,000 stakeholders.

The American Board of Pediatrics (ABP) conducted research and implemented several changes that were designed to improve process efficiency and the utility of resulting content outlines. Hear the research results supporting the use of a single “relevance” rating scale on job analysis surveys in lieu of commonly used “criticality” and “frequency” scales. Results from an investigation into the relationship between content outline relevance ratings and item-level relevance ratings will be shared, and which may be helpful for cross-validating job analysis results and improving future item development efforts. Finally, advancements, including a set of recently developed content outline guidelines aimed at maximizing the clinical relevance of exam items, will be shared.

Attendees will have the opportunity to ask questions and hear lessons learned and anticipated future directions.

Thank you Excelsoft for sponsoring the Test Development and Administration track.