Warning: Graphic content. Test developer discretion advised! Extending readability analyses to item exhibits.

During the development of most exams, test developers will caution against writing items that are “too long” or “too wordy,” but this impulse toward discretion is not commonly extended to the content that a candidate must comprehend in an item graphic or exhibit. Also uncommon are evaluations of whether the amount of information in an item – whether it’s text- or graphic-based – presents an unnecessary challenge to the candidate. The evaluations are out there, though, and they come in the form of readability formulae.

The concept of readability – measuring the ease of reading a sample of text – has been around for decades. So, too, have readability formulae that align text with grade level reading comprehension. But what reading level is fair for a credentialing test? And what about tests that include a significant number of exhibits, including not only images but graphs and tables?

In this session, practitioners will demonstrate how to apply traditional readability metrics to selected-response item text and how to evaluate the results against professional expectations. Examples will be provided from two professional fields: architecture and credentialing professionals. The presenters will also share a novel approach to evaluating the “readability” of item exhibits and graphic content. To close the session, the presenters will offer reflections, and solicit questions and discussion from the audience about how test developers can use readability formulae to understand how much discretion should be advised regarding the information processing demands of test items.