Does guessing matter? Differences between ability estimates from 2PL and 3PL IRT models in case of guessing
Does guessing matter? Differences between ability estimates from 2PL and 3PL IRT models in case of guessing

Summary/Abstract: Modern approaches to measuring cognitive ability and testing knowledge frequently use multiple-choice items. These can be simply and rapidly scored without problems associated with rater subjectivity. Nevertheless, multiple-choice tests are often criticized owing to their vulnerability to guessing. In this paper the impact of guessing was examined using simulation. Ability estimates were obtained from the two IRT models commonly used for binary-scored items: the two-parameter logistic model and the three-parameter logistic model. The latter approach explicitly models guessing, whilst the former does not. Rather counter-intuitively, little difference was identified for point estimates of ability from the 2PLM and 3PLM. Nevertheless, it should be noted that difficulty and discrimination parameters are severely downwardly biased if a 2PLM is used to calibrate data generated by processes involving guessing. Estimated standard errors for ability estimates also differ considerably between these models.

• Details
• Contents