Increasing measurement efficiency and validity of computer-based assessments using prior information and process data in the National Education Panel Study (NEPS)

Tobias Deribo, ZIB-associated Researcher DIP

The dissertation project investigates how prior information and process data from computer-based assessments (CBA) can be used to increase the validity of inferences based on test scores and the measurement efficiency of assessments in the National Education Panel Study (NEPS)➚. Thanks to the increased availability of CBA—and the corresponding process data—it seems possible to learn more about a test-taker than would be possible on the basis of their submitted (or not submitted) answers.

The topic is important, as each test session can represent a time burden as well as a psychological one for test-takers. Each further session can also generate economic costs for the test provider or administrator.

With regard to the first objective of increasing validity, it is investigated how process data can be used to reduce construct-irrelevant variance. The focus here is particularly on the detection and treatment of disengaged response behavior based on process data. In this context, especially the question of whether test-takers actually work on a task or simply try to guess an answer is investigated (e.g., Kroehne, Deribo, & Goldhammer, 2020). With regard to the second objective, the increase of measurement efficiency, a comparison of different test composition methods is planned. This comparison should allow for the investigation of whether or not and to what extent it is possible to reduce the test length, while maintaining measurement accuracy by including process information within the test composition.

The goal of the planned dissertation project is to show possible applications in which the additional information from CBA can lead to added value for scaling and test design. Therefore, the results of the dissertation project will be of interest for both measurement theorists and practitioners.

References:

Kroehne, U., Deribo, T., & Goldhammer, F. (2020). Rapid Guessing Rates Across Administration Mode and Test Setting. Psychological Test and Assessment Modeling, 62 (2), 147–177. [online available➚]

Betreuer: Prof. Dr. Frank Goldhammer➚