National add-on study on mode effects
in PISA 2018
PISA 2015 included an important change for the test students: the switch from paper- to computer-based testing. It took advantage of the far greater possibilities for designing tests and exercises unlocked by computers. However, this also raised many questions on the comparability of the two test forms, which will be scrutinised more closely as part of a national add-on study for PISA 2018.
On the one hand, the project will test whether response behaviour differs between processing exercises on paper and using a computer. In some cases, the test exercises for the computer differ substantially in design from the paper-based versions. This prompts questions of whether different requirements are thus made of students with consequential changes in respect of the measured competences (construct equivalence) and furthermore, whether the difficulty of the exercises changes (mode effects). The project will also explore whether there is a difference in how boys and girls process the exercises and if there is, what factors have a bearing on this.
A second important aspect is the comparability of the findings of all previous PISA studies. Besides the description of the results of the respective current PISA study, the national PISA collective report always also analyses and interprets development (trend). The switch to computer-based testing means that a straightforward comparison is no longer possible. Mode effects must always also be taken into account in analyses and comparisons. This add-on study will therefore research how strong mode effects are, and the extent to which they impact on comparability so that reliable assertions about national trends can also be derived in the future.
You can find information on the DIPF national add-on study (Mode effects in PISA 2018) here.