Methodological challenges in estimating trends and
cross-country comparisons in school performance studies
Lead: IPN
Joint Research with: DIPF
Current project members: IPN | Leibniz Institute for Science and Mathematics Education (Prof. Dr. Oliver Lüdtke➚, Dr. Alexander Robitzsch➚)
An important goal of international school achievement studies such as PISA is to compare the average competence of students of a certain age group in different countries. These comparisons are made for different domains (mathematics, reading or science) and may relate (1) to a specific point in time or (2) to trends over time. Both comparisons are based on the assumption that a common metric about the participating States as well as about time can be established with the help of so-called linkitems (test items of the different domains), which are used in different participating States and for different surveys. So far, however, little research has been carried out into which sources of uncertainty (e.g. selection of link items, drawing of students) are to be taken into account in the various comparisons and which statistical methods are best suited to quantifying this uncertainty.
In the present project a systematic evaluation of different linking methods is to be carried out. Furthermore, statistical methods will be developed to adequately determine the uncertainty of cross-sectional country comparisons and longitudinal trend estimates. In addition, there is also a close cooperation with the project on mode effects in PISA 2018, in which the developed methods are to be applied.
A key objective of the project is to gain a better understanding of the various factors on which the uncertainty of country comparisons and trend estimates depends. This will be used to improve statistical methods for estimating trends and to make recommendations for optimized test designs. In addition, the methods developed could also be used to optimize the national monitoring of federal states with regard to the achievement of educational standards (german Bildungsstandards➚ – only available in german).
