EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Replicate Me if You Can: Assessing Measurement Reliability of Individual Differences in Reading Across Measurement Occasions and Methods

Patrick Haller iD, Cui Ding iD, Maja Stegenwallner-Schütz iD, David R Reich iD, Iva Koncic, Silvia Makowski, Lena A Jäger iD

Cognitive Science · 2026-01 · 50 (1)

DOI: 10.1111/cogs.70121 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Cognitive Science, online erschienen am 30.12.2025. Verfasst von Patrick Haller und 6 weiteren Autor:innen. Beteiligte Einrichtung: Department of Computational Linguistics, University of Zurich. Hauptthemen laut MEDLINE: Reading, Individuality, Psycholinguistics, Eye Movements. Die Arbeit wurde bisher 4-mal zitiert (Crossref) und nennt 116 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Schweizerischer Nationalfonds zur Förderung der Wissenschaftlichen Forschung, Bundesministerium für Bildung und Forschung.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Psycholinguistic theories traditionally assume similar cognitive mechanisms across different speakers. However, more recently, researchers have begun to recognize the need to consider individual differences when explaining human cognition. An increasing number of studies have investigated how individual differences influence human sentence processing. Implicitly, these studies assume that individual‐level effects can be replicated across experimental sessions and different assessment methods such as eye‐tracking and self‐paced reading. However, this assumption is challenged by the Reliability Paradox. Thus, a crucial first step for a principled investigation of individual differences in sentence processing is to establish their measurement reliability, that is, the correlation of individual‐level effects across multiple measurement occasions and methods. In this work, we present the first naturalistic eye movement corpus of reading data with four experimental sessions from each participant (two eye‐tracking sessions and two self‐paced reading sessions). We deploy a two‐task Bayesian hierarchical model to assess the measurement reliability of individual differences in a range of psycholinguistic phenomena that are well‐established at the population level, namely, effects of word length, lexical frequency, surprisal, dependency length, and number of to‐be‐integrated dependents. While our results indicate high reliability across measurement occasions for the word length effect, it is only moderate for higher‐level psycholinguistic predictors such as lexical frequency, dependency distance, and the number of to‐be‐integrated dependencies, and even low for surprisal. Moreover, even after accounting for spillover effects, we observe only low to moderate reliability at the individual level across methods (eye‐tracking and self‐paced reading) for most predictors, and poor reliability for predictors of syntactic integration. These findings underscore the importance of establishing measurement reliability before drawing inferences about individual differences in sentence processing.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

ReadingIndividualityPsycholinguisticsEye MovementsHumansReproducibility of ResultsEye-Tracking TechnologyMaleAdultFemaleBayes TheoremYoung Adult

Stichwörter der Autor:innen

Eye‐trackingIndividual differencesMeasurement reliabilityNaturalistic reading corpusSelf‐paced readingSentence processingTwo‐task Bayesian modeling

Autor:innen und Institutionen

  • Department of Computational Linguistics, University of Zurich.
  • Department of Computer Science, University of Potsdam.
  • Department of Special Education, University of Koblenz.

ORCID-Profile: Patrick Haller, Cui Ding, Maja Stegenwallner-Schütz, David R Reich, Lena A Jäger

Förderung

  • Schweizerischer Nationalfonds zur Förderung der Wissenschaftlichen Forschung
  • Bundesministerium für Bildung und Forschung

Zitieren

APA

Haller, P., Ding, C., Stegenwallner-Schütz, M., Reich, D. R., Koncic, I., Makowski, S., & Jäger, L. A. (2026). Replicate Me if You Can: Assessing Measurement Reliability of Individual Differences in Reading Across Measurement Occasions and Methods. Cognitive Science, 50(1). https://doi.org/10.1111/cogs.70121

Vancouver

Haller P, Ding C, Stegenwallner-Schütz M, Reich DR, Koncic I, Makowski S, et al. Replicate Me if You Can: Assessing Measurement Reliability of Individual Differences in Reading Across Measurement Occasions and Methods. Cognitive Science. 2026;50(1). doi:10.1111/cogs.70121

BibTeX
@article{Haller2026,
  author = {Haller, Patrick and Ding, Cui and Stegenwallner-Schütz, Maja and Reich, David R and Koncic, Iva and Makowski, Silvia and Jäger, Lena A},
  title = {Replicate Me if You Can: Assessing Measurement Reliability of Individual Differences in Reading Across Measurement Occasions and Methods},
  journal = {Cognitive Science},
  year = {2026},
  volume = {50},
  number = {1},
  doi = {10.1111/cogs.70121}
}
Kostenlos starten →