The role of reliability in experiments
British Journal of Mathematical and Statistical Psychology · 2026-11 · 79 (3) · 687-701
DOI: 10.1111/bmsp.70042 ↗Kurz gefasst
Fachartikel in der Zeitschrift British Journal of Mathematical and Statistical Psychology, online erschienen am 16.03.2026. Verfasst von Jeffrey N Rouder und 2 weiteren Autor:innen. Beteiligte Einrichtung: University of California, Irvine, Irvine, California, USA. Hauptthemen laut MEDLINE: Research Design. Die Arbeit wurde bisher 4-mal zitiert (Crossref) und nennt 20 Literaturangaben. Gefördert von: Office of Naval Research.
Abstract (Originalsprache)
We are concerned about an emphasis on reliability for analysis of psychology experiments. Experiments have two elements of sample size: the number of individuals and the number of replicate trials within a task, and that complicates reliability measures. To account for these elements, we distinguish among three levels of analysis: (1) A foundational level that centers task properties without recourse to either element of sample size. An example statistic is intraclass correlation which is the proportion of variances without reference to sample sizes. (2) An intermediate level that centers the number of trials but not the number of individuals. An example statistic on this level is reliability which describes variabilities with reference to numbers of trials but not numbers of individuals. A final level centers both the numbers of individuals and trials. An example quantity is the uncertainty in a correlation coefficient, which, ideally, reflects sample size limits in individuals and trials. Reliability describes an intermediate level – neither useful for communicating foundational task properties nor interpreting correlations. We advocate that researchers consider all three levels and highlight the role of hierarchical models in doing so.
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Stichwörter der Autor:innen
Autor:innen und Institutionen
- University of California, Irvine, Irvine, California, USA.
- University of Mannheim, Mannheim, Germany.
ORCID-Profile: Jeffrey N Rouder, Mahbod Mehrvarz
Förderung
- Office of Naval Research
Zitieren
APA
Rouder, J. N., Mehrvarz, M., & Schnuerch, M. (2026). The role of reliability in experiments. British Journal of Mathematical and Statistical Psychology, 79(3), 687-701. https://doi.org/10.1111/bmsp.70042
Vancouver
Rouder JN, Mehrvarz M, Schnuerch M. The role of reliability in experiments. British Journal of Mathematical and Statistical Psychology. 2026;79(3):687-701. doi:10.1111/bmsp.70042
BibTeX
@article{Rouder2026,
author = {Rouder, Jeffrey N and Mehrvarz, Mahbod and Schnuerch, Martin},
title = {The role of reliability in experiments},
journal = {British Journal of Mathematical and Statistical Psychology},
year = {2026},
volume = {79},
number = {3},
pages = {687--701},
doi = {10.1111/bmsp.70042}
}
EUVIMED