EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Beyond AUC: Evaluating Deep Knowledge Tracing Through Temporal Coherence and Post Hoc Explanations

Ekaterina Krivich, Gustav Šír, Tommi Kärkkäinen, Danial Hooshyar

Electronics · 27.10.2026 · 15 (21) · 4975

DOI: 10.3390/electronics15214975 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Electronics, online erschienen am 27.10.2026. Verfasst von Ekaterina Krivich und 3 weiteren Autor:innen. Die Arbeit nennt 25 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Estonian Research Council.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Deep knowledge tracing (KT) models estimate students’ evolving skill mastery from sequences of question–response events and predict whether the next response will be correct. They are often compared using AUC, a score of how well predictions distinguish correct from incorrect answers. We evaluate two complementary aspects beyond AUC: temporal prediction behavior for 11 KT models on four educational datasets, and attribution diagnostics on ASSISTments 2017 data. The relationship between AUC and response-direction consistency varies by dataset: models with higher AUC sometimes show more, and sometimes fewer, prediction drops after correct answers or prediction rises after incorrect answers on repeated skills. Reference baselines further show that stability must be interpreted together with responsiveness, because a model that never updates its prediction also appears perfectly stable. In an exploratory ASSISTments 2017 analysis of 50 histories from 33 learners per model, gradient-based selected question–response events produce modest, model-dependent signed probability changes (original minus post-replacement probability) relative to recent-history, same-skill, and random controls. For practitioners comparing KT models, the paper contributes two practical checks to report alongside AUC: whether predictions respond coherently after student answers, and how explanation-selected history events compare with simple recent-history or same-skill alternatives under a shared intervention.

Förderung

  • Estonian Research Council

Zitieren

APA

Krivich, E., Šír, G., Kärkkäinen, T., & Hooshyar, D. (2026). Beyond AUC: Evaluating Deep Knowledge Tracing Through Temporal Coherence and Post Hoc Explanations. Electronics, 15(21), 4975. https://doi.org/10.3390/electronics15214975

Vancouver

Krivich E, Šír G, Kärkkäinen T, Hooshyar D. Beyond AUC: Evaluating Deep Knowledge Tracing Through Temporal Coherence and Post Hoc Explanations. Electronics. 2026;15(21):4975. doi:10.3390/electronics15214975

BibTeX
@article{Krivich2026,
  author = {Krivich, Ekaterina and Šír, Gustav and Kärkkäinen, Tommi and Hooshyar, Danial},
  title = {Beyond AUC: Evaluating Deep Knowledge Tracing Through Temporal Coherence and Post Hoc Explanations},
  journal = {Electronics},
  year = {2026},
  volume = {15},
  number = {21},
  pages = {4975},
  doi = {10.3390/electronics15214975}
}
Kostenlos starten →