UQuAD+: Benchmark Dataset for Urdu Machine Reading Comprehension
ACM Transactions on Asian and Low-Resource Language Information Processing · 28.02.2026 · 25 (2) · 1-34
DOI: 10.1145/3759455 ↗Kurz gefasst
Fachartikel in der Zeitschrift ACM Transactions on Asian and Low-Resource Language Information Processing, online erschienen am 14.02.2026. Verfasst von Samreen Kazi und 1 weiteren Autor:innen. Die Arbeit wurde bisher 1-mal zitiert (Crossref) und nennt 69 Literaturangaben.
Abstract (Originalsprache)
Machine Reading Comprehension (MRC) is a key task in Natural Language Understanding that enables automated systems to answer questions based on textual input. While MRC has made substantial progress for high-resource languages, low-resource languages pose significant challenges due to their complex linguistic features. This article presents a comprehensive human-annotated dataset for Urdu MRC, comprising 20,000 question-answer pairs derived from 1,540 articles across seven domains. Unlike previous translation-based datasets, this dataset contains question-answer pairs created through rigorous crowd-sourcing and expert annotation. The dataset encompasses diverse question types, including both answerable and unanswerable questions, with answers ranging from single words to complete sentences, effectively capturing Urdu’s morphological richness and syntactic diversity. To address the limitations of traditional evaluation metrics like Exact Match (EM) and \(F_1\) in assessing Urdu answers, we propose Semantic Match (SM), a metric designed to measure semantic equivalence between predicted and ground-truth answers. Our evaluation demonstrates the dataset’s increased complexity, with state-of-the-art models achieving only 0.82% SM accuracy. Together, the dataset and evaluation metric establish a robust framework for advancing Urdu MRC research, bridging critical gaps in both dataset quality and evaluation methodology.
Zitieren
APA
Kazi, S., & Khoja, S. (2026). UQuAD+: Benchmark Dataset for Urdu Machine Reading Comprehension. ACM Transactions on Asian and Low-Resource Language Information Processing, 25(2), 1-34. https://doi.org/10.1145/3759455
Vancouver
Kazi S, Khoja S. UQuAD+: Benchmark Dataset for Urdu Machine Reading Comprehension. ACM Transactions on Asian and Low-Resource Language Information Processing. 2026;25(2):1-34. doi:10.1145/3759455
BibTeX
@article{Kazi2026,
author = {Kazi, Samreen and Khoja, Shakeel},
title = {UQuAD+: Benchmark Dataset for Urdu Machine Reading Comprehension},
journal = {ACM Transactions on Asian and Low-Resource Language Information Processing},
year = {2026},
volume = {25},
number = {2},
pages = {1--34},
doi = {10.1145/3759455}
}
EUVIMED