TY - JOUR
T1 - Overview of the CLEF-2019 Checkthat! LAB
T2 - 20th Working Notes of CLEF Conference and Labs of the Evaluation Forum, CLEF 2019
AU - Hasanain, Maram
AU - Suwaileh, Reem
AU - Elsayed, Tamer
AU - Barrón-Cedeño, Alberto
AU - Nakov, Preslav
N1 - Publisher Copyright:
© 2019 for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0). CLEF 2019, 9-12 September 2019, Lugano, Switzerland.
PY - 2019
Y1 - 2019
N2 - We present an overview of Task 2 of the second edition of the CheckThat! Lab at CLEF 2019. Task 2 asked (A) to rank a given set of Web pages with respect to a check-worthy claim based on their usefulness for fact-checking that claim, (B) to classify these same Web pages according to their degree of usefulness for fact-checking the target claim, (C) to identify useful passages from these pages, and (D) to use the useful pages to predict the claim's factuality. Task 2 at CheckThat! provided a full evaluation framework, consisting of data in Arabic (gathered and annotated from scratch) and evaluation based on normalized discounted cumulative gain (nDCG) for ranking, and F1 for classification. Four teams submitted runs. The most successful approach to subtask A used learning-to-rank, while different classifiers were used in the other subtasks. We release to the research community all datasets from the lab as well as the evaluation scripts, which should enable further research in the important task of evidence-based automatic claim verification.
AB - We present an overview of Task 2 of the second edition of the CheckThat! Lab at CLEF 2019. Task 2 asked (A) to rank a given set of Web pages with respect to a check-worthy claim based on their usefulness for fact-checking that claim, (B) to classify these same Web pages according to their degree of usefulness for fact-checking the target claim, (C) to identify useful passages from these pages, and (D) to use the useful pages to predict the claim's factuality. Task 2 at CheckThat! provided a full evaluation framework, consisting of data in Arabic (gathered and annotated from scratch) and evaluation based on normalized discounted cumulative gain (nDCG) for ranking, and F1 for classification. Four teams submitted runs. The most successful approach to subtask A used learning-to-rank, while different classifiers were used in the other subtasks. We release to the research community all datasets from the lab as well as the evaluation scripts, which should enable further research in the important task of evidence-based automatic claim verification.
KW - Computational Journalism
KW - Evidence-based Verification
KW - Fact-Checking
KW - Fake News Detection
KW - Veracity
UR - https://www.scopus.com/pages/publications/85070501566
M3 - Conference article
AN - SCOPUS:85070501566
SN - 1613-0073
VL - 2380
JO - CEUR Workshop Proceedings
JF - CEUR Workshop Proceedings
Y2 - 9 September 2019 through 12 September 2019
ER -