TY - JOUR
T1 - Overview of the CLEF-2021 CheckThat! Lab Task 2 on detecting previously fact-checked claims in tweets and political debates
AU - Shaar, Shaden
AU - Haouari, Fatima
AU - Mansour, Watheq
AU - Hasanain, Maram
AU - Babulkov, Nikolay
AU - Alam, Firoj
AU - da San Martino, Giovanni
AU - Elsayed, Tamer
AU - Nakov, Preslav
N1 - Publisher Copyright:
© 2021 Copyright for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0).
PY - 2021
Y1 - 2021
N2 - We describe the fourth edition of the CheckThat! Lab, part of the 2021 Conference and Labs of the Evaluation Forum (CLEF). The lab evaluates technology supporting three tasks related to factuality, and it covers Arabic, Bulgarian, English, Spanish, and Turkish. Here, we present the task 2, which asks to detect previously fact-checked claims (in two languages). A total of four teams participated in this task, submitted a total of sixteen runs, and most submissions managed to achieve sizable improvements over the baselines using transformer based models such as BERT, RoBERTa. In this paper, we describe the process of data collection and the task setup, including the evaluation measures used, and we give a brief overview of the participating systems. Last but not least, we release to the research community all datasets from the lab as well as the evaluation scripts, which should enable further research in detecting previously fact-checked claims.
AB - We describe the fourth edition of the CheckThat! Lab, part of the 2021 Conference and Labs of the Evaluation Forum (CLEF). The lab evaluates technology supporting three tasks related to factuality, and it covers Arabic, Bulgarian, English, Spanish, and Turkish. Here, we present the task 2, which asks to detect previously fact-checked claims (in two languages). A total of four teams participated in this task, submitted a total of sixteen runs, and most submissions managed to achieve sizable improvements over the baselines using transformer based models such as BERT, RoBERTa. In this paper, we describe the process of data collection and the task setup, including the evaluation measures used, and we give a brief overview of the participating systems. Last but not least, we release to the research community all datasets from the lab as well as the evaluation scripts, which should enable further research in detecting previously fact-checked claims.
KW - COVID-19
KW - Check-worthiness estimation
KW - Computational journalism
KW - Detecting previously fact-checked claims
KW - Fact-checking
KW - Social media verification
KW - Veracity
KW - Verified claims retrieval
UR - https://www.scopus.com/pages/publications/85113494033
M3 - Conference article
AN - SCOPUS:85113494033
SN - 1613-0073
VL - 2936
SP - 393
EP - 405
JO - CEUR Workshop Proceedings
JF - CEUR Workshop Proceedings
T2 - 22nd Working Notes of CLEF - Conference and Labs of the Evaluation Forum, CLEF-WN 2021
Y2 - 21 September 2021 through 24 September 2021
ER -