CheckThat! at CLEF 2019: Automatic identification and verification of claims

Tamer Elsayed; Preslav Nakov; Alberto Barrón-Cedeño; Maram Hasanain; Reem Suwaileh; Giovanni Da San Martino; Pepa Atanasova

doi:10.1007/978-3-030-15719-7_41

CheckThat! at CLEF 2019: Automatic identification and verification of claims

Tamer Elsayed, Preslav Nakov, Alberto Barrón-Cedeño, Maram Hasanain^*, Reem Suwaileh, Giovanni Da San Martino, Pepa Atanasova

^*Corresponding author for this work

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

39 Citations (Scopus)

Abstract

We introduce the second edition of the CheckThat! Lab, part of the 2019 Cross-Language Evaluation Forum (CLEF). CheckThat! proposes two complementary tasks. Task 1: predict which claims in a political debate should be prioritized for fact-checking. Task 2: rank Web-retrieved pages against a check-worthy claim based on their usefulness for fact-checking, extract useful passages from those pages, and then use them all to decide whether the claim is factually true or false. Checkthat! provides a full evaluation framework, consisting of data in English (derived from fact-checking sources) and Arabic (gathered and annotated from scratch) and evaluation based on mean average precision (MAP) for ranking and F ₁ for classification tasks.

Original language	English
Title of host publication	Advances in Information Retrieval - 41st European Conference on IR Research, ECIR 2019, Proceedings
Editors	Benno Stein, Djoerd Hiemstra, Norbert Fuhr, Philipp Mayr, Claudia Hauff, Leif Azzopardi
Publisher	Springer Verlag
Pages	309-315
Number of pages	7
ISBN (Print)	9783030157180
DOIs	https://doi.org/10.1007/978-3-030-15719-7_41
Publication status	Published - 2019
Externally published	Yes
Event	41st European Conference on Information Retrieval, ECIR 2019 - Cologne, Germany Duration: 14 Apr 2019 → 18 Apr 2019

Publication series

Name	Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume	11438 LNCS
ISSN (Print)	0302-9743
ISSN (Electronic)	1611-3349

Conference

Conference	41st European Conference on Information Retrieval, ECIR 2019
Country/Territory	Germany
City	Cologne
Period	14/04/19 → 18/04/19

Access to Document

10.1007/978-3-030-15719-7_41

Cite this

Elsayed, T., Nakov, P., Barrón-Cedeño, A., Hasanain, M., Suwaileh, R., Da San Martino, G., & Atanasova, P. (2019). CheckThat! at CLEF 2019: Automatic identification and verification of claims. In B. Stein, D. Hiemstra, N. Fuhr, P. Mayr, C. Hauff, & L. Azzopardi (Eds.), Advances in Information Retrieval - 41st European Conference on IR Research, ECIR 2019, Proceedings (pp. 309-315). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 11438 LNCS). Springer Verlag. https://doi.org/10.1007/978-3-030-15719-7_41

Elsayed, Tamer ; Nakov, Preslav ; Barrón-Cedeño, Alberto et al. / CheckThat! at CLEF 2019 : Automatic identification and verification of claims. Advances in Information Retrieval - 41st European Conference on IR Research, ECIR 2019, Proceedings. editor / Benno Stein ; Djoerd Hiemstra ; Norbert Fuhr ; Philipp Mayr ; Claudia Hauff ; Leif Azzopardi. Springer Verlag, 2019. pp. 309-315 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)).

@inproceedings{d8bf9b0196d2460280d54d3ed7ea79ed,

title = "CheckThat! at CLEF 2019: Automatic identification and verification of claims",

abstract = "We introduce the second edition of the CheckThat! Lab, part of the 2019 Cross-Language Evaluation Forum (CLEF). CheckThat! proposes two complementary tasks. Task 1: predict which claims in a political debate should be prioritized for fact-checking. Task 2: rank Web-retrieved pages against a check-worthy claim based on their usefulness for fact-checking, extract useful passages from those pages, and then use them all to decide whether the claim is factually true or false. Checkthat! provides a full evaluation framework, consisting of data in English (derived from fact-checking sources) and Arabic (gathered and annotated from scratch) and evaluation based on mean average precision (MAP) for ranking and F 1 for classification tasks.",

author = "Tamer Elsayed and Preslav Nakov and Alberto Barr{\'o}n-Cede{\~n}o and Maram Hasanain and Reem Suwaileh and \{Da San Martino\}, Giovanni and Pepa Atanasova",

note = "Publisher Copyright: {\textcopyright} Springer Nature Switzerland AG 2019.; 41st European Conference on Information Retrieval, ECIR 2019 ; Conference date: 14-04-2019 Through 18-04-2019",

year = "2019",

doi = "10.1007/978-3-030-15719-7\_41",

language = "English",

isbn = "9783030157180",

series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",

publisher = "Springer Verlag",

pages = "309--315",

editor = "Benno Stein and Djoerd Hiemstra and Norbert Fuhr and Philipp Mayr and Claudia Hauff and Leif Azzopardi",

booktitle = "Advances in Information Retrieval - 41st European Conference on IR Research, ECIR 2019, Proceedings",

address = "Germany",

}

Elsayed, T, Nakov, P, Barrón-Cedeño, A, Hasanain, M, Suwaileh, R, Da San Martino, G & Atanasova, P 2019, CheckThat! at CLEF 2019: Automatic identification and verification of claims. in B Stein, D Hiemstra, N Fuhr, P Mayr, C Hauff & L Azzopardi (eds), Advances in Information Retrieval - 41st European Conference on IR Research, ECIR 2019, Proceedings. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 11438 LNCS, Springer Verlag, pp. 309-315, 41st European Conference on Information Retrieval, ECIR 2019, Cologne, Germany, 14/04/19. https://doi.org/10.1007/978-3-030-15719-7_41

CheckThat! at CLEF 2019: Automatic identification and verification of claims. / Elsayed, Tamer; Nakov, Preslav; Barrón-Cedeño, Alberto et al.
Advances in Information Retrieval - 41st European Conference on IR Research, ECIR 2019, Proceedings. ed. / Benno Stein; Djoerd Hiemstra; Norbert Fuhr; Philipp Mayr; Claudia Hauff; Leif Azzopardi. Springer Verlag, 2019. p. 309-315 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 11438 LNCS).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - CheckThat! at CLEF 2019

T2 - 41st European Conference on Information Retrieval, ECIR 2019

AU - Elsayed, Tamer

AU - Nakov, Preslav

AU - Barrón-Cedeño, Alberto

AU - Hasanain, Maram

AU - Suwaileh, Reem

AU - Da San Martino, Giovanni

AU - Atanasova, Pepa

N1 - Publisher Copyright: © Springer Nature Switzerland AG 2019.

PY - 2019

Y1 - 2019

N2 - We introduce the second edition of the CheckThat! Lab, part of the 2019 Cross-Language Evaluation Forum (CLEF). CheckThat! proposes two complementary tasks. Task 1: predict which claims in a political debate should be prioritized for fact-checking. Task 2: rank Web-retrieved pages against a check-worthy claim based on their usefulness for fact-checking, extract useful passages from those pages, and then use them all to decide whether the claim is factually true or false. Checkthat! provides a full evaluation framework, consisting of data in English (derived from fact-checking sources) and Arabic (gathered and annotated from scratch) and evaluation based on mean average precision (MAP) for ranking and F 1 for classification tasks.

AB - We introduce the second edition of the CheckThat! Lab, part of the 2019 Cross-Language Evaluation Forum (CLEF). CheckThat! proposes two complementary tasks. Task 1: predict which claims in a political debate should be prioritized for fact-checking. Task 2: rank Web-retrieved pages against a check-worthy claim based on their usefulness for fact-checking, extract useful passages from those pages, and then use them all to decide whether the claim is factually true or false. Checkthat! provides a full evaluation framework, consisting of data in English (derived from fact-checking sources) and Arabic (gathered and annotated from scratch) and evaluation based on mean average precision (MAP) for ranking and F 1 for classification tasks.

UR - https://www.scopus.com/pages/publications/85064878114

U2 - 10.1007/978-3-030-15719-7_41

DO - 10.1007/978-3-030-15719-7_41

M3 - Conference contribution

AN - SCOPUS:85064878114

SN - 9783030157180

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 309

EP - 315

BT - Advances in Information Retrieval - 41st European Conference on IR Research, ECIR 2019, Proceedings

A2 - Stein, Benno

A2 - Hiemstra, Djoerd

A2 - Fuhr, Norbert

A2 - Mayr, Philipp

A2 - Hauff, Claudia

A2 - Azzopardi, Leif

PB - Springer Verlag

Y2 - 14 April 2019 through 18 April 2019

ER -

Elsayed T, Nakov P, Barrón-Cedeño A, Hasanain M, Suwaileh R, Da San Martino G et al. CheckThat! at CLEF 2019: Automatic identification and verification of claims. In Stein B, Hiemstra D, Fuhr N, Mayr P, Hauff C, Azzopardi L, editors, Advances in Information Retrieval - 41st European Conference on IR Research, ECIR 2019, Proceedings. Springer Verlag. 2019. p. 309-315. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)). doi: 10.1007/978-3-030-15719-7_41

CheckThat! at CLEF 2019: Automatic identification and verification of claims

Abstract

Publication series

Conference

Access to Document

Other files and links

Fingerprint

Cite this