TY - GEN
T1 - Can Thinking Models Think to Detect Hateful Memes?
AU - Kmainasi, Mohamed Bayan
AU - Kutlu, Mucahid
AU - Ezzat Shahroor, Ali
AU - Hasnat, Abul
AU - Alam, Firoj
N1 - Publisher Copyright:
© 2026 Owner/Author.
PY - 2026/5/28
Y1 - 2026/5/28
N2 - Hateful memes often require compositional multimodal reasoning: the image and text may appear benign in isolation, yet their interaction conveys harmful intent. Although thinking-based multimodal large language models (MLLMs) have recently advanced vision-language understanding, their capabilities remain underexplored for hateful meme analysis. We propose a reinforcement learning-based post-training framework that improves reasoning in thinking-based MLLMs via task-specific rewards and a novel Group Relative Policy Optimization (GRPO) objective. Concretely, we (i) conduct a systematic empirical study of off-the-shelf MLLMs for hateful meme understanding, (ii) extend an existing hateful meme dataset by generating weakly/pseudo-supervised chain-of-thought (CoT) rationales via distillation, and (iii) introduce a GRPO-based objective that jointly optimizes meme classification and explanation quality to encourage fine-grained step-by-step reasoning. Experiments on the Hateful Memes benchmark show that our approach achieves state-of-the-art results, improving accuracy and F1 by approximately 1% and explanation quality by approximately 3%. We will publicly release our code, data extensions, and evaluation resources to support reproducibility.
AB - Hateful memes often require compositional multimodal reasoning: the image and text may appear benign in isolation, yet their interaction conveys harmful intent. Although thinking-based multimodal large language models (MLLMs) have recently advanced vision-language understanding, their capabilities remain underexplored for hateful meme analysis. We propose a reinforcement learning-based post-training framework that improves reasoning in thinking-based MLLMs via task-specific rewards and a novel Group Relative Policy Optimization (GRPO) objective. Concretely, we (i) conduct a systematic empirical study of off-the-shelf MLLMs for hateful meme understanding, (ii) extend an existing hateful meme dataset by generating weakly/pseudo-supervised chain-of-thought (CoT) rationales via distillation, and (iii) introduce a GRPO-based objective that jointly optimizes meme classification and explanation quality to encourage fine-grained step-by-step reasoning. Experiments on the Hateful Memes benchmark show that our approach achieves state-of-the-art results, improving accuracy and F1 by approximately 1% and explanation quality by approximately 3%. We will publicly release our code, data extensions, and evaluation resources to support reproducibility.
KW - hateful meme detection
KW - multimodal large language models
KW - multimodal reasoning
KW - reinforcement learning
UR - https://www.scopus.com/pages/publications/105041961483
U2 - 10.1145/3774905.3795465
DO - 10.1145/3774905.3795465
M3 - Conference contribution
AN - SCOPUS:105041961483
T3 - WWW Companion 2026 - Companion Proceedings of the ACM Web Conference 2026
SP - 935
EP - 944
BT - WWW Companion 2026 - Companion Proceedings of the ACM Web Conference 2026
PB - Association for Computing Machinery, Inc
T2 - 35th ACM Web Conference, WWW Companion 2026
Y2 - 29 June 2026 through 3 July 2026
ER -