Skip to main navigation Skip to search Skip to main content

Can Thinking Models Think to Detect Hateful Memes?

  • Mohamed Bayan Kmainasi*
  • , Mucahid Kutlu
  • , Ali Ezzat Shahroor
  • , Abul Hasnat
  • , Firoj Alam
  • *Corresponding author for this work
  • Qatar University
  • Hamad bin Khalifa University
  • Noisy Le Grand
  • Blackbird.AI

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

Abstract

Hateful memes often require compositional multimodal reasoning: the image and text may appear benign in isolation, yet their interaction conveys harmful intent. Although thinking-based multimodal large language models (MLLMs) have recently advanced vision-language understanding, their capabilities remain underexplored for hateful meme analysis. We propose a reinforcement learning-based post-training framework that improves reasoning in thinking-based MLLMs via task-specific rewards and a novel Group Relative Policy Optimization (GRPO) objective. Concretely, we (i) conduct a systematic empirical study of off-the-shelf MLLMs for hateful meme understanding, (ii) extend an existing hateful meme dataset by generating weakly/pseudo-supervised chain-of-thought (CoT) rationales via distillation, and (iii) introduce a GRPO-based objective that jointly optimizes meme classification and explanation quality to encourage fine-grained step-by-step reasoning. Experiments on the Hateful Memes benchmark show that our approach achieves state-of-the-art results, improving accuracy and F1 by approximately 1% and explanation quality by approximately 3%. We will publicly release our code, data extensions, and evaluation resources to support reproducibility.

Original languageEnglish
Title of host publicationWWW Companion 2026 - Companion Proceedings of the ACM Web Conference 2026
PublisherAssociation for Computing Machinery, Inc
Pages935-944
Number of pages10
ISBN (Electronic)9798400723087
DOIs
Publication statusPublished - 28 May 2026
Event35th ACM Web Conference, WWW Companion 2026 - Dubai, United Arab Emirates
Duration: 29 Jun 20263 Jul 2026

Publication series

NameWWW Companion 2026 - Companion Proceedings of the ACM Web Conference 2026

Conference

Conference35th ACM Web Conference, WWW Companion 2026
Country/TerritoryUnited Arab Emirates
CityDubai
Period29/06/263/07/26

Keywords

  • hateful meme detection
  • multimodal large language models
  • multimodal reasoning
  • reinforcement learning

Fingerprint

Dive into the research topics of 'Can Thinking Models Think to Detect Hateful Memes?'. Together they form a unique fingerprint.

Cite this