AI-GENERATED SUBTITLING AND CULTURAL REFERENCES: A QUALITY ASSESSMENT USING MQM FRAMEWORK AND THE FAR MODEL

Abstract

This thesis investigates how Maestra AI handles culture-bound expressions when translating them from English into Arabic, through an analysis of its generated subtitles and a comparison with professionally produced Disney+ subtitles. Culture-bound expressions pose a particular challenge in audiovisual translation because they often involve meanings closely tied to the source culture and may not have direct equivalents in the target language. This study aims to examine whether Maestra AI can render such expressions and how its output compares to that of human translators. To assess subtitle quality, two evaluation frameworks were applied: the Multidimensional Quality Metrics (MQM) and the FAR model. The study analyzes a dataset of 100 subtitles containing culture-bound expressions translated by Maestra AI and compares them with 100 corresponding translations produced by professional human translators. The results show a clear difference between AI and human performance. According to the MQM analysis, 62% of AI subtitles contained accuracy errors, while 38% were rendered without errors. In contrast, human subtitles achieved 72% of the examples without errors, while only 28% contained accuracy errors. At the subtype level, mistranslation (34%) and under-translation (23%) were the most frequent error types in Maestra AI-generated subtitles. Human subtitles showed significantly lower rates of mistranslation (13%) and under translation (6%), which suggest that humans achieved cultural meaning and context understanding. The FAR model results support these findings. Maestra AI-generated subtitles contained 54% semantic errors and 8% stylistic errors, compared to 25% semantic and 3% stylistic errors in human translations. Regarding the severity of errors, AI subtitles had 32% serious errors, while human subtitles contained only 7% serious errors. This indicates that AI translation significantly affects meaning in comparison to human translation. Overall, the findings demonstrate that Maestra AI-generated subtitles struggle with cultural depth. Although Maestra AI can successfully translate some culture-bound expressions, it does not yet match human translators in preserving semantic accuracy and contextual meaning. The study confirms that human revision remains essential in AI-generated subtitling, especially when dealing with culture-bound expressions.

Date of Award	2026
Original language	American English
Awarding Institution	HBKU College of Humanities and Social Science