Factual Editing in Large Language Models: A Study of Locality-Preserving Editing with AlphaEdit

Abstract

Modern systems for language understanding and generation are largely built on large language models (LLMs). Since these models are trained on extensive text corpora, some of the information they encode may become outdated or incorrect over time. Modifying such knowledge without retraining the entire model remains difficult. This thesis studies factual editing in LLMs and examines how factual editing can be made while preserving unrelated knowledge. As a case study, we examine the AlphaEdit methodology (Fang et al., 2025), which is designed to modify specific facts in a model while maintaining locality. The approach is evaluated on Fanar, an Arabic–English language model developed by QCRI. As part of this analysis, we first examine where factual knowledge is represented within the model by studying different layers of the network. Using the most effective layer region, we apply our chosen methodology to update multiple facts and study whether these edits interfere with one another. We then evaluate whether the modified knowledge is used correctly in reasoning tasks that require combining complex information. Finally, we examine whether the model’s general language and reasoning abilities remain safe after editing. Our results show that factual edits can be applied while maintaining stable reasoning performance. In the experiments conducted with AlphaEdit, the method demonstrates strong robustness compared with baseline approaches. Overall, these findings provide insight into how factual updates can be implemented in LLMs.

Date of Award	2026
Original language	American English
Awarding Institution	HBKU College of Science and Engineering