DEVELOPMENT OF AN INTELLIGENT LEGAL AGENT FOR ARABIC LEGISLATIVE FRAMEWORK: A HYBRID RAG AND EXPLAINABLE AI APPROACH FOR GOVERNMENT AUDITING IN QATAR

Abstract

Auditors at State Audit Bureau (SAB) Qatar encounter difficulties, since their role requires the examination of legal documents to ascertain compliance with relevant institutions. Thus, auditors spend time manually searching through many legal documents written in Arabic because of how difficult these are to manage using their current auditing tools. An Intelligent Legal Analytics Agent that uses a hybrid Retrieval-Augmented Generation (RAG) architecture has been developed to help solve this problem. The agent combines two different approaches/methods to improve Arabic legal information retrieval by enabling both a lexical search (BM25), and a semantic search (embeddings). To comply with government auditing regulations, the agent also makes sure that all citations are correct and that their explanations are clear. The scope of the dataset was narrowed to SAB department’s functionalities - Projects and Contracts, Audit and Financial Examination, Performance and Compliance, HR, Information Systems. The final dataset consists of 64 legal texts (laws, ministerial decisions, decrees, regulatory handbooks, circulars, as well as international auditing standards). Using a multi-dimensional scoring methodology that focused on Law match, Article match, Relevance, and Hallucination Prevention, the agent's performance was assessed; it achieved an average accuracy of 68.3 percent. The RAG design was validated by the agent's effective prevention of hallucinations, and its 93.3 percent response rate demonstrated its exceptional relevance. Its poor citation-level accuracy of 16.7 percent was caused by a lack of ground truth data in the retrieval of legal knowledge. This research introduces the first Arabic-centric legal analytics solution tailored for public sector auditors in Qatar, addressing the lack of researchers who have worked to tackle the challenges faced by these auditors utilizing extensive Arabic language documentation. This thesis identifies a unique and insufficiently examined gap in the literature by developing an Arabic-centric legal analytics solution designed for the needs of public sector auditors in Qatar. This research illustrates the capacity of Arabic NLP to adhere to international standards while addressing jurisdiction-specific requirements to advance Qatar National Vision 2030, thereby enhancing local AI capabilities for governmental applications requiring precision, accountability, and transparency.

Date of Award	2025
Original language	American English
Awarding Institution	HBKU College of Science and Engineering