Optimizing Smart Cities Applications with Multimodal Deep Learning and Adaptive Large Language Models

Abstract

Modern smart cities increasingly rely on AI systems deployed at the edge for environmental monitoring, transportation analytics, and intelligent urban services. These systems face three interconnected challenges: limited sensing infrastructure, high computational cost of optimizing deep learning models on resource-constrained devices, and the unreliability of Large Language Models caused by hallucinations. This dissertation addresses all three through a unified, progressive framework in which each contribution is directly motivated by the limitations of the previous. The first contribution is a multimodal Air Quality Index forecasting system that repurposes existing CCTV infrastructure, eliminating dense sensor deployment. A YOLOv8-based pipeline extracts vehicle counts from camera feeds, fused with weather and sensor data through a merit-based scheme feeding an LSTM model. Experiments demonstrate accuracy comparable to physical monitoring stations across short, medium, and long-term horizons. To further enhance the deep learning model at the core of this system, the second contribution focuses on optimizing its architecture more efficiently through an intelligent hyperparameter search strategy. The second contribution addresses hyperparameter tuning costs for edge-deployed models. A novel LLM-enhanced Particle Swarm Optimization framework selectively replaces underperforming particles with LLM-proposed candidates, reducing model evaluations by 20–60\% without sacrificing accuracy across benchmark optimization, regression, and classification tasks. To further enhance the reliability of the LLM responses driving this optimization framework, the third contribution focuses on mitigating hallucinations in real time through a principled budget-aware retrieval strategy. The third contribution develops the first online budget-constrained RAG allocation algorithm for hallucination mitigation, using semantic entropy to decide in real time. Evaluated across TriviaQA, Natural Questions, and SVAMP, the method achieves 91\% of offline-optimal performance with equivalent computational savings and exhibits emergent difficulty-aware allocation without task-specific supervision. Together, these contributions form a progressive enhancement framework, advancing from reliable environmental perception, through efficient model optimization, to trustworthy LLM-driven intelligence, enabling scalable AI deployment in smart city edge environments.

Date of Award	2026
Original language	American English
Awarding Institution	HBKU College of Science and Engineering