Deep-Learning-Based Cross-Layer Fingerprinting for Secure and Trustworthy Communication Systems

Abstract

Networks and communication systems serve as the foundational backbone of modern technological infrastructure, with billions of devices increasingly being connected. However, this rapid surge in connectivity and the proliferation of deployed devices have introduced an enormous number of security threats affecting all layers of the communication stack. Unfortunately, when such threats occur, they can disrupt the entire communication process—compromising the medium (e.g., through jamming), device identity (e.g., via spoofing), or active services (e.g., model substitution in cloud-based APIs). Traditional security techniques primarily rely on computational cryptographic methods and typically assume the availability of abundant computational resources. However, these assumptions often do not hold true in real-world scenarios, especially in resource-constrained environments. In this dissertation, we propose a set of novel cross-layer fingerprinting techniques based on Deep Learning (DL) and the extraction of intrinsic, immutable features across the physical and network layers for early jamming detection, robust device authentication, and LLM identification service. First, we present a publicly released, comprehensive dataset of indoor physical-layer wireless measurements collected under a variety of jamming scenarios. Building upon this dataset, we introduce BloodHound, a DL system for early detection and identification of jamming attacks. BloodHound leverages the shape and distortion of in-phase and quadrature (I/Q) samples at the physical layer to accurately detect and classify jamming threats. Next, we systematically investigate the end-to-end challenges that prevent the practical deployment of DL-based radio frequency fingerprinting (RFF) systems. We identify key limitations related to the development lifecycle. Furthermore, we reveal an oftenoverlooked factor—device power cycling—which significantly degrades RFF performance over time. While typically attributed to wireless channel variability, we show that power cycling introduces temporal inconsistencies that affect classification reliability. To mitigate this, we propose a data preprocessing strategy that enhances both robustness and accuracy in dynamic real-world environments. Finally, we extend our fingerprinting approach to the network layer by addressing the emerging issue of LLM service trustworthiness in remote black-box API deployments. We propose a novel, real-time passive fingerprinting technique that leverages encrypted traffic patterns to identify the specific LLM in use. Our technique enables the verification of model identity without requiring access to internal model logic or outputs. Overall, this dissertation introduces an integrated, cross-layer framework that advances the security and trustworthiness of communication systems—from the physical layer to application-level services—by leveraging deep learning and intrinsic feature-based fingerprinting techniques.

Date of Award	2025
Original language	American English
Awarding Institution	HBKU College of Science and Engineering