Quantum Fuzzy Neural Network for Malware Detection

Abstract

The rapid evolution of malware techniques makes conventional defenses increasingly ineffective. Although classical machine learning systems improve detection, they struggle when attempting to classify malware. Approaches that integrate fuzzy logic into machine learning models, specifically neural networks, alleviate this problem. However, this solution still exhibits frequent instability in the classification process. In this thesis, we introduce a novel Quantum Deep Fuzzy Neural Network (QDFNN) that combines quantum machine learning and fuzzy logic layer, enabling probabilistic reasoning through superposition and conditional activation gates to capture complex malware patterns. We derive multiple variants of the model and evaluate them on a popular Malware Detection dataset. The fully quantum variant (QDFNN) achieves an accuracy of 93.6%, outperforming several baseline models, including a hybrid variant, H-QDFNN, a shallow variant of QDFNN (QFNN), a shallow hybrid variant, H-QFNN, a quantum non-fuzzy variant, QNN, and several classical models, such as ANN, SVM, and logistic regression. We also extend our evaluation to include other important evaluation metrics, such as precision, recall, F1 score, and ROC-AUC, for which our proposed QDFNN outperformed all other baseline models. Finally, we evaluated the resilience of the proposed model against quantum noise by adopting six NISQ noise models and comparing our model with its variants, where we observed reasonable accuracy levels of the proposed model. This thesis demonstrates that integrating quantum fuzzy logic yields a potential competitive advantage, especially against conventional classical models, paving the way toward practical quantum-based malware detection systems.

Date of Award	2025
Original language	American English
Awarding Institution	HBKU College of Science and Engineering