DEEP LEARNING IN THE DCT-DOMAIN FOR MULTIVARIATE TIME-SERIES CLASSIFICATION

Abstract

Time Series Classification (TSC) is a critical task in many applications. Yet, it remains challenging due to the inherent complexities of sequential data, such as temporal dependencies, interchannel correlations, and non-stationarity. Parametric models like ARIMA and Hidden Markov Models are often used to extract features, such as model coefficients or transition probabilities, which are then used as input to a classifier. However, these models can struggle with high dimensionality and typically rely on assumptions of stationarity and linearity. Meanwhile, hand-crafted feature extraction methods require domain expertise and are computationally expensive and time-consuming. Deep learning models address these shortcomings by learning hierarchical feature representations directly from the raw time series, thereby reducing the need for manual feature design and often achieving superior performance on complex tasks. However, deep learning models lack domain-awareness, interpretability, and robustness to varying signal characteristics. This dissertation aims to address the limitations of current time series classification models by combining signal processing techniques with data-driven deep learning methods. Our investigation progressed through three main phases. Initially, we proposed a vector autoregression (VAR) framework that couples sliding window sampling with VAR coefficients to model temporal and interchannel dependencies. This approach resulted in interpretable features and competitive accuracy, particularly on EEG benchmarks. However, the susceptibility of the VAR to phase shift, which can lead to inconsistent feature extraction, and poor computational scalability with increasing time series length and dimensionality motivated a shift towards spectral domain transformations. In particular, we proposed AttDCT, a deep learning architecture that operates on DCT-transformed signals, with spectral enhancement and self-attention mechanisms. The time series is first segmented, and each segment is rearranged into a sliding window matrix. A two-dimensional discrete cosine transform converts this matrix to the spectral domain, where logarithmic contrast enhancement and spectral normalization refine the coefficients. The resulting representation is fed to a lightweight CNN classifier comprising a patch-wise self-attention block and a multilayer CNN. AttDCT achieved state-of-the-art performance in 13 TSC datasets, with a mean accuracy improvement of 2.1\% over existing models. However, AttDCT does not explicitly model the channel interdependencies between the CNN's feature maps. To address this, we introduce a DCT-based Channel Attention (DCA) mechanism that refines the standard channel attention paradigm by replacing global average pooling with DCT coefficients to capture a broader spectrum of temporal features. DCA improves channel-wise feature recalibration by selecting informative low-frequency DCT coefficients or by retaining the highest coefficients based on their spectral energy. Comprehensive experiments on twelve diverse TSC datasets demonstrate that DCA consistently outperforms state-of-the-art channel attention mechanisms, achieving an average improvement of 2.2\% in classification accuracy. The proposed VAR, AttDCT, and DCA frameworks show how carefully embedding signal processing techniques into deep learning models can close the accuracy-interpretability gap in TSC while retaining computational efficiency and robustness.

Date of Award	2025
Original language	American English
Awarding Institution	HBKU College of Science and Engineering