Per-Attribute Privacy in Large Language Models Using Matrix-Variate Gaussian Mechanism

Large Language Models (LLMs) have become central to modern NLP applications, yet their reliance on sensitive training data introduces significant privacy risks. Traditional approaches to differential privacy, such as DP-SGD, apply uniform noise at the gradient level and treat all features equally, ignoring the inherent correlations in structured sequence data. In this work, we propose a novel forward-pass privacy mechanism that applies per-attribute differential privacy across correlated sequence inputs. Our framework leverages the Matrix-Variate Gaussian (MVG) mechanism to inject structured, directional noise during the forward computation, enabling fine-grained privacy control that aligns with attribute sensitivity. Privacy budgets are assigned on a per-token basis using an Inverse Gaussian Distribution, allowing position-aware adaptation across input sequences. To propagate these budgets through the model, we introduce a data-independent Layer-Wise Contribution Propagation (LCP) algorithm that maps input sensitivity to output features, even in transformer architectures. We validate our method on the SST-2 sentiment classification benchmark, demonstrating improved utility over existing approaches such as DPSGD and DP-Forward, particularly under strict privacy regimes. Our results highlight the benefits of structured noise in preserving utility while ensuring strong, attribute-level privacy in models trained on correlated sequential data.