Resource-Aware Semantic Communication with Adaptive Vision Transformers for Digital Twins

In the Metaverse, real-time digital twin (DT) updates enable immersive, interactive environments by reflecting real world states. Metaverse can engage its users to share data in order to ensure the completeness of the DT. However, the limitations and heterogeneity in IoT devices' computation and transmission resources are critical challenges to synchronizing the vast volume of real-world objects with their digital replicas. In this work, we propose a novel adaptive Vision Transformer (ViT)- based semantic communication (SemCom) system to extract semantic information from raw data collected by IoT devices. The system dynamically adjusts the model size and computational complexity according to the resource constraints of individual IoT devices, enables broader participation of heterogeneous devices, enhances feature extraction capabilities, and ensures completeness and efficient DT representation. We formulate our problem as an utility-maximization optimization to manage ViT complexity under resource constraints, allowing the Metaverse Services Provider (MSP) to select high-performing IoT devices. To guide the optimization, we profile ViT models with varying architectural complexities and conduct an empirical analysis to capture the relationship between model scale and performance. To address the scalability and privacy limitations of the optimization, we propose a decentralized solution where each IoT device independently optimizes its own utility under local constraints. The MSP, in turn, selects among these devices to ensure quality and maximize its overall utility. We show that our distributed solution achieves near-optimal performance and significantly outperforms other approaches in terms of MSP utility, semantic data quality, and overall IoT utility.