Shapley-Based Client and LoRA Rank Selection for Heterogeneous Federated LLM Fine-Tuning

With the growing adoption of Large Language Models (LLMs), federated fine-tuning has emerged as a promising approach for leveraging distributed client data while preserving privacy. However, real-world clients have heterogeneous and often limited capabilities, hindering participation in full-scale model updates. Low-Rank Adaptation (LoRA) offers a potential solution by reducing the number of trainable parameters through low-rank matrix insertion. Existing federated LoRA methods either enforce homogeneous ranks that underutilize high-capability clients, or use static rank assignments across rounds, thus limiting personalization. Critically, these approaches also fail to capture client contributions among ranks and overlook synergistic effects between them. In this paper, we propose a parameter-efficient federated fine-tuning framework for LLMs using dynamic and resource-aware heterogeneous LoRA ranks. We introduce a novel lightweight Shapley-based method for contribution quantification across varying ranks and a hierarchical synergy estimation technique to capture collaborative effects. These enable the formulation of an optimization that jointly selects clients and assigns LoRA ranks aiming to enhance aggregation and training performance while minimizing latency under dynamic resource constraints. Our approach demonstrates near-optimal performance compared to the standard Shapley and better performance compared to federated LoRA approaches and baselines.