Quality-Aware Dynamic Client-Rank Selection for Resource-Constrained Federated LoRA

Federated fine-tuning of Large Language Models (LLMs) enables collaborative learning across distributed clients without centralizing private data. While Low-Rank Adaptation (LoRA) reduces computational overhead through parameter-efficient training, existing federated LoRA approaches assign fixed ranks throughout the training process, limiting personalization and failing to adapt to evolving client performance and changing resource availability. Moreover, these methods lack mechanisms that quantify client contributions in heterogeneous settings where devices operate at different LoRA ranks. This paper introduces an iterative federated learning framework that dynamically optimizes both client selection and rank allocation during the training through a feedback-driven process. We propose a combined quality assessment metric that integrates three weighted components: data distribution through sample weights, theoretical contribution through client-rank Shapley values, and empirical performance derived from online observation phases. At each iteration, our framework optimizes client selection and rank assignments under memory, latency, and energy constraints, conducts training rounds with quality-aware aggregation, and updates quality assessments through an observation phase before re-optimizing for the next iteration. The metric weights adapt automatically when training performance degrades, cycling through emphasis on data-driven, contribution-based, or performance-based strategies. This creates a closed-loop system where optimization decisions continuously reflect both resource constraints and actual training effectiveness. Experimental results demonstrate that our iterative approach achieves superior model accuracy compared to federated LoRA baselines and approaches while efficiently managing heterogeneous client resources.