Toward Delay-Conscious Federated Learning: A Systems Perspective

Federated Learning (FL) was introduced to mitigate centralized Machine Learning (ML) data privacy legislation, which restricts the diversity and performance of ML applications. FL allows training data on its source device, and only model updates can leave the device for learning aggregation at the Cloud/Edge. Despite FL advantages, it is challenging for it to realize a training performance that is comparable to traditional ML, especially in terms of training speed. In this article, we first introduce an original analysis of the systems factors contributing to FL training speed. We then define current FL limitations associated with these factors, propose novel and efficient solution strategies to over-come these limitations, and assess the impact of these strategies through preliminary evaluation. Finally, We identify challenges and future research directions that need to be addressed to realize the benefits of pushing FL limits.