FedDecay: Balancing Model Performance and Rapid Personalization in Federated Learning with Learning Rate Scheduling
研究了在联邦学习中，通过在每个本地训练轮次内使用学习率衰减，无需额外内存和计算即可提升模型在微调后的性能，效果接近个性化联邦学习方法。
Federated learning (FL) is a powerful technique for collaboratively training a single centralized model on distributed local data sources. By aggregating model information without disclosing the local training data, FL preserves data privacy. However, the inherent heterogeneity in local data sets challenges the performance of FL techniques, especially when data is very diverse across local sources. Personalized Federated Learning (PFL) can mitigate this challenge using multiple models but often requires additional memory and computation. This work does not propose a new PFL method but instead explores how learning rate decay within each local training round can improve model performance across local data sets after fine-tuning. We provide theoretical insights and empirical evidence of efficacy across diverse domains, including vision, text, and graph data. Our extensive experiments demonstrate that learning rate scheduling alone outperforms other FL methods regarding generalization to new data for both new and existing users. Moreover, it performs comparably to PFL methods, particularly for new users, while maintaining similar computational and memory requirements as FL techniques. Supplementary materials, including all code and instructions for reproducing the experiments, are available online.