Federated Learning

CORE DEFINITION

Federated learning is a machine learning technique in which algorithms are trained across multiple decentralized edge devices or servers that hold local data samples. This approach differs significantly from traditional centralized machine learning techniques, which upload all local datasets to a single server, and from more classical distributed methods, which typically assume that local data samples are identically distributed. Federated learning enables multiple participants to build a common, robust machine learning model without sharing data, thereby addressing critical issues such as data privacy, data security, data access rights, and access to heterogeneous data. Federated Averaging (FedAvg) is a generalization of FedSGD that allows local nodes to perform multiple batch updates on local data and exchange updated weights rather than gradients. Moreover, averaging adjusted weights from the same initialization does not necessarily harm the performance of the resulting averaged model. Despite its simplicity, it lacks theoretical guarantees in real-world settings.