Glossary
In this guide, there are many terms used. They are listed here for easier reference.
| Term | Description | |
|---|---|---|
| Synergos | Synergos is an open source Federated Learning platform that AI Singapore has been building. This is a Greek word, from which the English word “Synergy” was derived. It means “to work together” or “to cooperate”, which is the very gist of the vision that Federated Learning promises. | |
| Centralised training | Data is at the core of machine learning. Nevertheless, in many real-world projects, a single party’s data is often not sufficient and needs to be augmented with data from other parties. Typically data from multiple data are pooled together into a centralised location, where data is analysed and used to build machine learning model. Training a model with data pooled into a centralised location is called Centralised training, to differentiate from what Federated Learning is doing. | |
| Federated learning | Federated Learning (FL for short) is an emerging privacy-preserving machine learning (PPML) technology. It enables multile parties to jointly train a model together without exposing data to one another. One of the main issues of FL is that the data from different parties may not be IID (independently and identically distributed). Active research is going on about how to aggregate learning outcome from different parties to deal with non-IID data. | |
| Federated training | Federated training is often used interchangeably with Federated Learning. Nevertheless, in cases that a differentiation is needed, federated training usually means the model training process done with Federated Learning. | |
| Federated model | A machine learning trained with Federated Learning | |
| Blocks | As described here, Synergos has multiple functional blocks. There are currently nine blocks, namely, Federation, Federated Grid, Compute & Storage, Orchestration, Model Lifecycle Management, Model Serving, Meta-data Management, Contribution & Reward, and Dashboard. Each block is implemented as and supported by a number of components. Each block will be described in the corresponding entry in this glossary. | |
| Block - Compute & Storage | Compute & Storage acts as an interface to different compute and storage backends. Every local party has its own compute and storage backend, which others do not have direct access to. | |
| Block - Contribution & Reward | Usually, different parties incur non-negligible costs in acquiring and cleaning their data. They rarely altruistically share their data with others and risk losing their competitive edge. These parties would be more motivated to share their data when given enough incentives, such as a guaranteed benefit from the collaboration and a fair higher reward from contributing more valuable data. Otherwise, without any party motivated to contribute data, it could be detrimental to the sustainability of Federated Learning. Contribution & Reward is to evaluate contributions and reward different parties fairly based on their contributions, so that we can make Federated Learning more sustainable. |
|
| Block - Dashboard | Dashboard is the place for different parties to interact with Synergos platform and build federated models with a GUI. It also provides a one-stop view of all the information generated by the different functional blocks. | |
| Block - Federation and Federated Grid | The core of Synergos is its Federation block. Here is where the coordination among different parties to train a federated model (without exposing data) happens. The Federation block defines the application level protocol over WebSocket to form a Federated Grid. A federated grid is a star-architecture network formed by different parties, who exchange messages among themselves to coordinate the model training and inference. | |
| Block - Meta-data Management | Although different parties do not expose data to one another, they still need to "declare" what data and what compute resource hey are contributing. Those are "meta-data" need to be managed. We also envision that when the Federated Learnign ecosystem further grows, a open marketplace will be formed with multiple parties "advertising" the data they have. They will need to “register” their data to the marketplace's data catalog system so that others could search and discover what data are available. Meta-data Management acts as an interface to the data catalog system. |
|
| Block - Model Lifecycle Management | Usually, multiple models are usually built for the same problem statement and one of them will be selected to be served in production based on technical and business performance criteria. Model Lifecycle Management is responsible for tracking the running of multiple models trainings to record and compare results. It also serves as a model registry to manage the lifecycle of a federated model, including model versioning and stage transitions. | |
| Block - Model Serving | Model Serving makes sure the model is up and running and is able to receive inference requests from the users. | |
| Block - Orchestration | In Synergos, every federated learning job (either train or inference) happens within a federated grid. A federated grid is typically destroyed when a job is finished. There are scenarios that multiple federated grids are needed, e.g. when doing hyper-parameter tuning. Orchestration block manages and orchestrate different grids to complete the job. | |
| Synergos Driver | Synergos Driver is a Python package which enables different parties to interact with Synergos and build federated models programmatically. Depending on the configutation used, orchestrator uses Synergos Driver to interact mainly with Director or TTP. | |
| Component - Synergos Director | Synergos Director is a component inside the Orchestration block, which is able to orchestrate multiple federated grids running simultaneously. There are cases where multiple federated grids are running in parallel (e.g., hyper-parameter tuning in Synergos Cluster configuration). In such cases, the orchestrator instantiates one instance of Synergos Director in its compute resource. In such cases, the orchestrator is acting as a director, who orchestrates with the multiple TTPs, each of who subsequently coordinates with multiple workers within respective grids. | |
| Component - Synergos TTP | Synergos TTP is a component inside the Federation block, which sends instructions to Synergos Worker. To orchestate the federated learning process, the orchestrator instantiates on instance of Synergos TTP in its compute resource. In this case, the orchestrator is acting as a TTP. One TTP and multiple workers form a federated grid to complete the federated learning. There are also possible scenarios (e.g. in Synergos Cluster configuration) that the orchestrator may instantiates multiple instance of Synergos TTP in its compute resources. In such scenarios, a director is needed to orchestrate multiple TTP. | |
| Component - Synergos Worker | Synergos Worker is a component inside the Federation block, which excutes instructions received from Synergos TTP to conduct the local training. Each participant, who is contributing data, instantiates one instance of Synergos Worker component in its compute resource. In scenarios where there is no confusion, participant and worker are used interchangeably. There are also possible scenarios (e.g. in Synergos Cluster configuration) that each participant may instantiates multiple instance of Synergos Worker in its compute resources. In such scenarios, a participant is said to have multiple workers. | |
| Configuration - Synergos Basic | When Synergos is running with only the core components (those in the Federation and Federated Grid block), this configuration is called Synergos Basic. In this configuration, there is only one federated grid running. | |
| Configuration - Synergos Plus | When Synergos is running with the core components together with Synergos Logger and Synergos MLFlow, this configuration is called Synergos Plus. In this configuration, there is only one federated grid running. Synergos Logger captures the logs generated by different components. Synergos MLFlow is inside the Model Lifecycle Management block. | |
| Configuration - Synergos Cluster | When Synergos is running in Synergos Cluster configuration, besides those components used in Synergos Plus, Synergos MQ and Synergos Director are also used. Synergos MQ and Synergos Director are inside Orchestration block. They work together to coordinate multiple running federated grids | |
| Meta-data - Collaboration | In the Registration phase, both orchestrator and participants need to declare necessary meta-data. In this phase, orchestrator first needs to declare a collaboration. A collaboration defines a coalition of parties agreeing to work together for a common goal (or problem statement). For example, when multiple banks decide to work together to build an anti-money laundering solution, they form a collaboration. | |
| Meta-data - Project | Project is the next information orchestrator needs to provide after collaboration. Within a collaboration, there may be multiple projects. Each project corresponds to a collection of data different parties in the collaboration use. If there is any change in the data they are using, a new project should be defined. For example, those banks in the collaboration agree that each of them would use a set of local tabular data with a common schema. If there is any change to the agreed data (e.g. change in the common schema or some parties decided to use differnet rows of data), a new project will be defined. | |
| Meta-data - Experiment | Experiment is the next information orchestrator needs to provide after project. Under a project, there will be multiple experiments, each of them corresponds to one particular type of model to be trained, e.g. logistic regression, neural network, etc. | |
| Meta-data - Run | Run is the next information orchestrator needs to provide after experiment. There are multiple runs under each experiment, each of them uses a different set of hyper-parameters. | |
| Orchestrator | This is to be differentiated from Block - Orchestration. Orchestrator is one category of user of Synergos. Orchestrators are one type of users in Synergos. They do not contribute data. They coordinate with other participants to complete Federated Learning, e.g. they start the federated training and decide when to stop the training. Depending on the configuration run, diffent compnents in Synergos will be used to support orchestrator, e.g. Synergos Director or Synergos TTP. | |
| Participant | Participant is another category of user of Synergos, who joins an existing collaboration, and contributes compute and data in the federated learning. They run Synergos Worker component. In scenarios where there is no confusion, participant and worker are used interchangeably. | |
| Party | Party is used interchangeably with user in Synergos. A party can take the role of either orchestrator or participant. | |
| Phase - Registration | There are three phases in a federated learning cycle. Registration is the first phase. In this phase, all the parties (including orchestrator and participants) declare necessary meta-data, e.g., network connection information, data contributed by participants, etc. | |
| Phase - Training | There are three phases in a federated learning cycle. Training is the second phase. In this phase, the federated training is happening either in one federated grid or multiple grids, depending on the configuration of Synergos that is used. | |
| Phase - Evaluation | There are three phases in a federated learning cycle. Evaluation is the third phase. In this phase, the trained federated model is being evaluated for its performance with declared validation data. The participants can also submit new data for inferencing. |