Installation Guide

Different components in Synergos

As mentioned here, each block is supported by multiple components. The table below shows a mapping of key blocks and their supporting components.

Depending on the configuration of Synergos, different components are activated and used to support the running of Synergos. For example, in Synergos Basic, only the core components in Federation and Federated Grid and Synergos Driver are activated and used to support the running of Synergos.

Synergos Driver is a python package which enables interaction with Synergos programmatically.

Further in this installation guide, we share how to install Synergos from pre-built images or from source

Installing Synergos with pre-built docker images

The easiest way to install Synergos is to deploy with pre-built docker images.

The core components are essential ones to support most of the use cases. To extend the functionality and scalability of Synergos, the core components can be complemented with extended components, such as Synergos MQ and Synergos Director.

Installing the core components

1. Pull and re-tag the Synergos TTP image with the following command:

   # Pull image
   docker pull gcr.io/synergos-aisg/synergos_ttp:v0.1.0
   docker tag gcr.io/synergos-aisg/synergos_ttp:v0.1.0 synergos_ttp:v0.1.0
   

2. Pull and re-tag the Synergos Worker image with the following command:

   # Pull image
   docker pull gcr.io/synergos-aisg/synergos_worker:v0.1.0
   docker tag gcr.io/synergos-aisg/synergos_worker:v0.1.0 synergos_worker:v0.1.0
   

Installing the extended components

If you want to use additional features of Synergos, e.g. hyper-parameter tuning, etc., pull these images below to augment the core components.

1. Pull and re-tag the Synergos TTP image with the following command:

   # Pull image
   docker pull gcr.io/synergos-aisg/synergos_ttp_cluster:v0.1.0
   # Tag image
   docker tag gcr.io/synergos-aisg/synergos_ttp_cluster:v0.1.0 synergos_ttp_cluster:v0.1.0
   

   You may notice that the Synergos TTP image here is different from the one used when installing the core components. This is because Synergos TTP uses a different mechanism when working together with Synergos Director. Conceptually they are the same component, though different docker images have been created to build-in this difference.

2. Pull and re-tag the Synergos Director image with the following command:

   # Pull image
   docker pull gcr.io/synergos-aisg/synergos_director:v0.1.0
   # Tag image
   docker tag gcr.io/synergos-aisg/synergos_director:v0.1.0 synergos_director:v0.1.0
   

3. Pull the Synergos MQ image with the following command:

   # Pull image
   docker pull gcr.io/synergos-aisg/synergos_mq:v0.1.0
   # Tag image
   docker tag gcr.io/synergos-aisg/synergos_mq:v0.1.0 synergos_mq:v0.1.0
   

4. Pull the Synergos Federated MLOps image, which is a custom configured MLFlow image with the following command:

   # Pull image
   docker pull gcr.io/synergos-aisg/synergos_mlops:v0.1.0
   # Tag image
   docker tag gcr.io/synergos-aisg/synergos_mlops:v0.1.0 synergos_mlops:v0.1.0
   

5. Synergos Logger is formed by a network of inter-dependent subset of containers, therefore you are required to run docker compose to set up and run it. You will have to download the latest code for Synergos Logger and set it up with the following command:

   # Clone source repository
   git clone https://github.com/aimakerspace/synergos_logger.git
   
   # Navigate into the repository
   cd ./synergos_logger
   
   # Checkout to stable tag
   git checkout tags/v0.1.0
   
   # Set up and run network of dependent containers
   docker-compose up --build
   

6. Synergos Dashboard UI is formed by a network of inter-dependent subset of containers, therefore you are required to run docker compose to set up and run it. You will have to download the latest code for Synergos Dashboard UI, pull the requisite Docker images, and set it up with the following command:

   # Clone source repository
   git clone https://github.com/aimakerspace/synergos_ui.git
   
   # Navigate into the repository
   cd ./synergos_ui
   
   # Checkout to stable tag
   git checkout tags/v0.1.0
   
   # Pull images
   docker pull gcr.io/synergos-aisg/synergos_ui_track:v0.1.0
   docker pull gcr.io/synergos-aisg/synergos_ui_view:v0.1.0
   docker pull gcr.io/synergos-aisg/synergos_ui_nav:v0.1.0
   
   # Tag images
   docker tag gcr.io/synergos-aisg/synergos_ui_track:v0.1.0 synergos_ui:track
   docker tag gcr.io/synergos-aisg/synergos_ui_view:v0.1.0 synergos_ui:view
   docker tag gcr.io/synergos-aisg/synergos_ui_nav:v0.1.0 synergos_ui:nav
   
   # Set up and run network of dependent containers
   docker-compose up
   

Installing and building the containers from source

You can also install Synergos and build new docker images yourself from the source. This section describes the steps to build and install the core components of Synergos from source. Building other components follows similar steps.

Checkout the latest code

Before building the docker containers, you have to ensure that you have the necessary source files. Let's start by downloading the source code and setting up the virtual environment.

Core Component -Synergos Worker

# Clone source repository
git clone https://github.com/aimakerspace/synergos_worker.git

# Navigate into the repository
cd ./synergos_worker

# Checkout to stable tag
git checkout tags/v0.1.0

# Update Submodule
git submodule update --init --recursive
git submodule update --recursive --remote

# Build image
docker build -t synergos_worker:v0.1.0 --label "WebsocketServerWorker" .

Core Component - Synergos TTP

Depending on the configuration of Synergos that you are running, you need to include the --target flag, with the suitable argument value, in your build command.

# Clone source repository
git clone https://github.com/aimakerspace/synergos_ttp.git

# Navigate into the repository
cd ./synergos_ttp

# Checkout to stable tag
git checkout tags/v0.1.0

# Update Submodule
git submodule update --init --recursive
git submodule update --recursive --remote

# Build image - Synergos Basic Variant
docker build --target basic_ttp -t synergos_ttp:v0.1.0 --label "WebsocketClientWorker" .

# Build image - Synergos Cluster Variant
docker build --target syncluster_ttp -t synergos_ttp_cluster:v0.1.0 --label "WebsocketClientWorker" .

Extended Component - Synergos Director

# Clone source repository
git clone https://github.com/aimakerspace/synergos_director.git

# Navigate into the repository
cd ./synergos_director

# Checkout to stable tag
git checkout tags/v0.1.0

# Update Submodule
git submodule update --init --recursive
git submodule update --recursive --remote

# Build image
docker build -t synergos_director:v0.1.0 .

Extended Component - Synergos MQ

# Clone source repository
git clone https://github.com/aimakerspace/synergos_manager.git

# Navigate into the repository
cd ./synergos_manager

# Checkout to stable tag
git checkout tags/v0.1.0

# Build image
docker build -t synergos_mq:v0.1.0 .

Extended Component - Synergos Federated MLOps

# Clone source repository
git clone https://github.com/aimakerspace/synergos_mlops.git


# Navigate into the repository
cd ./synergos_mlops

# Checkout to stable tag
git checkout tags/v0.1.0

# Build image
docker build -t synergos_mlops:v0.1.0 .

Extended Component - Synergos Logger

# Clone source repository
git clone https://github.com/aimakerspace/synergos_logger.git

# Navigate into the repository
cd ./synergos_logger

# Checkout to stable tag
git checkout tags/v0.1.0

# Set up and run network of dependent containers
docker-compose up --build

Extended Component - Synergos Dashboard UI

# Clone source repository
git clone https://github.com/aimakerspace/synergos_ui.git

# Navigate into the repository
cd ./synergos_ui

# Checkout to stable tag
git checkout tags/v0.1.0

# Update Submodule
git submodule update --init --recursive
git submodule update --recursive --remote

# Set up and run network of dependent containers
docker-compose up --build

Installing Synergos Driver

Synergos Driver is a python package which enables interaction with Synergos programmatically. Install this package separately regardless whether you install other components from pre-built image or source.

Create a new Python (3.7 and above) environment and install the Synergos Driver package.

# Clone source repository
git clone https://github.com/aimakerspace/Synergos.git

# Navigate into the repository
cd ./synergos

# Checkout to stable tag
git checkout tags/v0.1.0

# Setup virtual environment
conda env create --name <synergos_env> --file=environment.yml

# Install in development mode
pip install -e .