Skip to content

londonaicentre/FLIP

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

937 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

flip-logo

Federated Learning and Interoperability Platform

License Python 3.12+ Documentation Status Coverage

flip-api flip-ui

data-access-api imaging-api trust-api

orthanc xnat-db xnat-nginx xnat-web

FLIP is an open-source platform for federated training and evaluation of medical imaging AI models across healthcare institutions, while ensuring data privacy and security.

FLIP is developed by the London AI Centre in collaboration with Guy's and St Thomas' NHS Foundation Trust and King's College London.

flip-architecture

Repositories

FLIP spans several repositories:

Repository Description
FLIP This repo: Central Hub API, Trust APIs, UI, and Docker deployment
flip-fl-base NVIDIA FLARE federated learning base application library, workflows, and tutorials
flip-fl-base-flower Flower federated learning base application library, workflows, and tutorials

This repository consolidates all FLIP services in a mono-repo that can be deployed together via Docker Compose. The federated learning images are pulled from flip-fl-base and flip-fl-base-flower.

Deployment

Prerequisites

For developer tooling and IDE setup, see CONTRIBUTING.md.

Using the Makefile

To start the services, you can use the Makefile provided in the root directory. The Makefile provides several convenient commands to manage the services defined in the deploy/compose.development.yml file.

For example:

Command Description
make up Run all services using Docker Swarm for XNAT (⚠️ This will not build the images, use make build first if needed)
make up-no-trust Run all services except the trust services related services
make up-trusts Run the trust services related services (uses Docker Swarm for XNAT)
make central-hub Run the central API service and the database (does not start the UI — use make ui for that)
make ui Start the flip-ui container only (no-op when PROD=stag/PROD=true, since the UI is served from S3 + CloudFront)
make ui-off Stop the flip-ui container
make build Build all Docker images
make down Stop all services and remove the containers (including Swarm stacks)
make restart Stop and start all services
make restart-no-trust Stop and start all services except the trust services related services
make clean Remove all stopped containers, networks, and images
make ci Run the CI pipeline locally using act
make up-local-trust Run a local (on-premises) trust (set PROD=true or PROD=stag for environment)
make unit_test Run unit tests across all services
make tests Run flip-ui unit tests and the full flip-api test suite (lint + mypy + pytest)
make debug SERVICE=<name> Restart one service in debug mode (waits for a debugger on port 5678). Services: flip-api, fl-api-net-1, trust-api, imaging-api, data-access-api
make debug-off SERVICE=<name> Take a single service back out of debug mode
make debug-all Restart every API service in debug mode
make debug-off-all Take every API service back out of debug mode

You can add new commands to the Makefile to create smaller deployments for testing and development.

Docker Swarm Deployment

The XNAT services are deployed using Docker Swarm mode for better resource management and scalability. Docker Swarm is automatically used when running make up or make up-trusts.

Key features of Swarm deployment:

  • Better resource allocation with CPU and memory limits
  • Automatic service recovery with restart policies
  • Overlay networking for secure service communication
  • Support for multi-node deployment (if configured)

Swarm-specific commands:

  • XNAT services are deployed as Docker stacks (xnat1 and xnat2)
  • The Swarm deployment uses the trust/xnat/docker-compose-stack.yml file
  • Networks are created as overlay networks with --attachable flag for flexibility

Note: Docker Swarm mode must be initialized on your system. If not already initialized, run:

docker swarm init

After that, you will need to restart the docker networks used by the services:

there is a command to create the networks, but you will need to remove them manually first if they are already running:

docker network rm deploy_trust-network-1
docker network rm deploy_trust-network-2

Then create the networks again:

make create-networks

To manually manage XNAT services (uses Docker Swarm):

cd trust/xnat
make up          # Start XNAT services
make down        # Stop XNAT services
make xnat-shell  # Get a shell in the XNAT container

Trust API Key Setup

Before starting the platform, generate per-trust API keys and the internal service key, and write them into .env.development using:

make generate-trust-api-keys
make generate-internal-service-key

generate-trust-api-keys generates a unique key for each trust found in .env.development, and writes both TRUST_API_KEYS and TRUST_API_KEY_HASHES (JSON dicts) directly into the env file. generate-internal-service-key writes INTERNAL_SERVICE_KEY and INTERNAL_SERVICE_KEY_HASH for fl-server-to-hub authentication. make up invokes generate-internal-service-key automatically.

To generate a key for a single trust (e.g. when adding a new trust):

make -C flip-api generate-trust-key TRUST_NAME=Trust_1

Basic Usage

To start the full platform locally:

make up

This will start all the services defined in the deploy/compose.development.yml file. The services will be started in detached mode, so you can continue using your terminal. Use docker compose ps to see the status of the services and see which ports they are running on.

To get a shell some of the services, you can run:

docker compose -f deploy/compose.development.yml exec < service-name > < command >

For example:

docker compose -f deploy/compose.development.yml exec flip-ui /bin/sh

This will give you a shell in the flip-ui container. You can run any command inside the container, including installing new packages, running tests, and debugging the code.

To stop the services:

make down

If you want to run a single service you can run:

docker compose -f deploy/compose.development.yml run --rm < service name >

Federated Learning Setup

The project supports NVIDIA FLARE and Flower Framework for federated learning. FLARE requires provisioned certificates and configuration files that are generated in the separate repository flip-fl-base (see that repository for instructions on how to provision the workspace).

  1. Path Resolution: While .env.development defines FL_PROVISIONED_DIR as a relative path (../flip-fl-base/workspace), the Makefile automatically converts this to an absolute path using:

    override FL_PROVISIONED_DIR := $(shell realpath $(dir $(lastword $(MAKEFILE_LIST)))/../flip-fl-base/workspace)

    This ensures Docker volume mounts work correctly (Docker requires absolute paths) while maintaining portability across different machines.

  2. Why This Matters: Docker Compose cannot resolve relative paths for volume mounts, so the absolute path conversion is essential for FL services to access their provisioned certificates and configuration files.

If you see errors like "fed_client.json does not exist" or "missing startup folder", verify that:

  • The flip-fl-base repository is cloned as a sibling directory
  • The workspace has been properly provisioned with NVFLARE certificates
  • The FL_PROVISIONED_DIR path is correctly resolved (check Makefile output)

AWS Deployment

For production deployments on AWS, see the AWS Deployment Guide. This covers provisioning infrastructure with OpenTofu (Terraform), configuring AWS services, and deploying the platform at scale.

For hybrid on-premises trust deployments, see the Local Trust Deployment Guide.

Project Structure

The repository is organised as follows:

  • deploy: Contains the Docker deployment and infrastructure files
  • docs: Contains the documentation files
  • flip-api: Contains the central hub API service
  • flip-ui: Contains the UI service
  • trust: Contains the services that would be deployed in individual trust environments.
    • data-access-api: Contains the data access API service
    • imaging-api: Contains the imaging API service
    • observability: Contains the observability stack (Grafana, Loki, Alloy)
    • omop-db: Contains a mocked OMOP database
    • orthanc: Contains a mocked PACS service (uses Orthanc)
    • trust-api: Contains the trust API service
    • xnat: Contains a mocked XNAT service

Trust Authentication

Trusts authenticate to the Central Hub using per-trust API keys. All trust communication is outbound — trusts poll the hub over HTTPS at the canonical subdomain (e.g. https://app.flip.aicentre.co.uk/api/...), which is fronted by CloudFront and proxied to the ALB. See CLAUDE.md for the full authentication model and deploy/providers/local/README.md for on-premises trust deployment.

Contributing

We welcome contributions from the community. See CONTRIBUTING.md for guidance on setting up a development environment, adding new services, coding standards, testing practices, and the pull request process.

Further Resources