Cross-silo federated learning in computational biology

APPFL Bio Suite

Four federated experiments in genomics, epigenomics and clinical prediction, one still in development, built on APPFL, hivewatch and Globus Compute, together with the tooling to run them across independently administered HPC centres. Each institution computes on its own data, on its own hardware, and returns only results. No data movement, no shared credential, no VPN, and no inbound network access.

Architecture

Coordinator and partner sites

The coordinator runs a driver and an aggregator. Each partner runs a privileged process on their own cluster, against data that stays there. What crosses the wire is whatever the experiment's aggregation requires: model weights for the multi-round experiment, per-variant summary statistics or second moments for the single-round ones.

Federation architecture A coordinator running a driver and an aggregator dispatches tasks to three partner sites. Each site computes on its own data on its own cluster and returns only weights or summary statistics. No raw data is returned. YOU · COORDINATOR driver dispatches the work aggregator combines what comes back task weights, or summary statistics PARTNER SITES ●  their cluster their scheduler, their accounts ●  their data genotypes never open to us ●  their consent code checked before a file is opened
Partners need no inbound network access: their endpoint reaches out to Globus Compute, and refuses tasks from anyone their own identity mapping does not name.

Requirements

Python 3.12, a Globus account, and partners willing to run a privileged process on their own clusters.

Not required

A shared filesystem, a VPN, matching operating systems or schedulers, or inbound network access to any participant.

Experiments

The four experiments

They differ by design. Two are multi-round model training: one on a public benchmark, one a methylation classifier still in development. The other two are single-round statistical federations, where the aggregation step is itself the method under test.

Multi-round · FedAvg

FLamby Fed-Heart-Disease

Multi-round federated training on a public tabular benchmark whose client partition follows the original hospital collection sites. The model is not the object of study; the infrastructure is. The question it answers is whether APPFL over Globus Compute can carry a multi-round job across separately administered clusters on different continents and still reach the published baseline.

Data
Partners download it; ~40 KB, CC BY 4.0
Splits
Four real hospital collection sites
Design and record of runs →
Single round · Meta-analysis

Federated GWAS

A genome-wide association study with polygenic-score evaluation, run across institutions on different continents. Each site performs a complete local GWAS and transmits only per-variant effect size, standard error and minor-allele frequency. The server combines these by inverse-variance fixed-effect meta-analysis.

Traits
BMI (linear), T2D (logistic score test)
Payload
A few MB, whatever the cohort size
Design and record of runs →
Single round · Exact aggregation

Cross-ancestry fine-mapping

Where a GWAS identifies an associated locus, fine-mapping asks which variant in it is causal. Answering that requires the linkage-disequilibrium structure, which does not decompose into per-variant summaries. Credible sets are instead built from per-site second moments. Given three stated harmonization conditions, the federated fit equals the centralized one — same credible sets, same posterior inclusion probabilities, up to floating-point summation order — and that equivalence is checked against a centralized path on a 79-locus benchmark rather than assumed.

Estimator
SuSiEx
Checked
79-locus benchmark against the centralized path
Governance
GA4GH DUO, DRS, TRS and TES
Design and record of runs →
Multi-round · Work in progress

Federated methylation

A DNA-methylation subtype classifier for sparse, nanopore-like input, built toward leukemia classification. FedAvg across simulated hospitals is compared with pooled training and with each hospital training alone, at several levels of CpG coverage. For now every site is simulated on one machine; partner execution comes later, once the model holds up on real data.

Model
Residual MLP over binary methylation calls
Data
Synthetic, or open TCGA-LAML from the NCI GDC
Status
Single-machine simulation; no partner bundle yet
Design and record of runs →

Implementation

Design decisions

Most of the difficulty in a cross-institution run is operational rather than statistical: version skew between sites, partially completed configuration, and setup instructions a partner has to interpret. The following address those.

A single configuration file

federation.yaml declares who you are, who your partners are, and what each is doing. Every generator, configuration file, document template and check reads from it. Nothing in src/ contains a coordinator identity, an endpoint UUID or a site name, and CI verifies this, since the repository has to be usable by someone other than its author.

Partner bundles are generated, not templated

A bundle is a directory containing a partner's configuration fully resolved: their site ID, their data assignment, the coordinator's identity, and their scheduler's provider block, along with their two setup documents with no placeholders remaining.

Partner-facing content is a closed set

Bundles are built from docs/partner/ and nowhere else, enforced in code, so nothing outside that tree can reach a partner by accident.

Exact version pinning

Every site runs byte-identical versions of the Globus Compute stack. Version skew does not fail at install time; it surfaces as a deserialization error several rounds into a run on another institution's cluster. constraints.txt pins the stack and preflight --check pins verifies it.

Troubleshooting is indexed by symptom

The troubleshooting registry is keyed on the error a partner actually sees, and records the underlying mechanism alongside the fix, so a message resolves to a cause rather than to a support thread.

The CLI

appfl-bio-suite preflight [--experiment X]           environment, pins, configs, endpoints
appfl-bio-suite endpoint status [SITE]               is it online
appfl-bio-suite endpoint smoke [SITE]                round-trip: does it actually work
appfl-bio-suite identity validate <mapping.json>     catch mapping bugs offline, in a second
appfl-bio-suite simulate EXPERIMENT --scenario NAME  generate and split synthetic data
appfl-bio-suite run EXPERIMENT                       run it
appfl-bio-suite partner-bundle EXPERIMENT --site X   a partner's complete setup package
appfl-bio-suite watch build | serve | export         the federation, on a map
appfl-bio-suite ga4gh duo | drs | trs | tes          data use, data objects, tools, tasks

Standards

GA4GH standards

The fine-mapping experiment is built on four GA4GH specifications. Each supplies a guarantee the design would otherwise have had to assume.

ServiceNameAnswers
DUOData Use OntologyMay this site's data be used for this study?
DRSData Repository ServiceExactly which bytes did a site compute over?
TRSTool Registry ServiceExactly which tool version computed them?
TESTask Execution ServiceRun that tool, there, on those bytes.
Taken together, they establish that a named tool version, pinned by checksum, ran over content-addressed data whose use terms permit the study that dispatched it — each clause verifiable after the fact rather than asserted.

Consent is enforced at the site

Each bundle carries a DATA_USE.json, which the site's worker matches against the study the coordinator declared, refusing before it opens a genotype file — on its own hardware, under the account it controls. The coordinator runs the same check offline before dispatch, but that one only saves a queue wait.

Bundles are content-addressed

A sha-256 per file and a Merkle identifier over the set reduce "did this site run on the bundle issued to it" to a string comparison. Without that, a site running an outdated bundle produces well-formed aggregates over the wrong individuals, which pool without complaint and are wrong in no visible way.

Provenance is recorded per run

The tool pin covers the source of the two modules APPFL ships to workers. It is verified at launch and written into each run's ga4gh_provenance.json, alongside what every site attests to having done.

All of this is opt-in per experiment: a federation.yaml with no ga4gh blocks produces configurations byte-identical to those produced before the GA4GH support existed. Read the GA4GH design →

Reproducibility

Provenance and reproducibility

Every simulation writes a manifest

The scenario, every seed, the suite commit, input and output checksums, and package versions. simulate gwas --verify <manifest> confirms a rerun mechanically.

The GWAS simulation is deterministic

For a fixed seed, all 35 outputs are byte-identical, verified against recorded checksums.

Documented limits

The per-experiment data documents state which input cohort is not regenerable, and what the shipped substitute does and does not reproduce.

Installation

Installing, and a federation on one machine

The loopback configuration runs a complete federation locally: no partners, no network, and nothing to fill in beforehand. It is the same pair of commands CI runs, and it is worth completing before recruiting any sites.

# install the release from PyPI (Python 3.12)
python3.12 -m venv .venv && source .venv/bin/activate
pip install "appfl-bio-suite[all]==0.3.2" \
    -c https://raw.githubusercontent.com/mfjoel01/appfl-bio-suite/v0.3.2/constraints.txt

# a whole federation, on one machine
appfl-bio-suite simulate gwas --scenario ci-tiny --out /tmp/gwas-ci
appfl-bio-suite run gwas --config loopback --driver serial --data-root /tmp/gwas-ci

# then stand up a real one, from the matching release tag
git clone --branch v0.3.2 https://github.com/mfjoel01/appfl-bio-suite.git
cd appfl-bio-suite
cp federation.yaml.example local/federation.yaml
$EDITOR local/federation.yaml
appfl-bio-suite preflight

docs/coordinator/new-federation.md covers going from here to a running experiment across institutions you recruit.

Citation

Authorship and citation

Developed by Matthew Joel in the Madduri Lab at Argonne National Laboratory. The suite is released under the MIT licence, with third-party terms recorded for APPFL, FLamby and PGS Catalog data.

Citing the software

Machine-readable metadata is in CITATION.cff, which GitHub renders through Cite this repository and which reference managers read directly. Cite the version you ran: every release is tagged and published on PyPI, and partners install one exact release rather than main, so the version string identifies the exact source.

Citing the study design

The GWAS experiment reproduces the simulation design of Simulation Benchmarking of Differential Privacy and Federated Learning for Secure Genetic Analysis and Polygenic Risk Scoring (Joel et al.). That reference is recorded in CITATION.cff, and docs/experiments/gwas/DATA.md states the documented limits on reproducing its published figures from this repository alone.

Network map

The federation on a map

Every institution plotted on a flat map or a globe, with the experiments each one runs, the kind of data it holds, the sample counts it declared, and whether its endpoint is currently answering. It is drawn from federation.yaml alone, so it is populated before any run has taken place. The preview on the next page shows the suite's current partner institutions and the stage each partnership has reached.

Open the network map →