Prov-GigaPath

At a glance

What it does

Two stages: a tile encoder produces embeddings, then a slide encoder built on LongNet (dilated attention) aggregates tens of thousands of tile embeddings into a slide representation. That slide vector feeds cancer subtyping, mutation prediction and vision-language tasks.

Notable uses in oncology

State of the art on 25 of 26 tasks at publication, including pan-cancer subtyping and prediction of common driver mutations from H&E; the first pathology model to be trained end-to-end at slide scale on a health-system archive.

Tasks, data types and cancers

Architecture

The slide encoder is the distinctive part: dilated attention keeps memory manageable for sequences of 10,000+ tiles, so context from across the slide informs each token.

Training data

Providence health system archive: 171,189 H&E and IHC whole-slide images from more than 30,000 patients across 31 tissue types, covering 28 cancer centres in the US; 1.3 billion tiles at 256×256.

Linked datasets

• evaluation TCGA — The Cancer Genome Atlas (via NCI Genomic Data Commons) Free registration

Evaluation

How to run

# generic timm loader — check the model card for the exact init args and image normalisation
import timm, torch
from huggingface_hub import login
login()  # gated repos: accept the licence on huggingface.co first
model = timm.create_model('hf-hub:prov-gigapath/prov-gigapath', pretrained=True)
model.eval()
cfg = timm.data.resolve_data_config({}, model=model)
transform = timm.data.create_transform(**cfg)
# emb = model(transform(tile).unsqueeze(0))  # 1 x D tile embedding

Weights are Apache-2.0 but gated (accept terms on Hugging Face). The repo ships the tile→slide pipeline; keep tile coordinates, the slide encoder needs them.

Regulatory status and intended use

Source →

Regulatory status is quoted from the source linked above and can change. Research-use-only models must not be used for clinical decisions.

Limitations and bias

• Single health system: scanner and staining conventions of Providence dominate the pretraining distribution.
• Slide-level encoder is memory-hungry; small labs may only use the tile encoder.
• Mutation predictions from H&E are associative, not a substitute for sequencing.

Sources

1. Xu H et al. A whole-slide foundation model for digital pathology from real-world data. Nature 2024
2. Hugging Face — prov-gigapath/prov-gigapath