MedSAM
Segment Anything adapted to medical images: a promptable segmentation model fine-tuned on over 1.5 million image–mask pairs across 10 imaging modalities and more than 30 cancer types.
🤗 3 637 downloads / month ♥ 24 updated 2023-05-03At a glance
What it does
Given a bounding box on a CT, MRI, ultrasound, endoscopy, pathology or X-ray image, returns a segmentation mask of the structure or lesion inside; useful for fast annotation and as a starting point for organ- or tumour-specific fine-tuning.
Tasks, data types and cancers
Architecture
Training data
1,570,263 image–mask pairs assembled from public datasets (including many TCIA collections and challenge sets) across 10 modalities; 3D volumes were sliced to 2D.
Linked datasets
- training TCIA — The Cancer Imaging Archive Open download — many TCIA collections are in the 1.57M-pair training set
Evaluation
| Benchmark / dataset | Metric | Value | External validation | Source |
|---|---|---|---|---|
| 86 internal validation tasks + 60 external tasks | Dice | consistently above SAM and specialist U-Nets on external tasks (paper) | yes | Source |
How to run
# generic transformers loader — see the model card for the task-specific head and preprocessing
from transformers import AutoModel, AutoProcessor
model = AutoModel.from_pretrained('wanglab/medsam-vit-base')
processor = AutoProcessor.from_pretrained('wanglab/medsam-vit-base')
Box prompts only in the released model (no point prompts). Works slice by slice — 3D consistency is up to you.
Regulatory status and intended use
Regulatory status is quoted from the source linked above and can change. Research-use-only models must not be used for clinical decisions.
Limitations and bias
- 2D only; volumetric structures need slice-wise prompting.
- Depends on the user's box: a loose box yields a loose mask.
- Training data is a union of public sets — check for overlap with your test data.
Sources
This page is educational — it is not medical advice and does not replace consultation with an oncologist. Diagnostic and treatment decisions are made solely by specialist physicians.