{"architecture":{"backbone":"ViT-B image encoder + prompt encoder + mask decoder","family":"Segment Anything (SAM)","input":"1024\u00d71024 2D image + bounding-box prompt","output":"binary mask","params":"~94M","pretraining":"SAM (natural images) then full fine-tuning on medical image\u2013mask pairs"},"article":null,"cancer_slugs":["pan-cancer"],"category":"radiology","confidence":"high","datasets":[{"name":"TCIA \u2014 The Cancer Imaging Archive","note":"many TCIA collections are in the 1.57M-pair training set","role":"training","slug":"tcia"}],"developer":"Bo Wang Lab, University of Toronto / University Health Network","evaluation":[{"benchmark":"86 internal validation tasks + 60 external tasks","external":true,"metric":"Dice","source":"https://www.nature.com/articles/s41467-024-44824-z","value":"consistently above SAM and specialist U-Nets on external tasks (paper)"}],"hf":{"downloads":3637,"fetched_at":"2026-09-09T21:33:09Z","gated":false,"last_modified":"2023-05-03","library":"transformers","license":"apache-2.0","likes":24,"pipeline_tag":"mask-generation"},"kind":"foundation","license":"apache-2.0","limitations":"- 2D only; volumetric structures need slice-wise prompting.\n- Depends on the user's box: a loose box yields a loose mask.\n- Training data is a union of public sets \u2014 check for overlap with your test data.","links":{"demo":null,"docs":null,"doi":"10.1038/s41467-024-44824-z","github":"https://github.com/bowang-lab/MedSAM","huggingface":"https://huggingface.co/wanglab/medsam-vit-base","paper":"https://www.nature.com/articles/s41467-024-44824-z","pmid":null},"modalities":["radiology-ct","radiology-mri","ultrasound","endoscopy","histopathology","radiology-xray","multimodal"],"name":"MedSAM","notable_uses":"","openness":"open-weights","regulatory":{"intended_use_en":"Research and annotation assistance.","intended_use_pl":"Badania i wsparcie adnotacji.","source_url":"https://github.com/bowang-lab/MedSAM","status":"research-only"},"regulatory_status":"research-only","release_date":"2024-01-22","run_snippet":"# generic transformers loader \u2014 see the model card for the task-specific head and preprocessing\nfrom transformers import AutoModel, AutoProcessor\nmodel = AutoModel.from_pretrained('wanglab/medsam-vit-base')\nprocessor = AutoProcessor.from_pretrained('wanglab/medsam-vit-base')\n","settings":["basic-research","treatment-planning"],"slug":"medsam","sources":[{"label":"Ma J et al. Segment anything in medical images. Nat Commun 2024","url":"https://www.nature.com/articles/s41467-024-44824-z"},{"label":"GitHub \u2014 bowang-lab/MedSAM","url":"https://github.com/bowang-lab/MedSAM"}],"summary":"Segment Anything adapted to medical images: a promptable segmentation model fine-tuned on over 1.5 million image\u2013mask pairs across 10 imaging modalities and more than 30 cancer types.","tasks":["segmentation"],"training":{"institutions":"public datasets (see paper supplement)","size":"1.57M image\u2013mask pairs","summary_en":"1,570,263 image\u2013mask pairs assembled from public datasets (including many TCIA collections and challenge sets) across 10 modalities; 3D volumes were sliced to 2D.","summary_pl":"1 570 263 pary obraz\u2013maska zebrane z publicznych zbior\u00f3w (w tym wielu kolekcji TCIA i zbior\u00f3w konkursowych) z 10 modalno\u015bci; wolumeny 3D poci\u0119te na przekroje 2D."},"updated_at":"2026-09-09T21:33:09.703726","url":"/ai-oncology/models/medsam","usage":{"hardware":"single GPU; CPU feasible for single images","library":"PyTorch (repo) / transformers","notes_en":"Box prompts only in the released model (no point prompts). Works slice by slice \u2014 3D consistency is up to you."},"verified_at":"2026-09-05T22:26:01.532196","verified_by":"editorial","version":null,"what_it_does":"Given a bounding box on a CT, MRI, ultrasound, endoscopy, pathology or X-ray image, returns a segmentation mask of the structure or lesion inside; useful for fast annotation and as a starting point for organ- or tumour-specific fine-tuning."}