Foundation model Open weights Not applicable protein-structure

ESM-2 / ESMFold

Protein language models from 8 million to 15 billion parameters trained on UniRef sequences; embeddings power variant-effect and function prediction, and ESMFold predicts structure directly from a single sequence without MSAs.

🤗 facebook/esm2_t33_650M_UR50D GitHub Paper DOI { } JSON

🤗 1 303 383 downloads / month ♥ 89 updated 2023-03-21

At a glance

DeveloperMeta AI (FAIR)

Released2022-11-01

LicenceMIT

AvailabilityOpen weights

KindFoundation model

Regulatory statusNot applicable

Parameters650M (this checkpoint)

What it does

Masked-language modelling over amino acids; the 650M checkpoint is the practical default for embeddings, zero-shot variant scoring (log-likelihood ratios) and fine-tuning on oncology protein tasks.

Tasks, data types and cancers

TaskFeature extraction (embeddings)Variant effect Protein structure

Data typeProtein sequence

Clinical settingBasic research Drug discovery

CancerPan-cancer

Inputamino-acid sequence (up to 1,024 tokens by default)

Outputper-residue embeddings (1,280-dim for 650M), masked-token logits

Architecture

FamilyTransformer encoder (BERT-style)

Backboneesm2_t33_650M (33 layers) — checkpoints from 8M to 15B

Parameters650M (this checkpoint)

Pre-trainingmasked language modelling on UniRef50/UniRef90

Training data

UniRef50 clusters (sampled from UniRef90 members), ~65 million unique sequences.

InstitutionsMeta AI; UniProt as data source

Evaluation

Benchmark / dataset	Metric	Value	External validation	Source
CAMEO / CASP14 (ESMFold)	TM-score	close to AlphaFold 2 on high-perplexity-free targets, faster by orders of magnitude (paper)	yes	Source

How to run

from transformers import AutoTokenizer, EsmModel
tok = AutoTokenizer.from_pretrained('facebook/esm2_t33_650M_UR50D')
model = EsmModel.from_pretrained('facebook/esm2_t33_650M_UR50D')
out = model(**tok('MKTAYIAKQRQISFVKSHFSRQ', return_tensors='pt'))
emb = out.last_hidden_state  # 1 x L x 1280

Librarytransformers / esm package

Regulatory status and intended use

Regulatory statusNot applicable

Intended useResearch tool.

Source →

Regulatory status is quoted from the source linked above and can change. Research-use-only models must not be used for clinical decisions.

Limitations and bias

Sequence-only: no ligands, no complexes.
Zero-shot variant scores are less accurate than supervised predictors for clinical classification.

Sources

At a glance What it does Tasks, data types and cancers Architecture Training data Evaluation How to run Regulatory status and intended use Limitations and bias Sources

Card provenance

Confidencehigh

Verified2026-09-05 · editorial

Updated2026-09-09

Hugging Facesnapshot from 2026-09-09

Facts on this card come only from the listed sources; the Hugging Face figures are a cached public snapshot.

Related models

This page is educational — it is not medical advice and does not replace consultation with an oncologist. Diagnostic and treatment decisions are made solely by specialist physicians.