{"architecture":{"backbone":"esm2_t33_650M (33 layers) \u2014 checkpoints from 8M to 15B","family":"Transformer encoder (BERT-style)","input":"amino-acid sequence (up to 1,024 tokens by default)","output":"per-residue embeddings (1,280-dim for 650M), masked-token logits","params":"650M (this checkpoint)","pretraining":"masked language modelling on UniRef50/UniRef90"},"article":null,"cancer_slugs":["pan-cancer"],"category":"protein-structure","confidence":"high","datasets":[],"developer":"Meta AI (FAIR)","evaluation":[{"benchmark":"CAMEO / CASP14 (ESMFold)","external":true,"metric":"TM-score","source":"https://www.science.org/doi/10.1126/science.ade2574","value":"close to AlphaFold 2 on high-perplexity-free targets, faster by orders of magnitude (paper)"}],"hf":{"downloads":1303383,"fetched_at":"2026-09-09T21:33:09Z","gated":false,"last_modified":"2023-03-21","library":"transformers","license":"mit","likes":89,"pipeline_tag":"fill-mask"},"kind":"foundation","license":"MIT","limitations":"- Sequence-only: no ligands, no complexes.\n- Zero-shot variant scores are less accurate than supervised predictors for clinical classification.","links":{"demo":null,"docs":null,"doi":"10.1126/science.ade2574","github":"https://github.com/facebookresearch/esm","huggingface":"https://huggingface.co/facebook/esm2_t33_650M_UR50D","paper":"https://www.science.org/doi/10.1126/science.ade2574","pmid":null},"modalities":["protein-sequence"],"name":"ESM-2 / ESMFold","notable_uses":"","openness":"open-weights","regulatory":{"intended_use_en":"Research tool.","intended_use_pl":"Narz\u0119dzie badawcze.","source_url":"https://github.com/facebookresearch/esm","status":"not-applicable"},"regulatory_status":"not-applicable","release_date":"2022-11-01","run_snippet":"from transformers import AutoTokenizer, EsmModel\ntok = AutoTokenizer.from_pretrained('facebook/esm2_t33_650M_UR50D')\nmodel = EsmModel.from_pretrained('facebook/esm2_t33_650M_UR50D')\nout = model(**tok('MKTAYIAKQRQISFVKSHFSRQ', return_tensors='pt'))\nemb = out.last_hidden_state  # 1 x L x 1280","settings":["basic-research","drug-discovery"],"slug":"esm-2","sources":[{"label":"Lin Z et al. Evolutionary-scale prediction of atomic-level protein structure with a language model. Science 2023","url":"https://www.science.org/doi/10.1126/science.ade2574"},{"label":"Hugging Face \u2014 facebook/esm2_t33_650M_UR50D","url":"https://huggingface.co/facebook/esm2_t33_650M_UR50D"}],"summary":"Protein language models from 8 million to 15 billion parameters trained on UniRef sequences; embeddings power variant-effect and function prediction, and ESMFold predicts structure directly from a single sequence without MSAs.","tasks":["feature-extraction","variant-effect","protein-structure"],"training":{"institutions":"Meta AI; UniProt as data source","summary_en":"UniRef50 clusters (sampled from UniRef90 members), ~65 million unique sequences.","summary_pl":"Klastry UniRef50 (pr\u00f3bkowane z cz\u0142onk\u00f3w UniRef90), ok. 65 mln unikalnych sekwencji."},"updated_at":"2026-09-09T21:33:09.703719","url":"/ai-oncology/models/esm-2","usage":{"library":"transformers / esm package","snippet":"from transformers import AutoTokenizer, EsmModel\ntok = AutoTokenizer.from_pretrained('facebook/esm2_t33_650M_UR50D')\nmodel = EsmModel.from_pretrained('facebook/esm2_t33_650M_UR50D')\nout = model(**tok('MKTAYIAKQRQISFVKSHFSRQ', return_tensors='pt'))\nemb = out.last_hidden_state  # 1 x L x 1280"},"verified_at":"2026-09-05T22:26:01.022006","verified_by":"editorial","version":null,"what_it_does":"Masked-language modelling over amino acids; the 650M checkpoint is the practical default for embeddings, zero-shot variant scoring (log-likelihood ratios) and fine-tuning on oncology protein tasks."}