Genecorpus-30M
Genecorpus-30M is a pretraining corpus of about 30 million human single-cell transcriptomes assembled from 561 publicly available datasets, of which 27,406,217 cells passed quality filters. It was built to pretrain Geneformer (Theodoris et al., Nature 2023) and is released under Apache-2.0. Crucially for oncology use: cells with high mutational burden — malignant cells and immortalized cell lines — were deliberately excluded from the corpus.
At a glance
Labels and annotations
No disease labels. The corpus is unlabelled single-cell expression data used for self-supervised pretraining; cells were retained on quality criteria only (total counts and mitochondrial reads within three standard deviations of the per-dataset mean).
Details
What is in it
- ~30 million human single-cell transcriptomes from a broad range of tissues, assembled from 561 publicly available datasets cited in the Methods of Theodoris et al., Nature 2023.
- 27,406,217 cells passed the quality filters and form the corpus proper.
- Contributing repositories include NCBI GEO, NCBI SRA, the Human Cell Atlas, EMBL-EBI Single Cell Expression Atlas, the Broad Institute Single Cell Portal, BBI-Allen Single Cell Atlases, the Tumor Immune Single-cell Hub (TISCH) and Panglao.
What it is for
- Self-supervised pretraining of transformer models over gene expression; it is the corpus behind Geneformer.
What is not in it — read this before using it for oncology
- Malignant cells and immortalized cell lines were excluded by design. The card states: cells with high mutational burdens were removed because they could lead to substantial network rewiring without companion genome sequencing to support interpretation.
- Even where cells were drawn from a tumour-focused repository (TISCH), malignant cells were excluded.
- There are therefore no cancer labels, no tumour/normal split and no clinical outcomes.
How to get it
- Downloadable from Hugging Face without registration, under Apache-2.0.
Known pitfalls
- Batch effects across 561 source studies are not removed; only per-dataset quality filtering was applied.
- A model pretrained on this corpus has not seen malignant expression profiles during pretraining — any oncological capability comes from fine-tuning, and this should be stated when such a model is evaluated on cancer tasks.
Models trained or evaluated on it
- pretraining Geneformer Theodoris Lab (Gladstone Institutes / UCSF)
Sources
This page is educational — it is not medical advice and does not replace consultation with an oncologist. Diagnostic and treatment decisions are made solely by specialist physicians.