deepspot-m

1# DeepSpot-M
2 
3## Overview
4 
5DeepSpot-M is a multimodal foundation model that maps a 224x224 H&E histology tile to
6spatial gene expression in log1p-CPM. The output is virtual spatial transcriptomics: one
7value per queried gene per tile, laid out on the grid the tiles came from.
8 
9A LoRA-adapted pathology foundation backbone (Midnight) tokenises the tile. A
10cross-attention gene decoder lets each gene query attend to the patch tokens, and a gene
11router hypernetwork builds gene-specific projections from frozen biological embeddings
12(Evo 2, Orthrus, ProtT5, scGPT, Apertus). Genes enter the model as queryable embeddings
13rather than fixed output slots, so the released model covers a ~19k protein-coding gene
14panel including genes unseen in training. The panel ships with the weights as
15`tokens.csv` and is exposed as `model.gene_names`; genes outside it cannot be queried in
16this release.
17 
18Applied to TCGA, the model produced a virtual spatial transcriptomics atlas of 28,664
19slides across 32 cancer types.
20 
21## Licensing
22 
23The code is PolyForm Noncommercial 1.0.0 and the weights are CC-BY-NC-SA-4.0.
24The current model access form additionally restricts eligibility to academic/public
25nonprofit researchers without concurrent commercial affiliations and excludes
26commercially funded/collaborative use. Read the live declarations before requesting
27access; a noncommercial intention alone does not establish eligibility. Research use
28…