Although single-cell foundation models are powerful in capturing complex gene and cell features, they focus exclusively on intracellular gene interactions and neglect extracellular signaling. Spatial transcriptomics (ST) technologies allow gene expression to be profiled within native tissue locations. However, existing single-cell foundation models have not exploited the additional spatial information.

To address this limitation, researchers from the Institute of Process Engineering of the Chinese Academy of Sciences, Shanghai University, and Shanghai Jiao Tong University have developed a transformer model adapted for ST data. This model improves upon cell- and gene-level analyses of a single-cell foundation model and enables in silico perturbation to investigate spatial ligand signaling responses.

The study was published in Cell Reports Methods on September 30.

The model, named stFormer, uses cross-attention to incorporate spatial ligand gene signals. The researchers derived a biased cross-attention formula to unify ST platforms across resolution and gene coverage. This enables the model to learn with single-cell resolution using whole-transcriptome and low-resolution Visium data.

To pretrain the model, the researchers assembled a corpus of approximately 4.1 million spatial samples from public human Visium datasets. After pretraining, stFormer can be applied to both single-cell-resolution and spot-resolution ST data. This allows stFormer to work across different spatial transcriptomics platforms.

"Spatial ligand expressions provide extra information for boosting performance upon the single-cell foundation model," said Prof. YUAN Ye, corresponding author of the study. He noted that stFormer outperforms scFoundation, a state-of-the-art single-cell foundation model, across diverse tasks, including cell clustering, batch correction, cell type prediction, and gene function prediction.

Moreover, stFormer enables in silico perturbation of spatial ligand-receptor interactions, thereby expanding its utility for mechanistic exploration. In two case studies, the authors demonstrate that stFormer successfully delineates the layer architecture of a developing human cortical region and identifies antigen-presenting genes that activate T cell immunity in an immune-infiltrated tumor microenvironment

This study establishes a model architecture that incorporates spatial information into a gene expression sequence model while accommodating diverse ST technologies. It offers a versatile tool for spatial transcriptomic analysis, which learned general knowledge from pretraining ST data and transferred it to specific ST data for biological exploration.

Overview of model architecture, pretraining dataset, and downstream tasks (Image by CAO Shenghao)