🎉 Excited to share that the last paper of my PhD is now published in PRX Life!

We introduce RAG-ESM, a retrieval-augmented framework that makes pretrained protein language models (like ESM2) homology-aware with minimal training cost.

📄 Paper: journals.aps.org/prxlife/abst...

Protein-protein interactions studied by @cyrilmalbranke.bsky.social #PragueBioML @elixircz.bsky.social

[8/8] 💻 Resources:
• Training dataset
• 4 pre-trained models (XS → L)
• Code & interactive notebooks
🔗 huggingface.co/collections/...
🔗 github.com/Bitbol-Lab/P...

[7/8] 📊 In conclusion, results show strong performances across species and benchmarks for both PPI prediction and gene essentiality. ProteomeLM makes proteome-wide analysis more practical, easing large-scale studies, including in complex eukaryotic proteomes.

Gene essentiality, showing performance outperrforms ESM-C, and that the prediction are good on E.coli, S. cerevisae and minimal cells

[6/8] 🎯 Beyond PPIs: ProteomeLM predicts gene essentiality across diverse taxa (e.g. E. coli, yeast, minimal cells), highlighting its potential for broad downstream applications.

Barplot showing speed improvement over classical DCA methods

Number of predictions in function of recall to show performance leap from classical DCA methods to ProteomeLM on human interactome (0.73 -> 0.826 AUROC)

Performance on the D-SCRIPT dataset on four organisms for supervised PPI

[5/8] ⚡ This allows unsupervised and supervised PPI prediction at proteome scale in minutes, several orders of magnitude faster than coevolution-based methods such as DCA.
Try it here: github.com/Bitbol-Lab/P...

Heatmap plot showing that ProteomeLM attention heads can distinguish interacting vs non interacting pairs in E.coli, S. cerevisiae, H. sapiens

[4/8] 🎯 Key finding: Attention heads spontaneously encode protein–protein interaction networks. Some heads can reach an AUC of 0.92 in discriminating interacting vs non-interacting pairs.

[3/8] 🧬 Encoding strategy: Instead of positional encoding, ProteomeLM introduces a functional encoding based on orthologous groups. Thus the model can leverage functional encoding and other proteins. This is especially important in eukaryotes, where gene order is less conserved.

Figure 1. ProteomeLM Architecture

[2/8] 🧬 Training objective: ProteomeLM uses a custom masked language modeling task, predicting masked ESM-C representations of proteins within the proteome.

ProteomeLM: A proteome-scale language model allowing fast prediction of protein-protein interactions and gene essentiality across taxa Language models starting from biological sequence data are advancing many inference problems, both at the scale of single proteins, and at the scale of genomic neighborhoods. In this paper, we introduce ProteomeLM, a transformer-based language model that reasons on entire proteomes from species spanning the tree of life. Leveraging protein language model embeddings, ProteomeLM is trained to reconstruct masked protein embeddings using the whole proteomic context. It thus learns contextualized protein representations reflecting proteome-scale functional constraints. We show that ProteomeLM spontaneously captures protein-protein interactions (PPI) in its attention coefficients. We demonstrate that it screens whole interactomes orders of magnitude faster than amino-acid coevolution-based methods, and substantially outperforms them. We further develop ProteomeLM-PPI, a supervised PPI prediction network that combines ProteomeLM embeddings and attention coefficients, and achieves state-of-the-art performance across species and benchmarks. Finally, we introduce ProteomeLM-Ess, a supervised predictor of gene essentiality that generalizes across diverse taxa. Our results highlight the power of proteome-scale language models for addressing function and interactions at the organism level. ### Competing Interest Statement The authors have declared no competing interest. European Research Council, https://ror.org/0472cxd90, 851173

[1/8] 📄 New preprint! With Gionata Paolo Zalaffi & Anne-Florence Bitbol, we introduce ProteomeLM, a transformer that processes entire proteomes (prokaryotes and eukaryotes), enabling ultra-fast protein–protein interaction (PPI) prediction across the tree of life.
🔗 www.biorxiv.org/content/10.1...