Jiarui Lu (@jiarlu) profile picture

doing things

Joined December 2018
Photo by Jiarui Lu (@jiarlu)Photo by Jiarui Lu (@jiarlu)Photo by Jiarui Lu (@jiarlu)Photo by Jiarui Lu (@jiarlu)Photo by Jiarui Lu (@jiarlu)Photo by Jiarui Lu (@jiarlu)Photo by Jiarui Lu (@jiarlu)Photo by Jiarui Lu (@jiarlu)Photo by Jiarui Lu (@jiarlu)Photo by Jiarui Lu (@jiarlu)
Jiarui Lu retweeted
I am excited to be joining CS@U Wisconsin-Madison in Fall 2027! I will recruit PhD students. If you are also enthusiastic about researching how LLMs work and/or applying them to other scientific problems, please apply and mention me in your application. @WisconsinCS
17
52
5
302
23,689
Jiarui Lu retweeted
We proudly gave @OpenAI our NVIDIA DGX-1 supercomputer to work on #AI's toughest problems. nvda.ws/2aRtOhE
216
1,276
399
11,567
This is so cool! Glad to see how quickly people are able to work with and build on the SimpleFold model! Nice work @OpenBioAI :)
Apple (yes that Apple) released SimpleFold, a new Protein folding model in their paper titled "SimpleFold: Folding Proteins is Simpler than You Think" yesterday. With SimpleFold, Apple makes a case that specialised architecture is not required for protein folding, which is a departure from famous AlphaFold like methods. SimpleFold uses standard transformer blocks trained via a generative flow-matching. And similar to Protenix-mini does not require MSA, which makes it faster. We have integrated SimpleFold into OpenBio. Trying it out is as simple as asking OpenBio to predict the structure of your protein sequence. You can use it now and see how good SimpleFold is.
2
1
12
951
Thanks @OpenBioAI for integrating SimpleFold! Glad to see the community start using our model
Apple (yes that Apple) released SimpleFold, a new Protein folding model in their paper titled "SimpleFold: Folding Proteins is Simpler than You Think" yesterday. With SimpleFold, Apple makes a case that specialised architecture is not required for protein folding, which is a departure from famous AlphaFold like methods. SimpleFold uses standard transformer blocks trained via a generative flow-matching. And similar to Protenix-mini does not require MSA, which makes it faster. We have integrated SimpleFold into OpenBio. Trying it out is as simple as asking OpenBio to predict the structure of your protein sequence. You can use it now and see how good SimpleFold is.
2
2
23
2,159
So I understand that was unexpected for a lot of people, @Apple MLR has released a protein folding model! arxiv.org/abs/2509.18480. Here’s a summary of what SimpleFold is and what it represents: - What is SimpleFold? A generative model that essentially treats protein folding almost exactly as if it were a text-to-image or text-to-3D problem. - What are we sharing? A research paper and a codebase under an MIT license github.com/apple/ml-simplefo… (looking forward to people contributing to it!). We are also releasing pre-trained checkpoints of different sizes so that researchers can best tradeoff performance for efficiency. - Why protein folding? We are doing this work largely because protein folding is an excellent benchmark for structured data generation and multi-modality. Protein folding is a very interesting problem from a generative modeling perspective and we do research on generative modeling :) - Why is it interesting? IMO SimpleFold is interesting because I believe in finding recipes (architectures, training objectives, etc.) that generalize across the board to many different data modalities. Let’s say you are an ML expert in text-to-image or text-to-3D, now you can apply your latest and greatest architectural blocks or efficient samplers to protein folding with SimpleFold. I believe this is a net benefit for ML research and science in general. Now getting more into the technical details: - Our architecture is very simple (hence the name), just a stack of transformer blocks with time-step conditioning. This is important because it makes the model efficient at inference time. You can run SimpleFold directly on your Mac and get results quickly without data ever leaving your laptop. - SimpleFold is not necessarily a model that “rejects” inductive biases, it just doesn’t enforce them directly on the architecture. For example, we apply rotation augmentation to all the protein structures during training. This makes the model “softly” invariant to this symmetry. - There were some concerns online about data leakage from AFESM and that driving performance of SimpleFold or making it overfit. We filtered AFESM data so that the CASP14 sequences are not seen during training. As a matter of fact we distilled structures from AF2/ESMFold models, which have the same cutoff data as SimpleFold for PDB data. Both AF2 and ESMFold train on self-distilled datasets, we just train SimpleFold on a bigger set of distilled data. I want to thank my awesome team of collaborators, they are all rockstars. That’s all, for now :)
SimpleFold: Folding Proteins is Simpler than You Think "we introduce SimpleFold, the first flow-matching based protein folding model that solely uses general purpose transformer blocks. Protein folding models typically employ computationally expensive modules involving triangular updates, explicit pair representations or multiple training objectives curated for this specific domain. Instead, SimpleFold employs standard transformer blocks with adaptive layers and is trained via a generative flow-matching objective with an additional structural term."
7
40
3
309
50,744
SimpleFold is out! I'm excited to see how the community receives this work from our research group at Apple, and I hope to see people trying out protein folding on their own laptops with MLX!
New preprint & open-source! 🚨 “SimpleFold: Folding Proteins is Simpler than You Think” (arxiv.org/abs/2509.18480). We ask: Do protein folding models really need expensive and domain-specific modules like pair representation? We build SimpleFold, a 3B scalable folding model solely built on general-purpose transformers + flow matching, and is trained on 9M structures. SimpleFold supports easy deployment and efficient inference on consumer-level hardware with PyTorch/MLX (try it on your MacBook!) (1/n)
1
3
11
1,494
SimpleFold: Folding Proteins is Simpler than You Think "we introduce SimpleFold, the first flow-matching based protein folding model that solely uses general purpose transformer blocks. Protein folding models typically employ computationally expensive modules involving triangular updates, explicit pair representations or multiple training objectives curated for this specific domain. Instead, SimpleFold employs standard transformer blocks with adaptive layers and is trained via a generative flow-matching objective with an additional structural term."
28
135
34
1,268
287,682
New preprint & open-source! 🚨 “SimpleFold: Folding Proteins is Simpler than You Think” (arxiv.org/abs/2509.18480). We ask: Do protein folding models really need expensive and domain-specific modules like pair representation? We build SimpleFold, a 3B scalable folding model solely built on general-purpose transformers + flow matching, and is trained on 9M structures. SimpleFold supports easy deployment and efficient inference on consumer-level hardware with PyTorch/MLX (try it on your MacBook!) (1/n)
12
87
19
350
105,881
Jiarui Lu retweeted
GenBio Workshop ORAL Presentation 📜 Title: FORT: Forward-Only Regression Training of Normalizing Flows 🕐 When: Fri 18 Jul 🗺️ Where: East Exhibition Hall A 🔗 arXiv: arxiv.org/pdf/2506.01158 w/ @danyalrehman17 @osclsd @jiarlu @tangjianpku @mmbronstein @Yoshua_Bengio @AlexanderTong7
1
5
9
2,285
Excited to release FORT, a new regression-based approach for training normalizing flows 🔥! 🔗 Paper available here: arxiv.org/abs/2506.01158 New paper w/ @osclsd @jiarlu @tangjianpku @mmbronstein @Yoshua_Bengio @AlexanderTong7 @bose_joey 🧵1/6
2
24
6
88
23,684
Aligning Protein Conformation Ensemble Generation with Physical Feedback 1.This work introduces Energy-based Alignment (EBA), a method to fine-tune protein structure diffusion models using physical feedback. Unlike conventional data-driven approaches, EBA integrates energy functions to better reflect the underlying biophysics of protein conformational ensembles. 2.The key innovation is aligning a generative diffusion model with the Boltzmann distribution through energy comparisons between conformational states. Instead of directly computing the intractable partition function, EBA uses energy-weighted losses across mini-batches. 3.EBA enables the model to favor low-energy, physically plausible conformations by leveraging relative energy differences (Boltzmann factors), ensuring generated ensembles reflect realistic thermodynamic landscapes. 4.Built on AlphaFold3's all-atom modeling backbone, the method retains full atomic resolution and avoids reliance on coarse-grained representations. This helps capture subtle yet important conformational variations. 5.The model is fine-tuned in two stages: supervised training on MD trajectories, followed by physical alignment using EBA. The latter introduces a novel loss objective combining aligned structural MSE and smooth LDDT to enforce SE(3)-invariant geometry-aware accuracy. 6.Experiments on the ATLAS MD ensemble benchmark show that EBA achieves state-of-the-art performance across a range of metrics, including RMSF correlations, Wasserstein distances, and contact map similarity, outperforming previous models like AlphaFlow and MDGen. 7.Ablation studies demonstrate EBA's robustness to the choice of mini-batch size (K), with consistent improvements across settings. The authors also show that DPO (Direct Preference Optimization) is a special case of EBA under binary energy comparisons. 8.EBA bridges the gap between physics-based and learning-based protein modeling. It offers a scalable solution for training energy-aware diffusion models that remain tractable without resorting to full molecular dynamics simulations. 9.While powerful, the method is limited to single-chain proteins and relies on approximate energy functions. Extending it to quantum-accurate energies or multimeric complexes remains a direction for future work. 10.This study reflects a growing trend of integrating domain knowledge into deep generative modeling. EBA represents a principled step toward physically grounded protein ensemble generation with applications in drug discovery and structural biology. 📜Paper: arxiv.org/abs/2505.24203 #ProteinDesign #DiffusionModels #MolecularDynamics #AlphaFold #ComputationalBiology #DeepLearning #Biophysics
6
31
1,884
🚨 BIG NEWS 🚨 APACHE 2.0 LICENSE ProteniX Model & Code were free for academic use. Now we're expanding the license to allow commercial activities! We can't wait to see what you will build with it !
The wait for trainable SoTA protein folding model is over! We are happy to introduce Protenix (Protein + X Molecules!), our Open Source reproduction of AF3, including inference code, dataset, weights, and training pipeline. Check our code: github.com/bytedance/Proteni… 🧵 1/N
2
14
4
46
15,259
The wait for trainable SoTA protein folding model is over! We are happy to introduce Protenix (Protein + X Molecules!), our Open Source reproduction of AF3, including inference code, dataset, weights, and training pipeline. Check our code: github.com/bytedance/Proteni… 🧵 1/N
3
43
7
116
20,258
Jiarui Lu retweeted
🌟Discover the potential of structural information to enhance Protein Language Models (PLMs)!🌟 🔬 Explore our method for distilling structural insights into PLMs, even without explicit structural inputs. 🧬 🔗 Arxiv: arxiv.org/abs/2402.05856 👨‍💻 Code: github.com/DeepGraphLearning…
2
18
63
6,222
(1/2) Our MoleculeSTM paper is now published in @NatMachIntell @SpringerNature! It is our primary exploration of using LLMs for drug design. Special thx to the team and advisors! @tangjianpku @ChaoweiX @AnimaAnandkumar Paper: bit.ly/4736BPH Code: bit.ly/4877YOS
3
13
2
51
11,747
Excited to share our recent work MoleculeSTM, a foundation model for small molecule discovery. ArXiv: arxiv.org/abs/2212.10789, website: chao1224.github.io/MoleculeS…. A nice collab. w/ @wn8_nie, Chengpeng, @jerrylujr, @ZhuoranQ, Ling, @tangjianpku, @ChaoweiX, @AnimaAnandkumar.
11
18
2
83
20,144