Rohan Yadav (@brohanster) profile picture

:thinking: @ Stanford about programming languages and computer systems Previously @CockroachDB

Joined November 2019
Photo by Rohan Yadav (@brohanster)
It’s been a long time since I’ve read an article as good as ⁦@brohanster⁩’s explanation of warp specialization from first principles. Gist: warp specialization lets us mimic an OOO processors without using up the chip space to implement OOO execution rohany.github.io/blog/warp-s…
2
16
756
Rohan Yadav retweeted
did you know you've been doing test-time learning this whole time? transformers, SSMs, RNNs, are all test-time regressors but with different design choices we present a unifying framework that derives sequence layers (and higher-order attention👀) from a *single* equation 🧵
8
96
11
539
100,333
a little late to the game, but: I am on the 2024 job market for tenure-track positions! My research focuses on parallel programming. Specifically: I want to make it simpler and safer to develop parallel software. site: cs.cmu.edu/~swestric/ research: cs.cmu.edu/~swestric/other/w…
3
30
2
109
20,385
Happy to announce that my internship project at NVIDIA this summer has been released as an alpha version: github.com/nv-legate/legate.…! It’s an aspiring drop-in replacement for `scipy.sparse` that automatically scales sparse matrix computations onto clusters of GPUs.
1
5
Combined with github.com/nv-legate/cunumer…, we envision end users taking advantage of supercomputer scale resources in pure Python. Happy to talk to anyone who is running SciPy programs on large data and wants to accelerate their code!
1
1
A short example of code using Legate Sparse is here: github.com/nv-legate/legate.…. This is a CG solver for the Poisson equation on a mesh. This pure NumPY + SciPy code is competitive with libraries like PETSc!
Excited to announce that our paper on compiling tensor algebra programs to distributed machines with DISTAL has been conditionally accepted to PLDI 2022 (preprint here: arxiv.org/abs/2203.08069). Some interesting ideas from our work in the thread 🧵.
4
1
9
This combined approach was an important factor in avoiding fragility of analysis that often accompanies compiler passes that try to analyze inter-processor communication, and enables easier composition with existing programs.
2
DISTAL is implemented with a hybrid static + dynamic approach. A compiler specializes generated code and distribution strategies, and describes the desired communication to Legion, a parallel runtime. At runtime, Legion manages data movement through deep memory hierarchies.
3
This separation allows for expression of a wide variety of algorithms — for example, a wide variety of distributed matrix multiplication algorithms (Cannon’s, SUMMA, 2.5D, etc) are all expressible in DISTAL.
2
DISTAL allows for separately defining how your computation and data are distributed onto a target machine. This separation allows for either adapting data layout to an algorithm, or reshaping the algorithm to keep program data at rest!
2
I gave a talk yesterday at the Oxford Tensor Computations seminar about my recent work on compiling tensor programs to run on supercomputers. Check out the talk here: youtube.com/watch?v=P2Tt2yhn…
6
I'll be presenting my recent research at this seminar! Should be a fun time :D
This week's seminar is @brohanster on "DISTAL: The Distributed Tensor Algebra Compiler". 16:00 UTC Friday1 19th, registration details on the series webpage. (Martin Elsman's talk from last week is postponed to 26th, @gckeller postponed to next term.)
1
10
Meet and learn from women engineers & leaders like @hellastellah, CEO of @fractionalapp, at our upcoming Women in Engineering seminar series! Apply to attend breakout sessions, get mentorship from YC alumna & eng managers, and join a community of peers: workatastartup.clicksocialca…
3
17
A couple days ago I gave a talk at the MIT Fast Code Seminar, where I presented our work on disentanglement and memory management for parallel programs. Check out the talk here: youtu.be/XCtIsgByg9w Big thanks to Julian Shun for inviting me to give the talk!
2
7
exciting news: we've released our *free forever* tier of @CockroachDB on Cockroach Cloud. a lot of you have been asking about this - now is your chance to sign up and try it out! cockroachlabs.cloud
1
6
32
Hype for the new release!! Lot's of big stuff I and my old team worked hard on are now out :D
🎉 Announcing CockroachDB 20.2 🎉 🚀 Support for Spatial Data 🚀 Kubernetes Operator 🚀 Backup & Restore moved to open source 🚀 TPC-C Benchmark Performance: 40% improvement 🚀 And plenty more: cockroa.ch/3kbDocY #CockroachDB #DistributedSQL
10