research scientist at @googledeepmind. co-author of JAX (https://xitter.cf/t.co/sS9COjJPsx)
sfba
Joined April 2008
- Tweets123
- Following812
- Followers1.6K
- Likes476
Roy Frostig retweeted
Curious how to write SOTA performance Blackwell matmul kernels using MGPU? We just published a short step-by-step tutorial: docs.jax.dev/en/latest/palla…
At each step, we show exactly what (small) changes are necessary to refine the kernel and the final kernel is just under 150 lines.
A nice and concise R1 inference jax:tpu port by @rdyro128523. Good for both reading and running. Watch the repo for more.
Roy Frostig retweeted
Training our most capable Gemini models relies heavily on our JAX software stack + Google's TPU hardware platforms.
If you want to learn more, see this awesome book "How to Scale Your Model":
jax-ml.github.io/scaling-boo…
It was put together by my @GoogleDeepMind colleagues
@_sholtodouglas, @charliexychen, @pchoy95, @albertwebson, @vinayramasesh, @froystig, @anselmlevskaya, @sharadvikram, and Fede Lebron, and describing the work of many, many people!
Our online book on systems principles of LLM scaling is live. We hope that it helps you make the most of your computing resources. Enjoy!
Roy Frostig retweeted
We now have a guide to writing distributed communication on TPU using Pallas, written by @JustinFu769512! jax.readthedocs.io/en/latest…
Overlapping comms + compute is a crucial performance optimization for large scale ML. Write your own custom overlapped kernels in Python!
Roy Frostig retweeted
Modula x JAX = Modulax
@gallabytes is cracked and ported Modula into JAX in a few days. I haven't had a chance to test yet, but I'm really excited about this project. Tagging @froystig and @SingularMattrix
github.com/GallagherCommaJac…
(1/3)
Roy Frostig retweeted
I've finally landed my first proper JAX feature since joining the team: a supported "foreign function interface", which makes it easier to call into external libraries from within JAX code. Check it out: jax.readthedocs.io/en/latest…
Roy Frostig retweeted
Finally got around to writing a guide for matrix multiplication on TPUs using Pallas. Check it out!
jax.readthedocs.io/en/latest…
Roy Frostig retweeted
Many of you are excited about H100 attention, so it’s a good time to show you Mosaic GPU: a Python DSL for H100s.
The attention example matches FA3 performance, while being only ~200 lines of Python: github.com/google/jax/blob/m…
It's easy to install too! Latest JAX packages have it.
Roy Frostig retweeted
Excited to share Penzai, a JAX research toolkit from @GoogleDeepMind for building, editing, and visualizing neural networks! Penzai makes it easy to see model internals and lets you inject custom logic anywhere.
Check it out on GitHub: github.com/google-deepmind/p…
Roy Frostig retweeted
Built with JAX!
Introducing Gemini 1.0, our most capable and general AI model yet. Built natively to be multimodal, it’s the first step in our Gemini-era of models. Gemini is optimized in three sizes - Ultra, Pro, and Nano
Gemini Ultra’s performance exceeds current state-of-the-art results on 30 of the 32 widely-used academic benchmarks. With a score of 90.0%, Gemini Ultra is the first model to outperform human experts on MMLU.
blog.google/technology/ai/go…
Roy Frostig retweeted
Today, I’m excited to announce the release of Levanter 1.0, our new JAX-based framework for training foundation models, which we’ve been working on @StanfordCRFM. Levanter is designed to be legible, scalable and reproducible. crfm.stanford.edu/2023/06/16…
In JAX and Dex, we do automatic differentiation (AD) in a distinctive way: by "linearizing" and then "transposing" programs. We wrote up what this looks like in a model language: arxiv.org/abs/2204.10923
with Alexey Radul, @apaszke, @SingularMattrix, @DougalMaclaurin
To get there, we need to identify linearity *in programs*. For functions expressed in code, we want a notion of linearity that implies mathematical linearity, but that also allows a compiler to delineate and transpose things automatically.