Principal at Gottbrath Technologies. Tweets: PyTorch, AI, Deep Learning, Computational Science, Software Dev, HPC, Debugging, and Astronomy.
San Jose, CA
Joined August 2012
- Tweets695
- Following511
- Followers296
- Likes1.3K
Chris Gottbrath retweeted
PyTorch 2.15 is here, with 3,117 commits from 583 contributors and updates across distributed training, NVGEMM, Apple Silicon, and accelerator platform support.
The release makes the rewritten NCCL backend the default and adds one-sided transports with a NIXL backend. FSDP2 and pipelining gain finer memory and communication control. It also brings faster 4-bit (NVFP4) LLM decoding on NVIDIA Blackwell GPUs, FP8 and faster decode on Apple Silicon, and broader platform support.
On Thursday, November 5, at 11 AM PST, join Tristan Rice (@rice_fry), Natalia Gimelshein, Huy Do, and Jane Xu (@janeyx99) (@Meta) for a live PyTorch 2.15 Q&A, moderated by Chris Gottbrath (Gottbrath Tech). The speakers will share an overview of the release and answer community questions about PyTorch and the new capabilities in this release.
🔗 Read the release blog and register for the live Q&A: pytorch.org/blog/pytorch-2-1…
I'll be at PyTorch Conference in San Jose today. I have a few things lined up but am mostly here to see old friends and make new ones. I'm angel investing, advising, and consulting. If you are a founder looking to navigate this space - lets chat.
I will be at Triton Conference on monday. Should be a delicious helping of deep GPU optimization. Catch me there if you want to talk PyTorch, Kernels, Trianing or Inference.
tritonconference2026.splasht…
This is my handle for my public facing work with MachGen.AI. If anyone in my circle of peeps is in LA and wants to catch up I'm here for an event today. DM me to catch up.
I will be at one more #LATechWeek event today. "From Story to Revenue: How AI Is Reshaping Microdrama". I am looking forward to learning how the creative community think about short form video. @MachgenAI makes it quick and cheap to iterate on AI but style and content still rule
Chris Gottbrath retweeted
Our latest blog presents work on Jagged Flash Attention (JFA) — the attention kernel behind @Meta's Generative Ads Model (GEM) — on @NVIDIA Blackwell (B200), built with TLX (Triton Low-level Extensions), which add explicit, hardware-aware control on top of Triton's high-level, tile-based programming model.
The TLX attention kernel is about 3.2K lines of concise Triton-level code — roughly 3× less than the ~10K-line CuteDSL kernels of the state-of-the-art FlashAttention-4 (FA4), and it outperforms FA4 on the jagged shapes that matter for GEM — by ~13% on the forward pass and ~50% on the backward pass.
Read it here: pytorch.org/blog/optimizing-…
✍️ Han Xu, Jacky Zhou, Jackie (Jiaqi) Xu, Hongtao Yu, Peng Chen (Dev Infra), Darren Liu, Dev (Devashish) Shankar (@dev17392), Max Leung, Nick Riasanovsky, Hao Yan, Manman Ren, Yuanwei (Kevin) Fang
Chris Gottbrath retweeted
PyTorch 2.13 is here, with 3,328 commits from 526 contributors and updates across FlexAttention, CuTeDSL, nn.LinearCrossEntropyLoss, torchcomms, FSDP2, Python 3.15 wheels, ROCm, Arm, and XPU.
The release blog and notes cover FlexAttention on Apple Silicon with up to ~12x speedup over SDPA on sparse patterns, a deterministic backward path on CUDA, the CuTeDSL "Native DSL" backend for Inductor, nn.LinearCrossEntropyLoss to reduce peak GPU memory by up to 4x, torchcomms for large-cluster training, and FSDP2 communication overlap improvements.
On July 22 at 11 a.m. PT, join @albanDesmaison (@Meta), Andrey Talman (@Meta), Piotr Bialecki (@NVIDIA), and Chris Gottbrath for a live 2.13 Q&A.
🔗 Read the release blog, and register for the live Q&A: pytorch.org/blog/pytorch-2-1…
Chris Gottbrath retweeted
PyTorch 2.12 introduces major updates across compilation, export, distributed training, and accelerator support.
Highlights include up to 100x faster batched linalg.eigh on CUDA, the new torch.accelerator.Graph API, Microscaling quantization support in torch .export.save, and fused Adagrad.
The release includes 2,926 commits from 457 contributors since PyTorch 2.11.
Have questions? Join @AndreyTalman (@Meta), @albanDesmaison (@Meta), and @joespeez (@reflection_ai), moderated by @Chris_AI_HPC (@Meta), on May 20 at 10:00 AM PT for a live Q&A covering the release and answering questions from the community.
🔗 Read the release blog and register for the webinar: pytorch.org/blog/pytorch-2-1…
#PyTorch #OpenSourceAI #MachineLearning #AIInfrastructure
Chris Gottbrath retweeted
🗓️ TUESDAY @ Booth #338
12PM: Wheel Next w/ @_seemethere 1PM: OpenEnv w/ Ali Sol & @mr_purelycloud 2PM: Core Maintainer Office Hours w/ @albanD @malfet @joespeez @ptrblck_de 3PM: Titan & Forge w/ @syedahmed, Davide Italiano, Kai Wu 4PM: Ambassador Program w/ @matthew_d_white 5PM: vLLM Inference w/ @woosuk_k
Chris Gottbrath retweeted
🗓️ MONDAY @ Booth #338
2PM: Shaping the Future w/ @matthew_d_white 3PM: TensorRT + PyTorch w/ Angela Yi & @narendasan 4PM: DeepSpeed Trillion-Param Training w/ @PKUWZP 5PM: PyTorch Export w/ Angela Yi 6PM: Ray Distributed Computing w/ @robertnishihara
#AI #GTC2025
If you are at GTC and you care about AI Frameworks, don't miss your chance to talk with the PyTorch experts at the booth. We have a packed schedule experts on different topics available for you to meet at booth #338.
🗓️ Plan your week: Check out the full "Meet the PyTorch Experts" schedule here: pytorch.org/event/nvidia-gtc…
We'll be posting the daily lineups here in this thread all week. See you at the booth! 🤝 @NVIDIADev
Chris Gottbrath retweeted
PyTorch 2.10 is now available, with updates focused on performance, determinism, and numerical debugging for modern training and post-training workflows.
Highlights include Python 3.14 support for torch.compile(), reduced kernel launch overhead in TorchInductor, a new varlen_attn() op for variable-length sequences, and improved tools for tracking numerical divergence.
🖇️ 🔥 Read the PyTorch 2.10 release blog and release notes: hubs.la/Q03_NHfT0
#PyTorch #OpenSourceAI #AIInfrastructure
Chris Gottbrath retweeted
🚀 Join PyTorch Compile H2, 25 Roadmap Q&A!
📅 Sep 3, 25 | ⏰ 9AM PST
📌 RSVP & Submit Questions: docs.google.com/forms/d/15RB…
📍 Location: fb.zoom.us/j/99817055012?pwd…
📜 Roadmap: dev-discuss.pytorch.org/t/me…
Hear from the leads, ask questions, & help shape the future of PyTorch Compile.
Sharing a little of what I have been up to recently (I help organize collaborative roadmap process for the PyTorch team at Meta) -- I've done a lot of roadmaps in my career and I like the way PyTorch does it best.
Meta's PyTorch teams published their actual roadmap documents publicly for the first time: dev-discuss.pytorch.org/t/me…
Our engineers' incentives and objectives are laid bare for folks to read 🙂
While all PyTorch development happens publicly on github, the actual planning and roadmap documents that teams at various PyTorch-affiliated companies write out weren't public, so we decided to change that for increased transparency.
Chris Gottbrath retweeted
Introducing ExecuTorch Alpha ⚡
ExecuTorch Alpha is focused on deploying large language models and large ML models to the edge, stabilizing the API surface, and improving installation processes.
Learn more in our latest blog: hubs.la/Q02vzrrW0
Chris Gottbrath retweeted
Announcing the alpha release of torchtune!
torchtune is a PyTorch-native library for fine-tuning LLMs. It combines hackable memory-efficient fine-tuning recipes with integrations into your favorite tools.
Get started fine-tuning today!
Details: hubs.la/Q02t214F0
Training a model (not something I do all that much these days) and feeling like myself in grad school "my computer is working hard on physics calculations therefore I am working hard, right?"
I just learned (rather belatedly) of Rusty's passing. He was a well known figure in HPC and just a great person to interact with. He'll be missed.
This year marks the 30th anniversary of the first draft of the MPI (Message Passing Interface) standard, and sadly also the passing of one of those original founders, Rusty Lusk, of @argonne National Lab. Vale. #HPC obituaries.neptunesociety.co…
Excited to see PyTorch taking on a structure that will help it continue to grow and scale.
Big announcement: PyTorch Foundation!
PyTorch has large core investments from many companies. So, we're creating a neutral foundation for securing assets and interests.
Technical Governance is separate & secure in a Maintainer model.
Here's more context:
pytorch.org/blog/PyTorchfoun…
Chris Gottbrath retweeted
Introducing torch.profiler! New PyTorch Profiler collects both GPU and framework related info, correlates them, performs automatic detection of bottlenecks in the model, generates recommendations on how to resolve these bottlenecks, and visualize.
Read 👉pytorch.org/blog/introducing…