Darshan (@neuronfitting) header
Darshan (@neuronfitting) profile picture

making gpu's that go brrrrr @AgraniLabs

Joined December 2024
Had this saved, just in case
28
Replying to @maharshii
Happens to the best of us
1
6
289
Replying to @maharshii
Reminds of this
1
1
57
4,759
Register files used to be the ultimate bottleneck for Tensor Core accumulators. Introducing Blackwell’s Tensor Memory (TMEM), a completely new address space inside the SM that isolates the accumulator entirely from the register file.
1
32
Hopper and Blackwell demand that you abandon standard thread copy loops. Introducing the Tensor Memory Accelerator (TMA) forklift and phase-based mbarrier synchronization. One thread dispatches; the hardware engine moves the data asynchronously.
1
19
Shared memory has 32 physical banks. If you access a tile via column-major strides, you hit a catastrophic 32-way bank conflict, dropping your throughput to 3.125%. Introducing the exact math behind Swizzle<B,M,S> to stagger your data like bricks.
1
1
1
105
"What I cannot create, I do not understand." Introducing: The Feynman GPU Lectures. Your H100s and B200s are running at a fraction of their peak utilization because your custom kernels are written with massive hardware bottlenecks. If you don't know what tcgen05. mma does at the wire level, you're lighting compute efficiency on fire.
1
2
67
Just managed to 5x the inference throughput of Qwen 3.5 on a B200 (from ~16 tok/s to ~83 tok/s) by ripping out PyTorch overhead and building custom fused kernels. I wrote a full deep dive on how to move from memory-bound PyTorch to compute-bound
3
2
260
Solution: Kernel Fusion. I wrote a custom Triton kernel for Qwen's Zero-Centered RMSNorm. But I didn't stop at the standard tutorial fusion. I broke the nn.Module boundaries to fuse the residual addition across layers, exactly how production engines do it.
1
2
52
Looking at the profiler traces, the compute cores were starving. Standard PyTorch separates residual + hidden_states and RMSNorm. This forces the GPU to do multiple round trips to HBM just to execute basic math. It's completely memory-bandwidth bound.
1
1
31
the beautiful, beautiful art of profiling kernels
3
56
Want the performance of CUTLASS/CuTe without the headache of C++ template metaprogramming? I just published a guide on cute-dsl, a library that brings advanced GPU memory layouts and vectorization directly to a Pythonic interface. Here is a breakdown of what the post covers: - Layouts & Strides: Visualizing how multi-dimensional logical coordinates map to flat memory. - Zipped Divides: Why regrouping data into (Tile, Grid) hierarchies is the best way to partition your data. - Vectorization: How to use zipped layouts to easily emit 128-bit hardware loads (like ld. global.v4) with zero raw pointer math. Check out the full deep dive below:
1
1
57