@hgpu

High performance computing on graphics processing units (GPU): AMD/ATI, nVidia, Intel Xeon Phi, CUDA, OpenCL, OpenGL, GPGPU, HPC

hgpu.org/
Joined May 2011
  • Tweets10.7K
  • Following119
  • Followers3.9K
10Media
ATTUNE: Agentic Throughput Tuning for LLM Training #CUDA #LLM #Package hgpu.org/?p=31410

ATTUNE: Agentic Throughput Tuning for LLM Training

Training recipes are tuned by hand for one model, cluster and workload, and every new model, accelerator, framework version or dataset means tuning again. Automated tuners search a fixed space of k…

hgpu.org
4
349
GPU Goldmines: Tunable WebGPU Kernels for Edge LLM Inference #CUDA #OpenCL #LLM hgpu.org/?p=31409

GPU Goldmines: Tunable WebGPU Kernels for Edge LLM Inference

As demand for LLM inference grows and consumer GPUs grow more capable, edge deployment is becoming increasingly attractive. WebGPU is compelling in this setting because it offers cross-platform por…

hgpu.org
4
515
A New Lens on Multi-GPU Communication: API-Level Memory Models for Portable Optimization #AMD #HPC #MemoryModel hgpu.org/?p=31408

A New Lens on Multi-GPU Communication: API-Level Memory Models for Portable Optimization

As network communication increasingly becomes a performance bottleneck for ML and HPC workloads, it is more important than ever to optimize multi-GPU data transfers. Recently, low-level device APIs…

hgpu.org
1
10
878
Understanding NVIDIA Grace-Blackwell Unified Memory Through Placement-Controlled Microbenchmarks #CUDA #Performance #NVIDIA #Blackwell #Memory hgpu.org/?p=31407

Understanding NVIDIA Grace-Blackwell Unified Memory Through Placement-Controlled Microbenchmarks

NVIDIA Grace–Blackwell systems make CPU–GPU memory increasingly unified at the programming level, but unified addressability does not make the memory system uniform. A CUDA managed allo…

hgpu.org
1
7
42
2,631
HPCAgent-Bench: A Framework for Sound Benchmarking of Code-Optimizing Agents, Harnesses, and Skills #Triton #ROCm #HIP #OpenMP #AMD #HPC #Package hgpu.org/?p=31406

HPCAgent-Bench: A Framework for Sound Benchmarking of Code-Optimizing Agents, Harnesses, and Skills

Coding agents are increasingly used to optimize code, but existing benchmarks fix one accelerator, programming model or harness, so they cannot isolate what each component contributes or show wheth…

hgpu.org
6
513
AgentGarten: Code Worlds for Evolving Agents #CUDA #Triton #LLM #Package hgpu.org/?p=31390

AgentGarten: Code Worlds for Evolving Agents

Interactive virtual worlds allow agents to learn through exploration and interaction. What agents can learn is bounded by the environments they practice in, which must be faithful, with consistent …

hgpu.org
3
301
A Unified Intermediate Representation and Execution Architecture for Heterogeneous and Distributed Neural Simulation #CUDA #Package hgpu.org/?p=31389

A Unified Intermediate Representation and Execution Architecture for Heterogeneous and Distributed...

Equation-based neural modeling offers flexibility, but executing models across processors and nodes requires explicit control of order, ownership, and numerical behavior. We present brian2-atlas, a…

hgpu.org
4
1,095
ActTune: Action-Aware Precision and GPU Operating-Point Adaptation for Energy-Efficient Vision-Language-Action Inference #LLM #Robotics hgpu.org/?p=31388

ActTune: Action-Aware Precision and GPU Operating-Point Adaptation for Energy-Efficient Vision-La...

Vision-language-action (VLA) policies repeatedly invoke inference to control robots, making graphics processing unit (GPU) energy a recurring cost of task execution. Reducing energy per inference c…

hgpu.org
1
185
GPU Acceleration of Awkward Arrays: Using Python cuda.compute #CUDA #Physics #Python hgpu.org/?p=31387

GPU Acceleration of Awkward Arrays: Using Python cuda.compute

Awkward Array is a widely used library in high-energy physics (HEP) for representing and manipulating nested, variable-length data in Python. Previous CHEP contributions have explored GPU accelerat…

hgpu.org
1
8
626
KernelLens: Deploying Triton Kernels to C++ Inference Engines #Triton #CUDA #PTX #Package hgpu.org/?p=31386

KernelLens: Deploying Triton Kernels to C++ Inference Engines

Writing custom GPU kernels using OpenAI Triton is a common approach for authoring deep learning operators. However, deploying models containing Python @triton.jit kernels into production C++ infere…

hgpu.org
1
7
485
[Book] Introduction to Computer Vision #Book #ComputerVision #OpenCV #Python hgpu.org/?p=31344

Introduction to Computer Vision

This book presents a code-first introduction to computer vision, spanning classical 2D image processing, classical 3D vision, and deep learning. Organized as 44 short chapters across three parts, t…

hgpu.org
1
10
643
KernelZero: Co-Evolving Proposer and Coder for Continuously Improved GPU Kernel Generation #CUDA #Triton #LLM #Package hgpu.org/?p=31343

KernelZero: Co-Evolving Proposer and Coder for Continuously Improved GPU Kernel Generation

High-performance GPU kernels are essential to modern machine learning systems, yet automatically generating kernels that are both correct and efficient remains challenging. Existing LLM-based appro…

hgpu.org
2
14
1,076
AIM: Agentic Idea Management for Automated Research #CUDA #LLM hgpu.org/?p=31342

AIM: Agentic Idea Management for Automated Research

Frontier LLMs are increasingly used to automate scientific research through iterative search. We distinguish idea-driven search from solution-driven search and identify three core challenges: organ…

hgpu.org
2
1
13
616
Automatically Generating ML Compiler Backends from Tensor Accelerator ISA Descriptions #ISA #CodeGeneration #ML #Compilers #Package hgpu.org/?p=31341

Automatically Generating ML Compiler Backends from Tensor Accelerator ISA Descriptions

Machine learning (ML) compilers play a key role in enabling high-performance implementations of ML workloads. These compilers use existing CPU and GPU backends to generate device-specific code. In …

hgpu.org
2
10
735
AI as a Compiler: Compiling Triton kernels without the Triton compiler #Triton #PTX #LLM hgpu.org/?p=31340

AI as a Compiler: Compiling Triton kernels without the Triton compiler

Compiler backends are expensive to build and maintain as programming models, workloads, and accelerators evolve. We investigate whether large language models can replace the conventional optimizing…

hgpu.org
2
20
1,421
KernelOPT: Dispatch-Aware Agentic Search for GPU Kernel Optimization #Triton #CUDA #DeepLearning #LLM hgpu.org/?p=31275

KernelOPT: Dispatch-Aware Agentic Search for GPU Kernel Optimization

Deep learning inference and training performance depends critically on GPU kernel efficiency. Modern compilers such as PyTorch Inductor automatically generate GPU kernels from high-level model code…

hgpu.org
1
2
15
671
Xtrace: High-Fidelity GPU Intra-Kernel Tracing via Binary-Level Instruction Splicing #LLM #Performance hgpu.org/?p=31274

Xtrace: High-Fidelity GPU Intra-Kernel Tracing via Binary-Level Instruction Splicing

Modern GPU kernels fuse increasingly more work into a single kernel, and intra-kernel tracing has become the mainstream method to profile them. Tracing inserts probes into the kernel to record its …

hgpu.org
1
20
1,421
Microarchitectural Memory Bandwidth Saturation, KV-Cache Paging Dynamics, and Time-to-First-Token Latency: A Comparative Benchmark of vLLM, TensorRT-LLM, and FlashAttention-3 on NVIDIA Hopper H100 versus AMD Instinct MI300X #CUDA #ROCm #AMD #vLLM hgpu.org/?p=31273

Microarchitectural Memory Bandwidth Saturation, KV-Cache Paging Dynamics, and Time-to-First-Token...

The commercial and scientific utility of Large Language Models (LLMs) hinges directly on the efficiency of hyperscale inference serving infrastructure. LLM inference operates across two fundamental…

hgpu.org
3
1
11
904
Accelerating the Mitigation of LLM Inference Nondeterminism Across GPU Architectures #CUDA #LLM #Package hgpu.org/?p=31272

Accelerating the Mitigation of LLM Inference Nondeterminism Across GPU Architectures

Large language model (LLM) outputs are expected to be reproducible under greedy decoding, yet in practice the same model, prompt, and software stack produce different outputs on different GPUs. The…

hgpu.org
1
3
11
708
pytest-gpu-proof: Enabling Cloud-CPU Continuous Integration for GPU Code with Local GPU Attestation #Python #Cloud #CI #Package hgpu.org/?p=31271

pytest-gpu-proof: Enabling Cloud-CPU Continuous Integration for GPU Code with Local GPU Attestation

GPU acceleration is now routine across robotics, but cloud-hosted GPU continuous integration (CI) runners are expensive, resulting in severe under-testing of GPU-accelerated code. We present pytest…

hgpu.org
1
3
243
Load more