Whatever AI infra means to you, we’ve got it covered. See for yourself: the SGLang Summit agenda is live → sglang.io/summit 🗓️ Nov 12–13 · 📍 Fort Mason, SF 🎟️ Early bird ends Oct 20 Two days of open models, AI hardware, recursive self-improvement, agents, robotics, and the systems behind them. Featuring @LipBuTan1 (Intel), @lilianweng (OpenAI), @soumithchintala (Thinking Machines), @AravSrinivas (Perplexity) and researchers and builders from across the AI ecosystem. Plus hands-on workshops on model training and serving decision models, hardware Expo, and plenty of community events. What does AI infra mean to you?

Today we launch SGLang-TorchTPU, built with @Google TPU team and Meta @PyTorch team. We’re excited to see it powering production inference at Meta! SGLang-TorchTPU is a TPU plugin for SGLang that brings: - Top open models on TPU: Muse Spark Open, GLM-5.3 Flash, Kimi K3, Ling 3.1 Flash, MiniMax H3 - SGLang’s production features for large agentic workloads - Custom MegaKernel: Muse Spark Open hits 770 tok/s decode at bs=1 on InferenceX 8K/1K SGLang-TorchTPU is available now: github.com/sgl-project/sglan…

We have been working on making Beam easier to customize at @reflection_ai. Seeing the recipes and integrations go live feels pretty amazing. We're putting the code, training configs, and examples out there so people can start building on Beam. Here's what's in the cookbook: Data generation: Use Beam to create training data, including software-engineering trajectories that pass tests and reasoning examples for tool calling. github.com/reflection-oss/co… Fine-tuning: Adapt Beam to your data and tasks with LoRA, QLoRA, or full fine-tuning. github.com/reflection-oss/co… Distillation: Use Beam to teach a smaller model. We've included recipes for coding trajectories and tool-calling reasoning. Coding: github.com/reflection-oss/co… Tool calling: github.com/reflection-oss/co… Reinforcement learning: Post-train Beam against verifiable rewards. github.com/reflection-oss/co… Huge thanks to our partners and the teams behind these integrations: • NeMo AutoModel, @NVIDIAAIDev: SFT with LoRA and full fine-tuning. docs.nvidia.com/nemo/automod… • Megatron Bridge, @NVIDIAAIDev: Large-scale full fine-tuning. github.com/NVIDIA-NeMo/Megat… • Axolotl, @axolotl_ai: LoRA, QLoRA, and full fine-tuning. github.com/axolotl-ai-cloud/… • HF TRL, @huggingface: Training smaller models on Beam-generated data. github.com/huggingface/trl • Miles, @radixark: RL post-training. miles.radixark.com/docs/mode… • NeMo RL, @NVIDIAAIDev: RL post-training on NVIDIA hardware. github.com/NVIDIA-NeMo/RL/bl… • SkyRL, @NovaSkyAI: LoRA and full-parameter RL. docs.skyrl.ai/docs/examples/… Each recipe has its own instructions, dependencies, sample inputs, and smoke test. I'm excited to see people run these experiments, adapt Beam to their own work, and contribute recipes we haven't thought of. Huge thanks to @Nazeri2010, @bhutanisanyam1, @kathleen_mhatre, @FGbelidji, @joespeez, and to Thibaut, Matthias, Travis, Kai, and Yoni for making this happen. Explore the cookbook: github.com/reflection-oss/co… Access the model: platform.reflection.ai Documentation: developers.reflection.ai Provide feedback on Discord: discord.com/invite/PxUfdsNny

We use Miles from @radixark as the RL framework, with @sgl_project samplers and a Megatron-LM trainer that updates the policy in BF16 on B200 GPUs. There are two setups: 1. Mixed samplers that span six kinds of accelerators and three precisions (BF16, FP8 and NVFP4). 2. Homogeneous samplers that run only on B200s with BF16 weights and a BF16 KV cache.

We brought SGLang to NVIDIA Vera Rubin and accelerated Kimi K3 inference. Working closely with @NVIDIA, we optimized attention, MoE, and speculative verification kernels on early-access Rubin hardware. Highlights: • Up to 20% faster FP8 MLA at batch 1 / 128K context • 20% faster KDA verification, with bitwise-identical output • 5.9% end-to-end inference speedup from MoE tail fusion, removing 276 kernel launches per decode step SGLang also powers rollouts for Miles' end-to-end RL training on Rubin, including agentic RL with 64 concurrent sandboxes on the Vera CPU. Full results and engineering details 👉 lmsys.org/blog/2026-10-09-ve…

Our training framework is built on top of miles. Training runs on Modal's on-demand GPU clusters and we use its serverless GPUs to flexibly scale our inference workloads Our sandboxing infrastructure is built on Kubernetes and able to handle millions of concurrent rollouts