all sorts of engineering
zsh 💻
Joined July 2013
- Tweets565
- Following135
- Followers67
- Likes6.7K
alberto retweeted
In March, I emailed Jensen when @inferact's compute provider took back our capacity, leaving us unable to develop @vllm_project for Blackwell. He replied within the SAME minute and had the issue resolved overnight.
The entire NVIDIA team, led by Jensen, operates like an unstoppable armada. Jensen moves fast, so is his "exo-skeleton". Everyone in the industry respects and trusts him. That's how you push open intelligence forward.
Thank you Jensen, for saving Inferact and vLLM.
In June @NaderLikeLadder emailed Jensen about a crazy idea we had over dinner to make Local AI 10x better on NVIDIA hardware.
Jensen replied in 7 minutes. He saw this as important work, and approved a war room for us to work on this at NVIDIA HQ.
For a month, NVIDIA HQ was the @exolabs satellite office. We were overwhelmed by the support we got. We were back to back the first week with smart NVIDIANs coming in to our war room to help us. Creators of CUDA, Dynamo, people who built NVIDIA.
Long story short with NVIDIA we managed to improve their Local AI playbooks by 9.6x (almost the 10x goal we set out with). We’ll be publishing a whitepaper on this work.
We were just a small startup with a crazy idea. Jensen took a bet on us. Jensen cares deeply about Open Source AI and Local AI.
I don't believe Jensen is replaceable, he's a 1 of 1 human, but I hope NVIDIA will continue to aggressively support Open Source AI and Local AI, and continue to take bets on small startups with crazy ideas.
Thank you Jensen
alberto retweeted
Getting benchmark numbers is easy. Getting the right ones is hard. KV cache offloading can cut TTFT from 60s to 10s and lift throughput up to 5.8x, but only when your working set and storage tier line up. Here's how we benchmarked it, caveats and all: developers.redhat.com/articl…
alberto retweeted
Join us to learn more about about the state of open source inference!
Hola, Madrid. vLLM and llm-d are coming to town: Red Hat and @nvidia are hosting an evening on open-source inference, with @vllm_project, LLM Compressor, vLLM-omni, safe agents, and llm-d on Kubernetes.
November 12 at MOA Institute. Register: luma.com/gl1d2loa
alberto retweeted
Today at PyTorch Conference 2026, we announced llm-d 1.0! 🎉
Distributed inference is too complex to solve behind closed doors.
17 months in, llm-d is now a @CloudNativeFdn project powering fleets worldwide.
Here’s what’s inside 1.0 🧵👇
alberto retweeted
"No man ever runs an experiment on the same infra twice, for it's not the same infra and he's not the same man." - Heraclitus
alberto retweeted
Right. Dario was already claiming that GPT2 was too dangerous to open source back in 2019.
I made fun of them then.
Everyone should make fun of them now.
alberto retweeted
Terence Tao just posted this really insightful thread which seems to have been written in reaction to the Navier Stokes announcement. Please give it a read. mathstodon.xyz/@tao/11723732…
alberto retweeted
It's honestly insane that Claude code is closed source. It's a JavaScript bundle that makes API calls and all your competitors are open. Are you a model harness company or an AI company
alberto retweeted
vLLM Office Hours #56 recording is up:
- What's new in vLLM 0.27
- Running Codex and Claude Code CLIs on open models
- Speculators v0.6 and v0.7 update
- GuideLLM v0.7.0: benchmarking reasoning, tool calling, and agentic traces
Watch + see slides: youtu.be/dQAByb7N5QQ
![[vLLM Office Hours #56] - GuideLLM v0.7: Benchmarking Reasoning and Agentic Traces - August 20, 2026](https://pbs.twimg.com/card_img/2119925513795801088/HNUD1Vdn?format=jpg&name=800x419)
alberto retweeted
On behalf of the vLLM community, @inferact just signed the open letter. Let's build an open ecosystem together!
Inferact is proud to co-sign. We were founded to grow @vllm_project into the world’s leading open-source inference engine, making open-weight AI faster, cheaper, and accessible across hardware and clouds so intelligence can be on tap for everyone.
alberto retweeted
extremely unprofessional. if kimi wants to make it as a frontier lab, they need to act like one: perhaps silently route people to worse models, and maybe write a blog post about the collapse of humanity
Kimi K3 has received far more love than we expected, and our GPUs are feeling it.
Over the past 48 hours, demand has pushed close to the limits of our current capacity. To protect the experience of existing subscribers, we're temporarily pausing new subscriptions and prioritizing compute for current members. Existing subscribed users are not affected.
We're adding capacity as fast as we can and will reopen new subscription spots in batches.
Going forward, we'll also split membership into two more focused plans: Kimi Membership for Kimi Web, App, and Work; and Kimi Code Membership for coding workflows. This will help us match compute more precisely and keep the experience stable.
Thank you for your patience and understanding!
alberto retweeted
Introducing NEO’s 25 Degrees of Freedom, tendon-driven hands — nearing or surpassing human-level dexterity, strength, speed, and reliability.
For seventy years, robotics worked around the hand problem. The humanoid bet is the reverse: it lives or dies at the fingertips.
if you're running kv-offload experiments you're probably not offloading anything, the cache just chills in hbm the whole time 🥶 here's the simple math to size your experiments albertoperdomo.me/posts/kv-c…
52