Jeff Willette (@TheOneJeffrey) profile picture

Jeff Willette

@TheOneJeffrey
jrwillette.com
Joined May 2012
  • Tweets35
  • Following41
  • Followers28
1Media
Photo by Jeff Willette (@TheOneJeffrey)
Jeff Willette retweeted
Minki Kang (@mkkang_1133) profile picture
Minki Kang@mkkang_113319 Sep 2025
Our Agent Distillation paper is accepted at #NeurIPS2025 Spotlight! 🚀 Turn your small LM into a strong agent 💪 Code: github.com/Nardien/agent-dis…
GitHub - Nardien/agent-distillation: Official Code Repository for the paper "Distilling LLM Agent...

GitHub - Nardien/agent-distillation: Official Code Repository for the paper "Distilling LLM Agent...

Official Code Repository for the paper "Distilling LLM Agent into Small Models with Retrieval and Code Tools" - Nardien/agent-distillation

github.com
Minki Kang (@mkkang_1133) profile pictureMinki Kang@mkkang_113326 May 2025
🚨 New preprint! Can small language models (sLMs) solve complex problems like LLMs? We show how to go beyond cloning reasoning—to distill tool-using agent behavior into sLMs as tiny as 0.5B. Meet Agent Distillation: 📄 huggingface.co/papers/2505.1… Here's the details 🧵👇:
Paper page - Distilling LLM Agent into Small Models with Retrieval and Code Tools

Paper page - Distilling LLM Agent into Small Models with Retrieval and Code Tools

Join the discussion on this paper page

huggingface.co
3
12
89
8,309
Noice! Our paper "Delta Attention: Fast and Accurate Sparse Attention Inference by Delta Correction" has been accepted to NeurIPS 2025! See you in San Diego (See part 2 of post for breakdown of our work) arxiv.org/pdf/2505.11254
Image from Jeff Willette (@TheOneJeffrey)
1
7
1
16
1,325
Most sparse attention methods focus on key-sparse attention. We found that key-sparse attention causes a distributional shift in outputs. Therefore, even if you switch to dense attention during decode, the queries and the keys might no longer match...
1
61
We propose to first perform query-sparse attention and combine the output with a generic key-sparse attn. method. This drastically boosts the performance of sparse attention while adding minimal overhead, and its dead simple to implement on pretrained models.
47
Jeff Willette retweeted
Jinheon Baek (@jinheonbaek) profile picture
Jinheon Baek
@jinheonbaek16 May 2025
So excited to share that five papers have been accepted to #ACL2025 🎉 Huge thanks to all my amazing collaborators. I am especially grateful that all the internship projects (which I have worked on during my PhD journey) have all found their way to publication. Its rewarding. 😊
Image from Jinheon Baek (@jinheonbaek)
1
5
69
3,001
There is a HUGE problem with sparse attention! We found it causes a misalignment of queries and keys, so even if you add a dense decode/generation phase, the sparsely encoded context can be forgotten. But wait... We can fix it with a simple correction! huggingface.co/papers/2505.1…
Paper page - Delta Attention: Fast and Accurate Sparse Attention Inference by Delta Correction

Paper page - Delta Attention: Fast and Accurate Sparse Attention Inference by Delta Correction

Join the discussion on this paper page

huggingface.co
3
1
11
661
I just published “Tern: Why it Breaks and How To Fix it” medium.com/p/tern-why-it-bre…
Monitor and get simple alerts on docker container metrics, now from within docker as well! github.com/deltaskelta/docke… #Docker #dockercon

GitHub - jeffwillette/docker-alertd: Monitor docker stats and send alerts

Monitor docker stats and send alerts. Contribute to jeffwillette/docker-alertd development by creating an account on GitHub.

github.com
React Awesome Resume github.com/deltaskelta/react… - A React component to display a resume in the jsonresume.org format #javascript #reactjs #github

GitHub - jeffwillette/react-awesome-resume: ReactJS Resume Component Based on the jsonresume.org...

ReactJS Resume Component Based on the jsonresume.org Format - jeffwillette/react-awesome-resume

github.com
CLI tool to quickly create React JS components for distribution to NPM github.com/deltaskelta/creat… #npm #javascript #reactjs

GitHub - jeffwillette/create-react-npm-component: CLI tool for creating a React JS component for...

CLI tool for creating a React JS component for distribution - jeffwillette/create-react-npm-component

github.com
Docker-Alertd v0.2 released github.com/deltaskelta/docke… with better monitoring features #Docker #Golang

GitHub - jeffwillette/docker-alertd: Monitor docker stats and send alerts

Monitor docker stats and send alerts. Contribute to jeffwillette/docker-alertd development by creating an account on GitHub.

github.com
Check out my project docker-alertd, and alert daemon for docker #Docker #dockermonitoring xitter.gq/awesome_docker/status/…
Awesome Docker List (@awesome_docker) profile pictureAwesome Docker List@awesome_docker17 Feb 2017
Latest Awesome #Docker List commit - Docker-Alertd: Send Alerts Based on Container Usage/Statistics docker-alertd: - is extremely light
Load more