Software Engineer of many stacks and a master of one. Senior Solidity dev @Token_Logic Agent overlord by night, creator of https://xitter.gq/t.co/VHBeVYpRn7
Montenegro
Joined February 2011
- Tweets6.5K
- Following1.6K
- Followers1.3K
- Likes38.8K
Pinned Tweet
I gave my agents a memory of all my meetings and work.
Five open-weight AI models, 6.8 GB total, all running locally on my MacBook. They transcribe my meetings, read my screen, and find anything I said or saw on it.
LokalBot 0.9.2 is out today. I've been quietly iterating on this for a while, rebuilding it for my needs, and it looks much better now thanks to Claude Opus 5.5's excellent design skills.
Meet my all-star AI team:
Nemotron 3 → who's talking
Qwen3-ASR 1.7B → transcribes
LFM2.5 1.2B → finishes my sentences
Harrier 0.6B → search by meaning
Qwen3.5 4B → writes the recap
Claude Code and Codex can pull context from my meetings over LokalBot MCP or CLI.
Free and open source. No subscriptions necessary.
Huge thanks to @NVIDIAAIDev , @Alibaba_Qwen and @liquidai for the open weights and @huggingface for hosting them!
Links below:
ALT Infographic: "The little AI team inside your Mac." A robot next to a MacBook running LokalBot app using LLMs, with six AI model cards: Nemotron 3 (Core ML, 0.2 GB), Qwen3-ASR 1.7B (MLX 8-bit, 2.47 GB), LFM2.5 1.2B (GGUF Q4_K_M, 0.73 GB), Harrier 0.6B (GGUF Q8_0, 0.64 GB), Qwen3.5 4B (GGUF Q4_K_M, 2.74 GB) and Apple Vision (built into macOS). 6.8 GB, all local.
Consumer memory bandwidth tier list aka LLM decode speed
tok/s = bandwidth ÷ model size
S:
RTX 5090 1.8 TB/s (32GB)
M5 Ultra 1.2 TB/s (512GB)
A:
M3 Ultra 819 GB/s (512GB)
M5 Max 614 GB/s (128GB)
B:
M5 Pro 307 GB/s (64GB)
DGX Spark 273 GB/s (128GB)
Strix Halo 256 GB/s (128GB)
C:
DDR5 dual channel ~100 GB/s
DDR4 dual channel ~50 GB/s
D:
Samsung 9100 Pro SSD 14 GB/s
Samsung 990 Pro SSD 7 GB/s
27B at Q4 (16GB):
- 110 tok/s on a 5090
- 75 on M5 Ultra
- 6 on DDR5
- <1 off an SSD
Shipped autocomplete for any Mac app.
Been dogfooding LokalBot for a few releases, finally good enough to share!
Gemma 4 E2B Q6 on-device via llama.cpp
prompt = text before cursor + local context
If unsure of the next word? no suggestion
Tab accepts word by word
Free and open source, on-device by default.
Download from lokalbot.com or
brew install --cask stevyhacker/tap/lokalbot
Low on RAM? You can use an external server or your agent subscription. Just check the Models tab in the Settings.
Local AI communities like r/LocalLLaMa remind me of the peak of Android custom ROMs and tinkering with your phone flashing new software on a weekly basis.
Tech hobbyists always find something new to tweak.
True
not only am i not worried about the crypto community's ability to handle the upcoming ai/quantum threat, i will venture to say that in the next 2 yrs the world will realize crypto is far better prepared for it than the rest of the internet and finance. we've had a trillion dollar *irreversible* honey pot for a decade, producing a far more battletested system and adversarial security culture than pretty much anywhere else on earth.
macOS permissions in the agentic era make no sense, you just get desensitized by all the constant permission asks and dialogs you start clicking yes to everything.
Only through sheer luck I migrated my personal side projects from Railway a few months ago and didn't install that OVHcloud cli as I mostly use my dedicated server only.
Wild west on the web right now and the agents are having a shooting contest.
October has been an absolute nightmare for software
> Railway - customer secrets exposure
> LiteLLM - malicious PyPI + Docker releases
> Zeabur - infrastructure breached
> OpenCode - AI API keys exposed
> OVHcloud CLI - official binaries backdoored
> Rank Math PRO - malware in official plugin
> TeamUpdraft - customer data compromised
> ConnectWise - cloud backup identities compromised
> Tensorlake - npm package infected
> SubQuery - backdoored npm package
> GhostAction - GitHub repos with malicious workflows
> PHP - critical RCE vulnerabilities
> Node.js - security vulnerabilities patched
> eMudhra - certificate authority keys compromised
> Tailscale - eMudhra incident
At this point, the biggest security risk might not be the code we write, but the code we trust...
Did a few Malazan inspired image generations and the castle one is really impressive to me, would never guess it was AI generated honestly.
🤖 Made with AI
I've heard it's popular to port apps to Rust these days so I did a few prompts via my phone's Claude app over the weekend and got 150k lines of LokalBot's Swift/macOS codebase to a GPUI based desktop app for Linux and Windows.
Only possible because I made a personal compute network tied with Tailscale and Claude remote control so everything is always on and easy to operate with just a bit of oversight.
Full writeup in the link below.
ALT Diagram of my setup: my phone controls Claude Code sessions on a MacBook, which never builds. The agents build and test over SSH on a rented 16-thread Ubuntu server and a Windows gaming PC on my Tailscale network, with GitHub Actions as fallback CI. 11 minutes of Mac use on Sunday; CI runs went from 32 to 9 minutes.
The full story, and LokalBot install builds for all platforms available at the end here:
bogosavljevic.me/blog/posts/…
If you are on Windows/Linux (whatever fancy distro @omarchy btw or good old Ubuntu) please report bugs.
P.S. the real goat is @github for subsidizing all this CI usage for open source repos.
Cathedrals of the 21st century
We trained our Muse foundational models, including Muse Wonder, at Prometheus, our gigawatt-scale supercluster in New Albany, Ohio.
Across a supercluster with hundreds of thousands of GPUs, even one underperforming host can slow an entire training run. We engineered Prometheus for resilience at scale, achieving 85%+ goodput on pretraining, 90%+ on reinforcement learning and ~47K GPU hours mean time between interruptions during Muse Wonder training.
Take a look at the tech powering Prometheus: go.meta.me/4yXXApZ
Had no idea about Google Colab and now I can test a RTX PRO 6000 Blackwell with 96 GB VRAM for free.
Most devs on the $20 Google AI Pro plan still don't realize Google quietly bundled Colab Pro (200 Compute Units/mo) directly into the subscription:
- Hardware unlocked: 80GB NVIDIA A100 GPUs (167GB System RAM), 96GB Blackwell G4 GPUs, L4 GPUs, and Trillium v6e-1 TPUs
- Monthly runtime: At ~6.77 CU/hr, your 200 monthly units give you ~30 hours/mo on an 80GB A100
- Agent bonus: With Google's Colab CLI, Antigravity or Claude Code can call those A100s straight from your terminal.
Felt this personally, it's very hard to push back against AI as it's simply superior in argumentation and can defend any position.
It takes a field expert to spot it's providing an incorrect answer or a misleading output so for coding in languages I'm familiar with I can still spot it's wrong but for everything else it's your pick Opus my overlord.
X algo is getting so bad again with promoting AI fun fact/video accounts instead of legit raw thoughts.
Will have to only use the Following tab from now on.
I've never really understood how LLM inference works and why memory bandwidth is all everyone is talking about when GPUs are concerned.
Essentially for each token generated each model weight has to go from RAM through the chip cache memory too and that's the main bottleneck for higher tokens per second.
Two interesting things I did with Claude today:
- Changed the system font of my android phone to Google Sans via adb and some scripts.
- Ran a self security audit of all my devices and found some very important things to address.
