Roberto Abdelkader retweeted
We're working on a wake-word detection model at @desertantlabs that is small enough to run on a 3$ ESP32-S3 microcontroller, with custom wake words tuned to your voice, set at runtime 🤏🗣️
Roberto Abdelkader retweeted
No more scrolling endless logs during Linux boot!
Visualize systemd services come to life in real time.
Powered by @ratatui_rs 🐀
codeberg.org/wvhulle/bootvin…
#rustlang #ratatui #linux
Four pull requests merged across SGLang and vLLM-Ascend today under the module name `qwen4_exp.py` (`Qwen4Exp`) just exposed how Alibaba's next-generation Qwen 4 / Qwen 3.8 architecture works under the hood.
First, `Qwen4Exp` uses massive Per-Layer N-gram Embedding (PLE) tables that don't even live on GPU VRAM—they sit in pinned CPU host memory shared across workers and get pulled on the fly by a masked Triton kernel alongside Gated Delta Net (GDN) recurrent layers and Qwen Sparse Attention (QSA). Second, SGLang just added a native engine feature specifically for `Qwen4Exp` that lets you hot-swap MoE experts in and out of GPU memory at runtime without restarting the server.
Offloading massive N-gram lookup tables to cheap system RAM (just like DeepSeek's Engram tables) while keeping the GPU focused on sparse attention and hot-swappable MoE experts is clearly the new blueprint for frontier models this year.
github.com/sgl-project/sglan…
![[Feat] moe: hot-replace fixed-capacity Qwen4 experts through native Engine by SuperGoodGame · Pull...](https://pbs.twimg.com/card_img/2118592491817746432/IDhPfSOM?format=jpg&name=800x419)
2
57
Roberto Abdelkader retweeted
Having EDR software in your laptop in today’s post-LLM era is just a ticking time bomb. I do not trust EDR vendors to not get compromised. We already had this lesson with SolarWinds but we’ve all seem to have forgotten
Roberto Abdelkader retweeted
おい、始まったな！！！！！！
「モデル内部に存在する知識の非対称性を活かして学習信号に応用すると、強力な外部教師に依存することなく、基盤モデルの事後学習が可能となる」
arxiv.org/abs/2610.32770
Roberto Abdelkader retweeted
Introducing Kimi K3.1 Pro: our most capable 3T-class open-weight model.
🔹 1M-token context, 2.8T total / 104B active params (MoE), Multimodal.
🔹 Full-project coding: games in Unreal, Unity & Godot.
🔹 Excels at parametric 3D modeling in Blender & CAD.
🔹 More reliable at finance work: Accurate modeling with hallucination down by 65%.
🔹 Matching the strongest proprietary models across coding, agentic and knowledge benchmarks.
Kimi K3.1 Pro and K3.1 Pro Highspeed (2–3× faster) are now live on Kimi.ai, Kimi Work, Kimi Code, and the Kimi API.
Open weights by Nov 4, 2026.
🔗 API: platform.kimi.ai
🔗 Tech blog: kimi.ai/blog/kimi-k3-1-pro
I just supported Niko on @buymeacoffee! 🎉
You can support them here — buymeacoffee.com/strataengin…
1
24
Roberto Abdelkader retweeted
New on our Frontier Red Team blog: using Claude to formally verify software at large scale.
Formal verification can prove that code is free of certain bugs, but historically it has only been feasible for small codebases (or required a lot of expert time).
With AI models, however, this is beginning to change. In collaboration with @theoremlabs, we used Claude to prove security properties in over one hundred real-world open-source libraries over the span of a few weeks.
Read our research in full here: anthropic.com/research/forma…
Roberto Abdelkader retweeted
🚨ThinkingCap: Qwen3.8-Flash-Next is out!
The most token-efficient Qwen3.8-Flash-Next available.
🚀Up to 51% fewer thinking tokens, 38.5% on average
🎯Negligible accuracy loss 1.03pp
🏢ThinkingCap PRO: 60% fewer thinking tokens on average - Contact our enterprise[at]bottlecapai [dot]com
📄Blogpost: bottlecapai.com/post/thinkin…
🤗Hugging Face: huggingface.co/bottlecapai/T…
🔗GGUF: huggingface.co/bottlecapai/T…
Roberto Abdelkader retweeted
You've gotta patch your dependencies super fast because they're full of vulns (PHP, Node, eMudhra, etc), but not _too_ fast because they're full of backdoors (LiteLLM). Good luck!
Roberto Abdelkader retweeted
You asked for speaker diarization, so we built it.
Who split this 43-minute podcast by speaker in 4.9s on an M5 chip, including the moments two people talk at the same time. The recording never left the machine.
Who is in our Swift, Kotlin and JavaScript SDKs, free up to 100k monthly active devices: desertant.com/models/who
Roberto Abdelkader retweeted
Don't know if I should spend less time coding and more on farming engagement, but either way here's a neat cat simulation.
It's not entirely accurate (still needs to model viscosity and surface tension), but it was more of a stress test anyway, made by a new 1.66 bpw quant of Qwen3.8-Flash-Next running on one RTX 5090; no experts offloaded to the CPU, only vision tower in pinned memory (negligible overhead), and embeddings on disk.
Stable inference throughout, no reasoning loops, just smooth sailing at 200-300 t/s (and a lot of compacting since one 32 GB GPU still doesn't leave a lot of room for context, even with cache quantization).
Get yours today at huggingface.co/turboderp/Qwe…
Roberto Abdelkader retweeted
running NVIDIA’s Dory1.0 4B on a single RTX 3060. 12GB VRAM, 16GB RAM. (config below)
256K context. ~50 tok/s decode, ~1580 tok/s prefill.
dory is a looped 4B. it reuses one shared block of layers for each loop, and last-loop KV keeps the cache at one-loop size, so the full 256K fits. weights, KV and embeddings all sit in VRAM. needs llama.cpp#30306, not upstream yet.
the -ot line keeps the tiny scale tensors on the GPU. without it decode was 38 tok/s.
benchmarks next!
config:
llama-server
-m NVIDIA-Nemotron-Labs-Dory1.0-4B-Q5_K_M.gguf
-ngl 999 -c 262144 -fa on --jinja
-np 1
--cache-type-k q8_0 --cache-type-v q8_0
-b 2048 -ub 1024
--override-kv “dory.num_loops=int:2,dory.recurrent_kv_cache_mode=str:last_loop”
-ot “blk.(scale|alpha).=CUDA0,token_embd.*=CUDA0”
--load-mode none -t 1 --cache-ram 0
--temp 1.0 --top-p 0.95 --top-k 0 --min-p 0.0
--repeat-penalty 1.0
--reasoning on