Piyush Katariya retweeted
Storing vectors is easy. Retrieving the right evidence is harder.
Introducing searchgres.js: BM25, semantic + hybrid search in Postgres, with hierarchy, metadata, regex + time filters.
Filtering belongs inside retrieval.
tsdb.co/searchgres-x
Piyush Katariya retweeted
🚀 Introducing DeepInfer, our in-house inference framework, now fully open source!
Powering DeepSeek-V4.1-Flash at scale with high throughput and low latency. We’re sharing our production stack to help the community lower compute costs, reduce energy use, and make AI more affordable.
🔗 github.com/deepseek-ai/DeepI…
1/4
Piyush Katariya retweeted
Today we're open-sourcing pg_vectorsearch, a new vector search extension for PostgreSQL.
The performance improvements over HNSW are pretty remarkable. At 100 million vectors:
◆︎ Faster queries, 4× less RAM: 6 ms vs. 8 ms at 95% recall, using 128 GB vs. 512 GB RAM. HNSW couldn't even build on 128 GB.
◆︎ 7× smaller indexes: 25 GB vs. 180 GB at 100M vectors.
◆︎ 50× faster index builds: 49 minutes vs. 42 hours on identical hardware.
And at 1 million vectors, PRISM delivers 15 ms queries at 99% recall on just 2 GB RAM, versus HNSW's 392 ms on 4 GB.
What leads to these improvements? Rather than traversing an HNSW graph with random memory accesses, our new PRISM index partitions vectors based on distance, compresses them using RaBitQ, and scans them efficiently using SIMD vectorization techniques. This takes better advantage of modern CPUs and memory locality.
This is actually our second open-source vector index. We learned a lot from building DiskANN-based pgvectorscale, including some important limitations of graph-based approaches. Those lessons convinced us there was a better architectural foundation, so we started over.
pg_vectorsearch is still in alpha, with more engineering ahead. But we believe fast, scalable vector search should live alongside your relational data, without requiring a separate vector database.
Released under the permissive PostgreSQL License, just like our pg_textsearch extension for BM25 keyword search. Together, they provide the building blocks for hybrid vector + keyword search in PostgreSQL.
Feedback and contributions welcome!
Code and blog post below.
⚡️⚡️⚡️
TensorFold 1.0.7 has landed with a major overhaul to Living Weights.
• Each fact you teach through /learn becomes about ten new neurons inside the model's own weights, learned about 4x faster, with a live 3D view of what it has learned. Macs and DGX Spark.
• Structured output and required tool calls on the native server, with drafting still on.
• Qwen3.8-27B serves concurrent requests, and sampled replies draft too: 125 tok/s vs 20 on code on an M3 Ultra.
• RTX 50-series and RTX PRO 6000 Blackwell support, NVFP4 Nemotron on DGX Spark, and faster GLM-5.3-Flash on two Macs.
• Various other bugs and issues squashed
I would like to thank the people who reached out to me and gave me access to their hardware to speed up testing. 🫶🏼
Thank you to everyone who sent PRs, reports and receipts, and to everyone using it: BobClawblaw BlivionIaG jschmied ZackSample ThinkOffApp jayleaton chaog992 Nipale-ai
@volatilemarkts @MiaAI_lab @petruspennanen @plotarmordev
github.com/ashhart/TensorFol…
Piyush Katariya retweeted
We are currently working on Qwen3.8-27B-Coder390-EfficientThink-Opus5.5-GPT6Astra-Grok4.7-DSV4Pro-K3-SFT-RLOO-MTP-DFlash2, the model with the longest name. We only need to ablate 5 weights (i.e., 3 layers) to achieve the ablation effect... We’re still testing to see if it can be made even smaller.
TensorFold 1.0.3 lands with a few cool things.
- Living Weights (beta with Nemotron on Metal) and coming to CUDA in 1.0.4
- Nvidia Ampere (beta) for Nemotron, and native Qwen3.8-27B with DFlash2
- Conversation prefix caching
- Various bugs fixed and improvements
Twelve contributor PRs helped make this happen, so a massive thank you to @niklaslenz_ai and GitHub contributors akol1, jschmied, chaog992, AdrianBinDC and BobClawblaw, and everyone testing, reporting bugs and helping us improve TensorFold.
FYI, Living Weights is in very early beta and has sharp edges, but I think I’m on to something.
I’d love community feedback to make it better: github.com/ashhart/TensorFol…
@plotarmordev @petruspennanen @MiaAI_lab @volatilemarkts
tensorfold.dev
Piyush Katariya retweeted
Opus 5.5 has one very serious, catastrophic problem that nobody is talking about: it's unfit for code review. It's been made so "safe" and so restricted that, out of a hundred confirmed critical issues, it will flag two at most. Opus 5.5 writes completely insecure software that can't be shipped to production
I'm convinced Anthropic has gone overboard with restrictions and safety. This needs to be fixed urgently
Keep this in mind!
Piyush Katariya retweeted
You can now train your own Decision model with our free notebook! 💡
Qwen3.5-4B will generate decisions instead of text on just 8GB VRAM locally.
Learn to data prep (state, questions, gold answers), train, serve.
Notebook: colab.research.google.com/gi…
Guide: unsloth.ai/docs/basics/train…
You can now train your own Decision model like Jev locally!
We increased Qwen3.5 0.8B’s aggregate accuracy from 20.7% to 74.3% across 3 decision benchmarks - on just 4GB VRAM.
Turn any LLM like Qwen3.8, Gemma 4 into decision models with our open-source Unsloth repo.
We fine-tuned with a Clef head using Unsloth and LoRA (r=64) for one epoch, increasing downstream accuracy from 30–37% to 78%.
GitHub: github.com/unslothai/unsloth
Guide and Notebooks: unsloth.ai/docs/basics/train…
Piyush Katariya retweeted
Goose: my latest programming language. Safer, faster and less memory than Rust/C++ for algorithms where tightly packed, flat data structures matter.
It can represent more kinds of data structures as flat, pointer-free bytes than any language I know.
github.com/aardappel/goose
1/
Piyush Katariya retweeted
GTA 5 has been ported ported to WebAssembly using AI. We live in the wildest timeline.
playgta5.com/
This media is unavailable
Calling classifiers "decision models" is a crime against machine learning. Those models decide *less* than LLMs, as you can think the CoT as a form of serial processing of information that such classifiers lack. Those models can be either *small* LLMs that are not let think, but instead the logits at the last position of the prefill are used to categorize in classes or, when they are BERT-alike, they compress the input meaning and project a class: in both cases they decide a lot less than an LLM.
Piyush Katariya retweeted
1/ In RL environments, reward design is everything.
Here's what we learned from the 7,780 environments Xiaomi open-sourced for MiMo 🧵
Wake up, ppl 👀
Xiaomi just open-sourced ~7K of the RL environments it used to train MiMo on HuggingFace, across domains like code, cyber, general, music and web dev
I think this is one of the biggest things to happen in frontier open-source RL environment data.
Indepth analysis and learnings coming soon 👀
Piyush Katariya retweeted
Dev Day announcement overview, as short as reasonably possible.
- Dots: personal assistant (similar to GrokBot, Muse etc)
- $500/month plan including Ultrafast
- GPT-6.1 Sol: really smart and cheap, way better than 6 was
- Decisions API: Jev killer with vision capabilities
- ultrafast is real (300tps on Astra). It’s way expensive though.
- sign in with ChatGPT: use your sub and inference in 3rd party apps
- extensions in ChatGPT can now be full apps
- 1 banked reset for all
I think that’s most of it!
Advices for young programmers.
Replying to @GergelyOrosz
Accept the present and the future. Embrace it. Have an evidence based career where you try doing meaningful things. Create open source software that does not exist currently since in the past it required teams of people and there were not the economic incentives but now with AI it is possible for individuals to do it.
Write-up and megakernel: inferact.ai/blog/tpu-megaker…
2/2
Piyush Katariya retweeted
Introducing Xiaomi MiMo-V2.6 — Pro & Flash.
Frontier intelligence, all the modalities, built in public.
🔹 Two omnimodal models, advancing through scaled reinforcement learning
🔹 Pro performs on par with Claude Opus 5 and GPT-5.6 Sol across most agent benchmarks
🔹 Pro scores 46 on the Artificial Analysis Intelligence Index — the highest among open-source models
🔹 Stronger coding, computer use, 3D reasoning and creative capabilities
🔹 Open model weights, technical report, RL environments and training code
Blog:mimo.xiaomi.com/mimo-v2-6
How can ARM CPUs compete with Intel/AMD when NEON is only 128-bit, while x86 can go all the way to 512-bit with AVX-512?
Because 512-bit SIMD does NOT mean 4x the CPU performance of 128-bit SIMD.
AArch64 NEON uses fixed 128-bit vectors. AVX2 can use 256-bit vectors, while AVX-512 can use 512-bit vectors.
But vector width is only one part of the equation.
Performance also depends on:
-> How many vector operations can execute in parallel
-> Execution throughput and latency
-> Clock frequency
-> OoO execution
-> Cache and memory bandwidth
-> How well the workload can actually be vectorized
-> Power and thermal limits
A CPU with multiple 128-bit execution pipelines can process several vector operations in parallel.
Meanwhile, AVX-512 is incredibly powerful when the workload has enough parallel data to actually use those 512-bit operations.
And ARM is no longer just 128-bit NEON either.
SVE/SVE2 introduced scalable vectors, while SME/SME2 adds scalable streaming vectors and matrix processing.
Apple's newer M4/A18-generation CPUs support SME/SME2, so describing modern Apple Silicon as simply "128-bit SIMD" isn't accurate anymore.
The important point is:
128-bit vs 512-bit tells you how much data ONE vector instruction can describe.
It does NOT tell you how fast the entire CPU is.
AVX-512 can win massively in the right vector-heavy workload.
But for general CPU performance, a wider SIMD instruction alone doesn't guarantee a faster CPU.
That's how a CPU built around 128-bit NEON can still compete with, or sometimes outperform, a CPU that supports 512-bit AVX-512.
SIMD width tells you how wide an instruction can be.
Microarchitecture + workload tell you how much of that width actually turns into performance.

































