Software Architect | Meditator

Pune, India
Joined September 2012
Storing vectors is easy. Retrieving the right evidence is harder. Introducing searchgres.js: BM25, semantic + hybrid search in Postgres, with hierarchy, metadata, regex + time filters. Filtering belongs inside retrieval. tsdb.co/searchgres-x
2
13
2
85
13,377
Piyush Katariya retweeted
🚀 Introducing DeepInfer, our in-house inference framework, now fully open source! Powering DeepSeek-V4.1-Flash at scale with high throughput and low latency. We’re sharing our production stack to help the community lower compute costs, reduce energy use, and make AI more affordable. 🔗 github.com/deepseek-ai/DeepI… 1/4
1,642
4,891
2,332
43,249
6,758,234
Piyush Katariya retweeted
Today we're open-sourcing pg_vectorsearch, a new vector search extension for PostgreSQL. The performance improvements over HNSW are pretty remarkable. At 100 million vectors: ◆︎ Faster queries, 4× less RAM: 6 ms vs. 8 ms at 95% recall, using 128 GB vs. 512 GB RAM. HNSW couldn't even build on 128 GB. ◆︎ 7× smaller indexes: 25 GB vs. 180 GB at 100M vectors. ◆︎ 50× faster index builds: 49 minutes vs. 42 hours on identical hardware. And at 1 million vectors, PRISM delivers 15 ms queries at 99% recall on just 2 GB RAM, versus HNSW's 392 ms on 4 GB. What leads to these improvements? Rather than traversing an HNSW graph with random memory accesses, our new PRISM index partitions vectors based on distance, compresses them using RaBitQ, and scans them efficiently using SIMD vectorization techniques. This takes better advantage of modern CPUs and memory locality. This is actually our second open-source vector index. We learned a lot from building DiskANN-based pgvectorscale, including some important limitations of graph-based approaches. Those lessons convinced us there was a better architectural foundation, so we started over. pg_vectorsearch is still in alpha, with more engineering ahead. But we believe fast, scalable vector search should live alongside your relational data, without requiring a separate vector database. Released under the permissive PostgreSQL License, just like our pg_textsearch extension for BM25 keyword search. Together, they provide the building blocks for hybrid vector + keyword search in PostgreSQL. Feedback and contributions welcome! Code and blog post below. ⚡️⚡️⚡️
38
99
12
861
39,635
Piyush Katariya retweeted
TensorFold 1.0.7 has landed with a major overhaul to Living Weights. • Each fact you teach through /learn becomes about ten new neurons inside the model's own weights, learned about 4x faster, with a live 3D view of what it has learned. Macs and DGX Spark. • Structured output and required tool calls on the native server, with drafting still on. • Qwen3.8-27B serves concurrent requests, and sampled replies draft too: 125 tok/s vs 20 on code on an M3 Ultra. • RTX 50-series and RTX PRO 6000 Blackwell support, NVFP4 Nemotron on DGX Spark, and faster GLM-5.3-Flash on two Macs. • Various other bugs and issues squashed I would like to thank the people who reached out to me and gave me access to their hardware to speed up testing. 🫶🏼 Thank you to everyone who sent PRs, reports and receipts, and to everyone using it: BobClawblaw BlivionIaG jschmied ZackSample ThinkOffApp jayleaton chaog992 Nipale-ai @volatilemarkts @MiaAI_lab @petruspennanen @plotarmordev github.com/ashhart/TensorFol…
21
39
10
278
25,314
Piyush Katariya retweeted
We are currently working on Qwen3.8-27B-Coder390-EfficientThink-Opus5.5-GPT6Astra-Grok4.7-DSV4Pro-K3-SFT-RLOO-MTP-DFlash2, the model with the longest name. We only need to ablate 5 weights (i.e., 3 layers) to achieve the ablation effect... We’re still testing to see if it can be made even smaller.
39
34
14
577
45,541
Piyush Katariya retweeted
TensorFold 1.0.3 lands with a few cool things. - Living Weights (beta with Nemotron on Metal) and coming to CUDA in 1.0.4 - Nvidia Ampere (beta) for Nemotron, and native Qwen3.8-27B with DFlash2 - Conversation prefix caching - Various bugs fixed and improvements Twelve contributor PRs helped make this happen, so a massive thank you to @niklaslenz_ai and GitHub contributors akol1, jschmied, chaog992, AdrianBinDC and BobClawblaw, and everyone testing, reporting bugs and helping us improve TensorFold. FYI, Living Weights is in very early beta and has sharp edges, but I think I’m on to something. I’d love community feedback to make it better: github.com/ashhart/TensorFol… @plotarmordev @petruspennanen @MiaAI_lab @volatilemarkts tensorfold.dev
36
32
17
255
32,618
Piyush Katariya retweeted
Opus 5.5 has one very serious, catastrophic problem that nobody is talking about: it's unfit for code review. It's been made so "safe" and so restricted that, out of a hundred confirmed critical issues, it will flag two at most. Opus 5.5 writes completely insecure software that can't be shipped to production I'm convinced Anthropic has gone overboard with restrictions and safety. This needs to be fixed urgently Keep this in mind!
107
37
10
1,457
198,777
Piyush Katariya retweeted
You can now train your own Decision model with our free notebook! 💡 Qwen3.5-4B will generate decisions instead of text on just 8GB VRAM locally. Learn to data prep (state, questions, gold answers), train, serve. Notebook: colab.research.google.com/gi… Guide: unsloth.ai/docs/basics/train…
You can now train your own Decision model like Jev locally! We increased Qwen3.5 0.8B’s aggregate accuracy from 20.7% to 74.3% across 3 decision benchmarks - on just 4GB VRAM. Turn any LLM like Qwen3.8, Gemma 4 into decision models with our open-source Unsloth repo. We fine-tuned with a Clef head using Unsloth and LoRA (r=64) for one epoch, increasing downstream accuracy from 30–37% to 78%. GitHub: github.com/unslothai/unsloth Guide and Notebooks: unsloth.ai/docs/basics/train…
111
450
67
3,755
351,751
Piyush Katariya retweeted
Goose: my latest programming language. Safer, faster and less memory than Rust/C++ for algorithms where tightly packed, flat data structures matter. It can represent more kinds of data structures as flat, pointer-free bytes than any language I know. github.com/aardappel/goose 1/
111
155
60
1,364
140,246
Piyush Katariya retweeted
GTA 5 has been ported ported to WebAssembly using AI. We live in the wildest timeline. playgta5.com/
201
506
116
12,391
1,349,993
Piyush Katariya retweeted
Calling classifiers "decision models" is a crime against machine learning. Those models decide *less* than LLMs, as you can think the CoT as a form of serial processing of information that such classifiers lack. Those models can be either *small* LLMs that are not let think, but instead the logits at the last position of the prefill are used to categorize in classes or, when they are BERT-alike, they compress the input meaning and project a class: in both cases they decide a lot less than an LLM.
57
34
4
657
37,637
Piyush Katariya retweeted
1/ In RL environments, reward design is everything. Here's what we learned from the 7,780 environments Xiaomi open-sourced for MiMo 🧵
Wake up, ppl 👀 Xiaomi just open-sourced ~7K of the RL environments it used to train MiMo on HuggingFace, across domains like code, cyber, general, music and web dev I think this is one of the biggest things to happen in frontier open-source RL environment data. Indepth analysis and learnings coming soon 👀
11
60
6
501
44,595
Piyush Katariya retweeted
Dev Day announcement overview, as short as reasonably possible. - Dots: personal assistant (similar to GrokBot, Muse etc) - $500/month plan including Ultrafast - GPT-6.1 Sol: really smart and cheap, way better than 6 was - Decisions API: Jev killer with vision capabilities - ultrafast is real (300tps on Astra). It’s way expensive though. - sign in with ChatGPT: use your sub and inference in 3rd party apps - extensions in ChatGPT can now be full apps - 1 banked reset for all I think that’s most of it!
284
291
72
7,773
484,074
Piyush Katariya retweeted
Advices for young programmers.
Replying to @GergelyOrosz
Accept the present and the future. Embrace it. Have an evidence based career where you try doing meaningful things. Create open source software that does not exist currently since in the past it required teams of people and there were not the economic incentives but now with AI it is possible for individuals to do it.
21
45
1
559
55,472
Piyush Katariya retweeted
Introducing Xiaomi MiMo-V2.6 — Pro & Flash. Frontier intelligence, all the modalities, built in public. 🔹 Two omnimodal models, advancing through scaled reinforcement learning 🔹 Pro performs on par with Claude Opus 5 and GPT-5.6 Sol across most agent benchmarks 🔹 Pro scores 46 on the Artificial Analysis Intelligence Index — the highest among open-source models 🔹 Stronger coding, computer use, 3D reasoning and creative capabilities 🔹 Open model weights, technical report, RL environments and training code Blog:mimo.xiaomi.com/mimo-v2-6
383
870
484
8,832
1,527,044
Piyush Katariya retweeted
How can ARM CPUs compete with Intel/AMD when NEON is only 128-bit, while x86 can go all the way to 512-bit with AVX-512? Because 512-bit SIMD does NOT mean 4x the CPU performance of 128-bit SIMD. AArch64 NEON uses fixed 128-bit vectors. AVX2 can use 256-bit vectors, while AVX-512 can use 512-bit vectors. But vector width is only one part of the equation. Performance also depends on: -> How many vector operations can execute in parallel -> Execution throughput and latency -> Clock frequency -> OoO execution -> Cache and memory bandwidth -> How well the workload can actually be vectorized -> Power and thermal limits A CPU with multiple 128-bit execution pipelines can process several vector operations in parallel. Meanwhile, AVX-512 is incredibly powerful when the workload has enough parallel data to actually use those 512-bit operations. And ARM is no longer just 128-bit NEON either. SVE/SVE2 introduced scalable vectors, while SME/SME2 adds scalable streaming vectors and matrix processing. Apple's newer M4/A18-generation CPUs support SME/SME2, so describing modern Apple Silicon as simply "128-bit SIMD" isn't accurate anymore. The important point is: 128-bit vs 512-bit tells you how much data ONE vector instruction can describe. It does NOT tell you how fast the entire CPU is. AVX-512 can win massively in the right vector-heavy workload. But for general CPU performance, a wider SIMD instruction alone doesn't guarantee a faster CPU. That's how a CPU built around 128-bit NEON can still compete with, or sometimes outperform, a CPU that supports 512-bit AVX-512. SIMD width tells you how wide an instruction can be. Microarchitecture + workload tell you how much of that width actually turns into performance.
2
11
1
95
7,161
Piyush Katariya retweeted
AMD's new ray tracing technique slashes BVH memory usage from 80 GB to just 1.7 GB, runs 500 million animated triangles at 60+ FPS on RX 9070 XT. 🔗 wccf.tech/1l9d4
23
120
12
1,172
51,109
Piyush Katariya retweeted
Most predictions I see are still way too conservative. Here's mine
540
904
395
8,939
2,403,541
Piyush Katariya retweeted
learning rust is basically just letting a compiler aggressively bully you for six months until you develop stockholm syndrome and start telling everyone else how great the safety is.
52
130
38
2,018
50,595