Research scientist at Google DeepMind | previously at Microsoft Research and Postdoc at UMass. Views are my own. Most tweets in Mongolian 🇲🇳.
Bay Area
Joined January 2010
- Tweets22.1K
- Following638
- Followers4.7K
- Likes7.3K
LLM memory atm is akin to stone age. LLM is learning to harness, i.e taking notes for itself. Humans as an individual takes notes too or writes book or as a society harness collective mind and stores information in culture and legacy. But unlike LLM, we internalize them quickly.
Memory is what allows Claude to learn from experience. In a new Engineering Blog post, we share how “dreaming” helps Claude organize what it’s learned, correct its mistakes, and find what it needs more easily the next time. anthropic.com/research/compo…
Tsendsuren retweeted
Lovely! "Weight-space programming" is such an important concept in DL; we covered it 4 times in my Yale course this semester!
1. Gradient descent as a weight-space programmer.
2. Fast weight programming.
3. Metalearning: GD writes learning algorithms
4. Self-referential WM & RSI
We asked GPT-6 Astra to edit Qwen3.5-0.8B weights to improve math.
It directly changed only 0.016% of weights, fixing 0.8B's reasoning loops & improving math up to 6x and even instruction following 3x, without gradients or training.
We call this Weight-Space Programming: frontier LLMs can now natively read/write neural network weights! 🧵
Blog: rikiyat.github.io/blog/weigh…
Readers added context they thought people might want to know
LLM was given weights explicitly as text in prompt and used code/forward passes (per authors' logs) to ablate Qwen looping direction, a known thinking-mode issue per model card. Primarily fixed repetition in that mode, not general 'natively' improving weights/math.
rikiyat.github.io/blog/weight-sp…
github.com/RikiyaT/lm_und…
huggingface.co/Qwen/Qwen3.5-0…
huggingface.co/RikiyaT/Qwen3.…
Tsendsuren retweeted
Needless to say, we also covered the neuroscience connection.
More in the primer paper: arxiv.org/abs/2508.08435
I love neural nets updating their own weights!! That was the whole idea around MetaNets and Fast-Weights. We are about see another regime of training data and scaling of point-wise weight update mechanisms.
We asked GPT-6 Astra to edit Qwen3.5-0.8B weights to improve math.
It directly changed only 0.016% of weights, fixing 0.8B's reasoning loops & improving math up to 6x and even instruction following 3x, without gradients or training.
We call this Weight-Space Programming: frontier LLMs can now natively read/write neural network weights! 🧵
Blog: rikiyat.github.io/blog/weigh…
Readers added context they thought people might want to know
LLM was given weights explicitly as text in prompt and used code/forward passes (per authors' logs) to ablate Qwen looping direction, a known thinking-mode issue per model card. Primarily fixed repetition in that mode, not general 'natively' improving weights/math.
rikiyat.github.io/blog/weight-sp…
github.com/RikiyaT/lm_und…
huggingface.co/Qwen/Qwen3.5-0…
huggingface.co/RikiyaT/Qwen3.…
Tsendsuren retweeted
Gemini 4 Carbon is our next frontier model.
It’s state-of-the-art across cybersecurity and long-horizon capabilities.
Tsendsuren retweeted
This is Genie 4, our most realistic and interactive world model yet. 🌐
Explore dynamic environments in real time to see how the scene responds, sounds, and how the people in it move, react, and speak.
Tsendsuren retweeted
The complexity theory result is actually huge.
P problems can be solved in polynomial time. It was widely believed that NP-complete problems require exponential time.
P = NP states that NP-complete problems can be solved in polynomial time. OpenAI didn't prove that. But, they proved that (some) NP-complete problems can be solved faster than exponential time, meaning:
- P = NP, or
- the complexity of NP problems is a "middle thing" between polynomial and exponential, eg. O(n^log n)
If it survives peer reviews, this could be one of the biggest CS breakthroughs in decades.
Announcing three major scientific discoveries, one each in mathematics (harmonic analysis), computer science (complexity theory), and physics (quantum field theory).
AI starting to help us improve our understanding of the universe! openai.com/index/building-sc…
Isn’t this expected as pre-training data has such information?
this work from @GenReasoning is fascinating - training agent organizations to work as a collective
“During training, organisations develop additional layers of hierarchy, adopt different divisions of labour across tasks, and devote more resources to peer review. Agents also increasingly reason about their teammates and offer help. In safety evaluations, however, they become less likely to report problems to human overseers.”
Tsendsuren retweeted
Samsung open-sourced a method that shrinks 13B parameter LLM into less than 1 GB.
It's called "LittleBit"
Instead of storing AI weights as standard numbers, they used latent factorization to crush them down to extreme sub-1-bit levels.
In some configurations, they hit 0.1 bits per weight.
But here is where the architecture gets crazy.
When you compress an AI this much, you can stop doing math.
Instead of forcing the hardware to do heavy floating-point multiplication, LittleBit replaces the core computation with a bitwise XOR operation.
It swaps complex matrix math for basic sign flips.
The results rewrite the rules of model deployment:
• Unlocks a massive 11.6x inference speedup relative to standard FP16 models.
• Radically reduces memory footprint and loading bandwidth.
• Maintains robustness in extreme sub-0.5 bit regimes where previous compression methods catastrophically fail.
This is not a clever optimization.
It is the blueprint for running massive, state-of-the-art AI locally on cheap, resource-constrained devices.
Tsendsuren retweeted
BREAKING: OpenAI’s solution to Navier–Stokes does not match its Lean verification.
The most important article to read today is not one of OpenAI’s 700 AI-generated math papers.
It is this other paper, making a deep and worrying point:
A Lean-verified proof does not automatically validate the proof written in natural language, nor does it mean that the formal statement captures the intended theorem.
During translation, an AI can change an assumption, weaken a statement, or replace the argument entirely.
It can hallucinate another theorem.
Lean correctly verifies the result.
But the proved result may no longer be what the paper claims.
This is a general problem. Things get spicy when the authors examine OpenAI’s proposed Navier–Stokes solution.
They identify at least two mismatches between the written intermediate results and their Lean counterparts:
One estimate claims that four additional input derivatives suffice. The Lean version requires five: a weaker result.
A pressure-flux estimate is obtained through a different bound, and proved through a different argument.
It is not clear whether these mismatches invalidate the entire proof.
But they raise an important issue.
OpenAI is flooding us with claimed revolutionary breakthroughs. Yet nobody knows whether the proofs are correct or whether they prove what they claim to be proving.
Epistemia at scale.
*
Paper in the first reply
This is the way. I am a big believer in hybrid computing. This is much more efficient, accessible and secure.
BREAKING: @JensenHuang just said @nvidia and @Microsoft have "reinvented the personal computer completely."
Two things change:
1 - Your computer becomes the home for your agents.
2 - Your AI runs partly on your machine and partly in the cloud.
Right now your laptop sits on your desk all day and waits for you. Huang wants it to keep working while you're out. You give an agent a task, and it uses your apps and files on its own. You check in from your phone. He's said he'll text his agents on WhatsApp and they'll text back.
The agent lives on a computer you own, at home or on your desk, and runs around the clock.
I like the idea. Now I want to see one agent finish one boring, real task for me from start to finish.
Tsendsuren retweeted
Today marks a new chapter for Windows, as we bring unmetered intelligence to every desk and every home, and make every PC a place where agents can work securely on your behalf.
Some highlights of what we announced:
• MAI-Code-1.1 Flash: 137B parameter coding model w/ 256K context window, which is now optimized to run on your PC!
• GitHub Copilot now hands off work to local models like MAI-Code-1.1 Flash, helping projects cost a lot less without sacrificing quality.
• With Hybrid Intelligence, Copilot can now take action directly on the PC and keep sensitive work on your device.
• And with Code in Copilot, you can essentially build any software you need on your desktop, without any cloud token spend, and it’s just super at it. You’re no longer limited to what’s in an app store! Your PC becomes an infinite software factory.
• Security is foundational to all this, which is why we are also bringing together Windows and Agent 365 so agents can work within secure boundaries on-device, including MXC a local sandbox for agent execution. Windows becomes your secure agent box!
• All this comes to life on a new generation of devices, like Surface Laptop Ultra, powered by NVIDIA RTX Spark.
Can’t wait to see what you build with all this.
Tsendsuren retweeted
openai/math: 722 papers from an internal OpenAI model, 372 results
I built a search engine for it 🧮
• plain English: "how many shuffles randomize a deck?"
• names/jargon: "UGC", "Kaplansky"
• filter: Lean-verified, negative results, reasoning traces
mishig-openai-math-explorer.…
We’re releasing a broad range of new mathematical results produced by an internal frontier model.
We’ve been consulting with the independent Advisory Group on Mathematics and Artificial Intelligence at the Institute for Advanced Study, and we have drawn on their advice and public recommendations to inform how we release these results.
github.com/openai/math
Tsendsuren retweeted
1/ Introducing PhilosophyBench from @StanfordAILab @StanfordHCI, the first independent, large-scale benchmark for evaluating AI’s philosophical capabilities.
philosophybench.org
Some RL plots are more fun to watch
Nearly half a year of silence. We spent it studying one problem: how far RL can scale.
MiMo-V2.6 is in the middle of its RL run right now. Three things we scaled: compute (~2B tokens per step, 1568 prompts × 16 rollouts, fully async), environments and harnesses (multi-task agentic RL, mixed across multiple harnesses in one run), and grader compute (agentic in-group credit assignment, with test-case and rubric-based rewards). We'll open-source the details piece by piece over the coming weeks.
Streaming the run: mimo.xiaomi.com/rl/
IMO Chinese startup ecosystem also deserves some credit here.
Moonshot AI CEO Yang Zhilin surprised some of his Carnegie Mellon professors by choosing to return to China. Now, his Beijing-based startup’s latest private funding round values it at $50 billion. 🔗 on.wsj.com/4A9SV5g
