Research scientist at Google DeepMind | previously at Microsoft Research and Postdoc at UMass. Views are my own. Most tweets in Mongolian 🇲🇳.
Bay Area
Joined January 2010
- Tweets22.1K
- Following638
- Followers4.7K
- Likes7.3K
Tsendsuren retweeted
Lovely! "Weight-space programming" is such an important concept in DL; we covered it 4 times in my Yale course this semester!
1. Gradient descent as a weight-space programmer.
2. Fast weight programming.
3. Metalearning: GD writes learning algorithms
4. Self-referential WM & RSI
We asked GPT-6 Astra to edit Qwen3.5-0.8B weights to improve math.
It directly changed only 0.016% of weights, fixing 0.8B's reasoning loops & improving math up to 6x and even instruction following 3x, without gradients or training.
We call this Weight-Space Programming: frontier LLMs can now natively read/write neural network weights! 🧵
Blog: rikiyat.github.io/blog/weigh…
Readers added context they thought people might want to know
LLM was given weights explicitly as text in prompt and used code/forward passes (per authors' logs) to ablate Qwen looping direction, a known thinking-mode issue per model card. Primarily fixed repetition in that mode, not general 'natively' improving weights/math.
rikiyat.github.io/blog/weight-sp…
github.com/RikiyaT/lm_und…
huggingface.co/Qwen/Qwen3.5-0…
huggingface.co/RikiyaT/Qwen3.…
Tsendsuren retweeted
Needless to say, we also covered the neuroscience connection.
More in the primer paper: arxiv.org/abs/2508.08435
Tsendsuren retweeted
Gemini 4 Carbon is our next frontier model.
It’s state-of-the-art across cybersecurity and long-horizon capabilities.
Tsendsuren retweeted
This is Genie 4, our most realistic and interactive world model yet. 🌐
Explore dynamic environments in real time to see how the scene responds, sounds, and how the people in it move, react, and speak.
Tsendsuren retweeted
The complexity theory result is actually huge.
P problems can be solved in polynomial time. It was widely believed that NP-complete problems require exponential time.
P = NP states that NP-complete problems can be solved in polynomial time. OpenAI didn't prove that. But, they proved that (some) NP-complete problems can be solved faster than exponential time, meaning:
- P = NP, or
- the complexity of NP problems is a "middle thing" between polynomial and exponential, eg. O(n^log n)
If it survives peer reviews, this could be one of the biggest CS breakthroughs in decades.
Announcing three major scientific discoveries, one each in mathematics (harmonic analysis), computer science (complexity theory), and physics (quantum field theory).
AI starting to help us improve our understanding of the universe! openai.com/index/building-sc…
Tsendsuren retweeted
Samsung open-sourced a method that shrinks 13B parameter LLM into less than 1 GB.
It's called "LittleBit"
Instead of storing AI weights as standard numbers, they used latent factorization to crush them down to extreme sub-1-bit levels.
In some configurations, they hit 0.1 bits per weight.
But here is where the architecture gets crazy.
When you compress an AI this much, you can stop doing math.
Instead of forcing the hardware to do heavy floating-point multiplication, LittleBit replaces the core computation with a bitwise XOR operation.
It swaps complex matrix math for basic sign flips.
The results rewrite the rules of model deployment:
• Unlocks a massive 11.6x inference speedup relative to standard FP16 models.
• Radically reduces memory footprint and loading bandwidth.
• Maintains robustness in extreme sub-0.5 bit regimes where previous compression methods catastrophically fail.
This is not a clever optimization.
It is the blueprint for running massive, state-of-the-art AI locally on cheap, resource-constrained devices.
Tsendsuren retweeted
BREAKING: OpenAI’s solution to Navier–Stokes does not match its Lean verification.
The most important article to read today is not one of OpenAI’s 700 AI-generated math papers.
It is this other paper, making a deep and worrying point:
A Lean-verified proof does not automatically validate the proof written in natural language, nor does it mean that the formal statement captures the intended theorem.
During translation, an AI can change an assumption, weaken a statement, or replace the argument entirely.
It can hallucinate another theorem.
Lean correctly verifies the result.
But the proved result may no longer be what the paper claims.
This is a general problem. Things get spicy when the authors examine OpenAI’s proposed Navier–Stokes solution.
They identify at least two mismatches between the written intermediate results and their Lean counterparts:
One estimate claims that four additional input derivatives suffice. The Lean version requires five: a weaker result.
A pressure-flux estimate is obtained through a different bound, and proved through a different argument.
It is not clear whether these mismatches invalidate the entire proof.
But they raise an important issue.
OpenAI is flooding us with claimed revolutionary breakthroughs. Yet nobody knows whether the proofs are correct or whether they prove what they claim to be proving.
Epistemia at scale.
*
Paper in the first reply
Tsendsuren retweeted
Today marks a new chapter for Windows, as we bring unmetered intelligence to every desk and every home, and make every PC a place where agents can work securely on your behalf.
Some highlights of what we announced:
• MAI-Code-1.1 Flash: 137B parameter coding model w/ 256K context window, which is now optimized to run on your PC!
• GitHub Copilot now hands off work to local models like MAI-Code-1.1 Flash, helping projects cost a lot less without sacrificing quality.
• With Hybrid Intelligence, Copilot can now take action directly on the PC and keep sensitive work on your device.
• And with Code in Copilot, you can essentially build any software you need on your desktop, without any cloud token spend, and it’s just super at it. You’re no longer limited to what’s in an app store! Your PC becomes an infinite software factory.
• Security is foundational to all this, which is why we are also bringing together Windows and Agent 365 so agents can work within secure boundaries on-device, including MXC a local sandbox for agent execution. Windows becomes your secure agent box!
• All this comes to life on a new generation of devices, like Surface Laptop Ultra, powered by NVIDIA RTX Spark.
Can’t wait to see what you build with all this.
Tsendsuren retweeted
openai/math: 722 papers from an internal OpenAI model, 372 results
I built a search engine for it 🧮
• plain English: "how many shuffles randomize a deck?"
• names/jargon: "UGC", "Kaplansky"
• filter: Lean-verified, negative results, reasoning traces
mishig-openai-math-explorer.…
We’re releasing a broad range of new mathematical results produced by an internal frontier model.
We’ve been consulting with the independent Advisory Group on Mathematics and Artificial Intelligence at the Institute for Advanced Study, and we have drawn on their advice and public recommendations to inform how we release these results.
github.com/openai/math
Tsendsuren retweeted
1/ Introducing PhilosophyBench from @StanfordAILab @StanfordHCI, the first independent, large-scale benchmark for evaluating AI’s philosophical capabilities.
philosophybench.org
Tsendsuren retweeted
Our group discovered that reasoning models produce fractals when asked to solve hard problems. We can use nonlinear dynamics to probe the thinking processes of recurrent depth models on Sudoku, mathematics, and even ARC-AGI (1/N)
arxiv.org/abs/2609.04963
Tsendsuren retweeted
🚨 Excited to share our new work on WikiSkill (led by the awesome @LiyanTang4)!
We propose a method that co-evolves agent skills and a wiki that integrates agent experience into a growing knowledge base. WikiSkill substantially outperforms current SoTA skill evolution methods 👇
Tsendsuren retweeted
Будда-н сургаал ч үгүй,
коммунист ёс суртахуун ч үгүй
Тэнгэр дээдэсээс ч айдас үгүй
… ер н л Ёс үгүй болохоор: үсгүй мич л болдог юм шив
.. улам олоноороо
Tsendsuren retweeted
Intuition: Compaction is agentic recurrence (RNNs), whereas recursion (RLMs) is agentic attention.
Recurrence maintains a constant-size state at all times by compressing the past.
Attention keeps all context fully represented and then decides how to re-process it for each step.
Tsendsuren retweeted
Speculate While You Reason: Teaching Agents to Predict Their Next Tool Call via Joint Agent-Speculator RL
Jiabao Ji, Yujian Liu, Li An, Rohit Jain, Gungor Polatkan, Siyu Zhu, Shiyu Chang
arxiv.org/abs/2607.25816 [𝚌𝚜.𝙰𝙸]
ALT Large language model agents often spend substantial wall-clock time waiting for tool call results. Tool-call speculation can hide this latency by predicting and pre-executing an agent's next tool call if the prediction matches the agent's eventual tool call, but existing speculators are typically separate draft models or cached traces that are poorly aligned with the deployed agent's own behavior. We identify this speculator-agent gap and show that the target agent itself is a strong next-call speculator. This points to a simpler design: unifying the agent and speculator within the same model. In this paper, we introduce the self-speculating agent, a single model that both solves tasks in agent mode and predicts its next tool call from partial trajectories in speculator mode, fully reusing prefix KV cache. To enable this dual-mode agent without degrading performance, we propose a joint agent-speculator reinforcement learning method, which derives speculation targets from the agent's ow
Tsendsuren retweeted
1/Again about continual learning.
What if catastrophic forgetting isn't a bug, but a mathematical necessity?
We build complex systems to retain old knowledge, but new theory proves that in non-stationary streams, perfect memory is an optimization liability. Here is why. 🧵
Tsendsuren retweeted
Kimi K3 license. It's inspired by MIT but distinctly non-commercial, where any company making over $20M/yr must get a specific commercial deal (and display Kimi K3 if over 100M users or $20M/mo revenue)
Tsendsuren retweeted
Releasing the model weights and technical report of Kimi K3.
Kimi K3 is our most capable model: a 2.8T MoE model with native visual understanding and a 1M-token context window.
New model architecture: 2.5x the intelligence per unit of compute, not just more params.
Alongside Kimi K3, we're opening up more of the stack behind it — high-performance attention kernels, MoE communication library, and infrastructure for running agent environments at scale.
Model weights: huggingface.co/moonshotai/Ki…
Tech report: github.com/MoonshotAI/Kimi-K…
Tech blog: kimi.com/blog/kimi-k3
Tsendsuren retweeted
Reminds me a lot about turing machine:
the state register - root lm
tape - REPL context var
read/write head - sub-calls
Transformers struggle to generalize to tasks they were not explicitly trained on. Instead, we propose in 2026 that it is the job of the harness to generalize through composition.
We observe a powerful property when training RLMs: for tasks with shared structure that look different, the root model naturally learns the same trajectory, meaning it views the two task trajectories as the same! In other words, the Transformer does not need additional generalization capabilities to transfer capabilities from one task to the other, the harness induces it.
We find that well-designed harnesses form a quotient set over task trajectories, meaning their individual LLM calls can see structurally “similar” tasks as near-identical, token-for-token! Harnesses can effectively generalize for the Transformer during training, without relying on any intrinsic generalization capability from the model.
For example, RLMs can see problems of different lengths as the same: we show that RLMs can train exclusively on short tasks, and fully generalize to similar but unseen tasks 8-32x longer because it produces near identical trajectories for both.
Taking this further, we show that tasks across different domains (e.g. math solutions vs. essay writing) that share a decomposition strategy exhibit the same generalization effect. RLMs can train on the problem of finding which essays belong to the same author and improve performance on finding math problems that share similar solutions.
The full blogpost, experiments, and discussion are in the thread below.
