Ray (@ray3Xray) header
Ray (@ray3Xray) profile picture

day: Mag7 | night: trenches || shitposting round tripping gen wealth..

Trenches
Joined February 2021
Photo by Ray (@ray3Xray)Photo by Ray (@ray3Xray)Photo by Ray (@ray3Xray)Photo by Ray (@ray3Xray)Photo by Ray (@ray3Xray)Photo by Ray (@ray3Xray)
Looking for a reference manual for building a frontier LLM? We wrote one 👇 The K2 Horizon technical report is now on arXiv: 134 pages on how we built six models, from architecture to inference. It may be the most detailed LLM training reference you can read today. • Architecture: we bring two new attention designs, MoVA and softplus attention gating, to strengthen reasoning. With them, our 36B model nearly matches a 32B dense one on one-seventh of the active parameters. • Pre-training: we trained four models, from 3.7B to 36B, on the same 22T tokens. Their normalized loss curves collapse onto nearly the same line, so a recipe that works on a small model carries over when scaling up. • Data: 60+ pages on the part most reports skip. How we generated ~10T synthetic tokens, and why we chose the data mix on predicted scores at scale, which reversed the small-run ranking. • Post-training: we document our full production-scale run for the 375B flagship. • Inference: Uno gives an autoregressive model diffusion speed with one small adapter. Up to 2.2× faster decoding, lossless by construction, no draft model. This is the last piece of the openness puzzle. K2 Horizon already shipped with 380 checkpoints, 13 TB of pre-training data, 58 training runs and 17 code repositories. The report explains the decisions behind them. More details: arxiv.org/abs/2611.21003
24
72
22
494
44,853
Students who can't solve math problems - with pencil and paper and on their own - aren't learning math. They're just learning 𝘢𝘣𝘰𝘶𝘵 math.
19
51
2
573
14,284
Frontier models have made this a golden age for training tiny models 🤏 I just trained a tiny autocomplete model on the Enron emails dataset for 35 cents: • 944K parameters, 0.98 MB • trained in 14 minutes on HF Jobs ($0.35) • runs in your browser • always out of the office huggingface.co/spaces/cfahlg…
14
30
6
289
12,936
Fine-tuning an SLM locally is difficult. I’ve learnt it the hard way. I’ve been pulling my hair out over the last 3 days trying to train an SLM that texts just like me. So much scope to improve. Full blog here: sskorde.com/writing/fine-tun…
15
35
1
354
15,717
How can we build LLM social simulations that capture meaningful human differences and generalize beyond a single experiment? Excited about our new work identifying which axes of behavioral variation matter for predicting human decisions across configurations of a task family Great working with @RobinNa_ @Kehang_Zhu and @bakkermichiel!
Prompting LLMs to behave like people can improve simulations, but how should we design this? What info is needed, and how much? With @Kehang_Zhu @locross @bakkermichiel, we build “behavioral profiles” that help LLMs simulate human behavior under distribution shifts. 1/8 🧵
4
7
79
4,412
I see a lot of people concluding that most research about LLMs are short-lived, have poor external validity, and, at worst, “useless.” There’s a lot of truth to this! But my hot take(?) is that this misses another important point — the value of research isn’t just about the findings!! There’s a lot to learn from the methods, justifications and thought processes behind each decision, and the questions researchers choose to ask on a particular problem. These contributions can transcend a particular model or findings being studied!!
How long does a finding about LLMs stay true? We re-tested 50 widely cited findings about LLM behavior (various biases, hallucination, etc). Within 15 months, most no longer held or had substantially reduced effects. LLM findings has short shelf life zou-group.github.io/llm-clai… Great job by @oq_35!
5
3
2
50
4,216
Tech report for EmbeddingGemma 2 is now out! Check it out: arxiv.org/pdf/2610.37364
2
1
18
3,628
Open-source robots you can build or buy today: - Microduck, mini duck biped, $399, starts shipping before Christmas - NexArm, robot arm, €440 starter kit - Microban, 30cm humanoid, about $500 in parts - Mini Pupper 2, quadruped, about $600 - LeRobot humanoid, legs only, $2,500 in parts - Berkeley Humanoid Lite, 0.8 m, under $5,000 - ToddlerBot by Stanford, 0.56 m, under $6,000 as per reports - OpenArm 2.0, 7-DoF arm, starts from $6,500 - Asimov by Menlo Research, 1.2 m humanoid, about $15–20K as an unassembled kit
15
14
227
10,153
My kids are eagerly waiting for Microduck to arrive by Christmas. I ordered within couple hours of announcement. Their questions are pretty fun can this play dead? Can it pick mess from room? Can it do homework? can it roller skate with me when I bike? Gonna be fun digging into policies
This live demo of four Microducks with Jensen Huang went completely viral, and we're ecstatic about it. But there's also a lot of confusion, so I want to clarify what Microduck can and cannot do, and what it will realistically be able to do in the future. I count 60M+ views on this demo alone, based on the few big posts I see on social media, and everything has been buzzing. Even my own simple emotional design post on a newly created Instagram account got 1M+ views. We are selling Microducks at an incredible rate, even faster than during its historic launch. I've been trying to answer all the questions I see on our socials: people are asking if X or Y is possible on the robots, and this post aims to answer most of them. First of all, the demo you've seen was not fully autonomous. We had one engineer piloting each Microduck. The motions on the robots were trained with reinforcement learning, using a sim-to-real pipeline: walking, standing up after a fall, sitting, doing a somersault, roller skating. All of those are policies trained this way. What the pilots did during the live event was send commands: directions (move forward, rotate), head orientation (look here), quacks, and special actions (do the somersault, sit down). If you buy a Microduck, will you get an autonomous companion right out of the box? The current answer is no. But there is no impossible technical barrier to get there. I'm not sure how much time we'll have to work on this before shipping, but since everything is open source, I fully expect the community to build it, and to build it fast. Now let's understand what's already done, what will be easy to add, and what's an open problem. What you'll be able to do out of the box, and you can already do it in simulation right now, is train new policies with reinforcement learning. You could teach your duck to walk differently from the default gait we provide, to do a weird dance, or to walk on stilts. The community has been crazy creative with this, even though they only have the simulator. Note that you don't need reinforcement learning when the movements are simple. For example, the early emotional design I did is mostly coded movements plus pre-generated sounds. A simple mental model: use reinforcement learning when balance is part of the movement. If you try to code a cinematic walk by naively putting one foot in front of the other, your robot will realistically never walk, because you need to account for the dynamics of the movement, the inaccuracies of the sensors, the friction with the ground, and potential perturbations from the outside, like being pushed. So training new policies is definitely doable, but you can't train a policy to get a genuine long-term companion in your home (well, technically you could, but that's a whole other range of policy: a research-grade problem, and the resulting neural network would probably be huge and never able to run on such a small compute). Instead, you need a few more building blocks: voice recognition, facial detection/recognition, localization, some simple navigation, and a behavior layer, which is less technical and harder to describe. As a rule of thumb, a prototype of any of these is within reach of anyone if they have a modern coding agent and if the computation doesn't run on the robot (meaning the robot can reach the internet or an external computer and use external APIs). A rough prototype for each of the above can be a chill weekend project. As always, going from the prototype to a high-quality implementation is another story, but still doable for most if you put in the time and test a lot. However, having all of this run locally, on the onboard chip, I would call a very hard task. You'd need some very clever engineering to get a good enough version of everything working together. But I can't say it's impossible, because I've seen some crazy efficient implementations of all of the above. And here we don't need a super high success rate: it's okay if the duck sometimes does something a bit random. So I wouldn't be surprised if folks managed a fully local, cool, simple version of this autonomous companion. For example, you don't need Whisper-grade open-vocabulary transcription: being able to recognize a few keywords would already be great for interaction. One of the fastest ways to turn this robot into an autonomous companion would be to stream Microduck's sensors (the camera, the depth sensor, the IMU and the joint positions) to a computer, and use a software stack like the one we use for the conversation app on Reachy Mini: a local VAD, then a cloud speech-to-text model, then a call to a frontier LLM that decides on actions. Actions could be quacking, looking at things/faces/a direction, walking to a given position, sitting down, playing an emotion, kicking or doing a somersault. A SLAM algorithm would run in parallel so the robot can localize itself in your home, and you would prompt the LLM for a lifelike behavior: moving to random places, sometimes following people, and having emotional responses that fit what you're saying and the duck's environment. Of course someone is going to build a Tamagotchi-like layer on top of this, and it will be insane. How much demand is there for a software suite that would do all of this? I for one think it would be super cool. A word about the humans. We had five engineers backstage, and they are pretty much the core team behind Microduck. Their impact per head is absolutely off the charts. Antoine Pirrone piloted Graphite, Augustin Crampette Lavender (the rebel one), Matthieu Lapeyre Sky, Coralie Deplane Cream, and Pierre Rouanet was stressing over the Wi-Fi. Give them a follow! (Antoine @antoinepirrone, Augustin on LinkedIn, Matthieu @matth_lapeyre, Coralie @Drenaya_, Pierre on LinkedIn). I get visibility because I communicate a lot, but I mainly work on Reachy Mini (you might want to check it out: it's a very cool robot, and it ships much faster than Microduck :) ). My contribution to Microduck has been minimal: I'm mostly a first power user. My main added value at GTC was having hands big enough to carry four Microducks below the stage. Where I take pride is that Antoine and Coralie are former students and former interns of mine. I'm therefore entitled to 3% credit on anything (good) they do, for life. If they do something bad, however, it was clearly the fault of their parents. About the super smart engineering that can be done on board: Antoine just released two very cool blog posts (links below). The first is on odometry, the robot estimating its own movement from its own internal sensors. The second is about SLAM, which allows long-term localization: it adds the depth sensor to the odometry to get a stable position over time. Think of it as recognizing known landmarks in your home to know where the robot is. The crazy part is that all of this runs on the robot. So this gives me hope that we'll be able to do a lot of cool stuff on board.
1
166
Ever wondered what it’s like to learn robot autonomy @Stanford? Now you can. Today, we’re opening up the robot autonomy teaching materials that my collaborators and I have been developing over the past several years—for anyone, anywhere, completely free. At the heart of this effort is Principles of Robot Autonomy (PoRA), a textbook inspired by the Stanford robot autonomy course series I’ve been teaching since 2017. Robot autonomy has evolved from a research discipline into a technology deployed in the real world. Building capable autonomous systems requires more than understanding individual algorithms: it requires understanding how perception, state estimation, planning, control, and learning work together in a complete autonomy stack. That’s what we set out to teach with PoRA: bringing classical robotics and modern AI together in one coherent framework, grounded in both theory and practice. Coming soon in print from Cambridge University Press, Principles of Robot Autonomy is also freely available online and comes with a comprehensive collection of open educational resources: 📖 Textbook, arxiv.org/abs/2608.03496 — Foundations of robot autonomy, from classical methods to modern AI-based approaches. 🎥 Video lectures, porabook.com/lectures/ Lectures from my Stanford courses on robot autonomy and on optimal and learning-based control. 💻 Interactive exercises, porabook.com/exercises/ — Hands-on Python implementations and Jupyter notebooks. 📝 Teaching materials, porabook.com/instructors/ — Lecture slides and resources for instructors. 🌐 Explore all PoRA resources, porabook.com/ — Including Ask the Book, an AI study assistant integrated throughout the textbook to answer questions, explain derivations, and connect ideas across chapters. 📣 Read the Stanford announcement: news.stanford.edu/stories/20… Our goal is simple: to make high-quality education in robot autonomy accessible to students, researchers, engineers, educators, and anyone curious about how autonomous robotic systems work. I believe that sharing knowledge is fundamental to the mission of a university. Autonomous robotics—and Physical AI more broadly—are among the defining technological frontiers of our time. We want to help more people understand these technologies, build with them, and advance the field. A huge thank you to my co-authors, @DanieleGammelli, Joseph Lorenzetti, @katielulula, and @GioeleZardini, and to everyone who helped bring this project to life. Explore the resources, use them in your own learning or teaching, and please share them with anyone who might find them useful! #Robotics #PhysicalAI #ArtificialIntelligence #AutonomousSystems #Education @Stanford @StanfordEng @StanfordAILab @StanfordHAI
35
207
27
1,024
90,767
A consequential choice will be made soon by government and companies as they are increasingly flooded with agentic AI inquiries that overcome the frictions that gated access (see the graphs - it has already started!) Two options: do you make services worse or better as a result?
267
178
79
1,371
165,679
I see someone writing COE.. pain in the butt. Also, mist likely the order will be cancelled.
151
Just 7 days left until the Identity MD hackathon comes to a close! ⌛️ $IMD Here’s the latest list of all current submissions 👇 @imdusd @IMD6900 @keelpad_imd @pepe2pepe_fun @perkshireimd @bunkervaultapp @pupateIMD @pfwafun @AgentCityrh @marginalia_md @ArewaOS26 @Pixel_Pool @Musefren @Quantum_Canary @PlayCourierWld @tungweb3 @pinkybond_fun @SovrnIMD @HarvestIMD @swarmpepes @imdplace @Numosimd @Pepesdotfamily @0xCompanyIMD
Introducing the first Identity MD Hackathon! Funded & organised by the community. Judged by IMD agents. 💰 $15,000+ in rewards 📅 Submissions: 7–21 October 🏆 Winners announced by 23 October Hire the swarm to launch on Ethereum, Robinhood or Base Criteria below 👇
8
8
4
74
5,128
Just another day at the Solana Stonk Exchange.
348
250
85
1,605
70,182
team keeps delivering.
eight days of $hooks. this is everything that shipped. 1. flap audited the pickhooks vault @flapdotsh reviewed it for eleven rounds. every finding fixed and proven on a fork before the next round, until it passed. every coin from the factory carries flap's low risk badge, and our panel runs inside flap's own token page. a tax pot that plays one of five games: bell, seat, persuade, ransom, war. 2. a sixth game: prophecy the dev stakes coins on a call about the market. holders lock on yes or no. at the deadline flap's ai oracle reads the market and the losing side burns. live since today. 3. coins with rules v2: 40 rules written into the coin. pools: the rules sit in the pancakeswap pool, the coin stays a plain token. sticky rules: the rule sits on the coins, not the wallet. split a bag into ten wallets and it still sells as slowly as one. hook script: write your own rules, no code. the chain checks them. cz mode: the dev's first buy lands in a lock. 4. markets inside the coin curve calls: rent a slice of the locked curve, keep the bnb above your strike. lever: long or short up to 5x from the coin's own curve. no oracle, no liquidation bot. dip bids: someone's dip bid is your floor. pump, keep the coins. dump, hand them in and take the bnb. 5. floors every pickhooks coin can get a bnb floor that only goes up. protected buy gets you the coin and a floor under it in one transaction. floor coins send part of the dev's own fee into that floor. 6. hook any coin any coin that trades on bnb can get a pickhooks pool next to its main pool. when a trade opens a gap between the two, the hook closes it in the same transaction. that profit used to go to mev bots. now a quarter goes to the trader. 7. launches nobody can front run dark launch: nobody can front run the first minute, because the coin does not exist yet. plus dev cover, rule auction and launch desk. 8. agents give your coin an agent. it rewrites the rules while it trades, inside limits fixed on chain. holders can veto it or fire it. and with @pickhooks/mcp on npm, your own claude can be the pilot. the part that matters for $hooks: a share of what pickhooks earns buys $hooks back and burns it. eight days in numbers: more than 195 million $hooks burned, 19.5% of the supply. 167 bnb ($124k) of buybacks in 843 rounds. 57 bnb ($43k) and 21 million $hooks locked as liquidity, lp at 0xdead. pickhooks.com back to building.
1
156
Lol. Remember Zuck says similar shit to convince people that Meta cares about kids safety. The product, business, tech, “partners”, and incentives are all aligned to extract money from vulnerable gamblers. In real world there are laws against this predatory behavior. Every ecommerce, payment, or trading platform has fraud protection for its users. Its legal and moral obligation of doing business. Pump has everything tailored to enable fraud. Stay away from that shit.
PUMPFUN IS REWARDING SCAMMERS! ...is what I would say if it were true Given the amount of takes surrounding a completely manipulated Callout Rewards table over the past day, I thought it's best if we give you data straight from the source. Below is a table which ranks the biggest earners alongside the exact same filters used in the table from that post - with one VERY important caveat: We INTENTIONALLY removed any callouts over 500m for illustrative purposes, which completely skews the average MC of all these users' last 100 callouts to the downside (sorry) Additionally, for the sake of the KOLs that I am exposing, I added a column for their average callout, since the previous author intentionally used median to hurt the optics of their callouts. I expect there to be some cherrypicking with some of the people on this list, but the fact remains: for as many 'bad traders' we reward, there are countless GREAT traders who deserve every penny they receive in rewards Building the most lucrative creator payouts program was never going to be an easy feat, and one of our main objectives continues to be rewarding ethical traders who are safe for users on our app, but we will get there eventually. If you are a good trader, you're entitled to a huge share of these rewards. Join Pumpfun now to be automatically enrolled in Callout Rewards.
Readers added context they thought people might want to know
On-chain analysis by Bubblemaps shows top earner 0xEthan (listed) used linked side wallets to buy before callouts and sell after, extracting ~$125k incl. rewards. Pump.fun admitted V1 incentivized spam and adjusted for quality over quantity. x.com/bubblemaps/sta… x.com/a1lon9/status/…
1
258
Don't even remember when I liked a project on BSC, but $HOOKS @pickhooks seems very promising. Things I like: a) Narrative: V4 hooks on @binance, Market validated on SOL and RH with no other hooks play on BSC b) Undervalued at 1.5m Vs Comps on SOL (solana:C1mBfBoDkwWfd6uTFZp62ARHLjeVp3bDpCDMfMZtPngE $10m+) & RH (robinhood:0x18e674231a58c239dc7daedcffe15ec3a24cff5c $20m+ mcap) c) Supply: ~19% supply burnt in first week d) Team: transparent + consistently releasing features. providing update on fomo; it appears they no their stuff and are committed. e) Smart wallets: @theunipcs + few others holding ~10% total; these wallets have a history of patiently holding selective tokens. f) Chart: ranging/consolidation for 5 days. floor + distribution is improving gradually with top wallets holding. Copy Trading wallets that aped after @theunipcs have slowly bled away at loos. Always a good leading indicator when a organic floor establishes. --- Risks: a) needs a runner from the launchpad & broader adoption on @binance .. seems like its progressing but no concrete example yet b) Strange vamp attempts by few KOLs. Rasmr entered and exited three times in last 3 days causing huge dips. Not sure if it is the usual dick behavior or intentional attempt to break the chart. Disclosure: watch the chart and do your research before aping, it is ranging a bit and you will get better entries with patience.. my entry is ~300K mcap so I am up significantly and generally I have much larger appetite to hold for long term.
hooks infra is live on pickhooks.com/infra. two new pieces, both on chain, open to any coin launched on pickhooks: 1. the HOOKS floor a bnb reserve per coin that only goes up. anyone can feed it. a holder can hand coins in at reserve / circulating, the coins burn, and the floor per coin rises for everyone who stays. nothing ever leaves except to a holder who hands in. $hooks has its floor open. 2. protected buy buy a coin and a floor under it in one transaction. the executor routes the swap through pancakeswap and takes a dip bid at the same time, so the position has a strike from block one. the executor keeps 0.25% of the bnb swapped, nothing else, and that fee buys $hooks back like every pad fee. and a new launch option: floor coins. when you launch a pool coin, you can give a share of your creator fee to the coin's floor. for good. no setter, no owner, the contract that receives your fee splits it on its own. the chosen share feeds the floor, the rest comes to you. and because a contract is the creator on record, the dev trades under the coin's rules like every buyer, first buy included. a coin where the dev's own fee lifts the floor under every holder. that is the pitch. the same calls are open to any wallet, bot or terminal. new contracts, tested on a fork and on mainnet, not audited by a third party yet.
1
15
1,022
These are good ideas but almost all of these require specialized domain specific context that remains current, guardrails, and evals. Better suited for vertical agents. Alternatively, the personal agent can tap into registry of vertical agents, hand off request, receive the results, triangulate with personal context, and then deliver to the user. There aren’t many consumer/personal use cases that can justify the cost. For companies its a way to capture person’a context and monetize it later.
If AI labs want personal assistants to take off at scale, they need to stop marketing vacations, reservations, party planning, etc. These are not problems. Focus on *the* problem facing Americans: the cost of living. Agents can offer a ton of help. Some ideas: (🧵)
150
Updating my @LinkedIn to “SI Lead at ..”
BWENEWS: TRUMP: WHITE HOUSE CONSIDERS ANYONE USING TERM ARTIFICIAL INTELLIGENCE INSTEAD OF SUPER INTELLIGENCE THE ENEMY 方程式新闻: 特朗普:白宫将任何使用“人工智能”而非“超级智能”一词的人视为敌人
91