Ayush Saraf (@ayush29feb) profile picture

ML Engineer @ Meta Reality Labs

New York, NY
Joined April 2010
Photo by Ayush Saraf (@ayush29feb)Photo by Ayush Saraf (@ayush29feb)Photo by Ayush Saraf (@ayush29feb)Photo by Ayush Saraf (@ayush29feb)Photo by Ayush Saraf (@ayush29feb)Photo by Ayush Saraf (@ayush29feb)Photo by Ayush Saraf (@ayush29feb)Photo by Ayush Saraf (@ayush29feb)Photo by Ayush Saraf (@ayush29feb)Photo by Ayush Saraf (@ayush29feb)
The most useful thing you can do for your company is build a CLI that can use everything inside the company. code, calendars, communication channels, docs. This is when agents really start thriving.
35
Ayush Saraf retweeted
Interesting learning rate schedule
18
97
9
2,045
75,827
Ayush Saraf retweeted
this is the coolest AirDrop you’ve ever seen, I promise. I vibe-coded the interaction with 6.2 Astra, turned one photo into a 3D scene with ML-SHARP, and made the ginkgo leaves with Tripo. It runs on Apple Vision Pro. Pinch a photo off your iPhone. Open your hand and you’re inside it.
109
347
65
4,253
219,764
Ayush Saraf retweeted
Meet AVM, a generative in-context Autoregressive View Model. Given one or more images or videos of a complex 3D scene, AVM streams 2D views as you steer a virtual camera. It reconstructs what the context supports and imagines the rest without building an explicit 3D representation. Find out more at autoregressive-view-model.gi….
10
41
13
325
25,314
Ayush Saraf retweeted
this is SICK
1
9
403
Ayush Saraf retweeted
Small bird, fast wings, Kolibri is here. 78B parameters. 3.46B active. Up to 1M tokens of context. Built in Europe. Now the weights are yours. Run it on your own hardware, under Apache 2.0.
435
1,008
490
9,792
2,695,991
Ayush Saraf retweeted
Introducing Praxis-1, an open-weight World Action Model that turns our video pre-training expertise into real world control for robots. Robot demonstration data is limited. But video data is infinite. Praxis-1 extends our bet that the best policy models will learn from video, and use that scale to bring embodied intelligence to every industry. Praxis-1 is built on our large-scale video pre-training and real-time infrastructure, and acts as a policy model for robotics developers and researchers that works across any embodiment or environment, no matter how complex. We are actively testing with early partners including Noble Machines, Standard Bots and Ultra, and will be releasing the weights publicly in the coming months. Learn more and request early access at runway.com/Praxis-1
50
55
42
493
120,404
Ayush Saraf retweeted
CS academia is dead. So where does that leave AI PhDs like me? Did my ICLR reviewer bidding today. Skimmed a few abstracts, and the methods are the exact same recipe I learned when I got into 3DV two years ago. Swap in a newer video gen base model and boom, new paper 🥲 Makes you wonder how many of the 60k submissions were actually thought up by AI. Meanwhile, World Labs' Atlas has basically solved 4D scenes. Academia is so far behind it's not even funny. I still remember the day GPT-6 Astra dropped. My feed was flooded with GPT + Blender doing inverse graphics and GPT driving robot arms through manipulation tasks. The results were so good I literally had to sit down. A year ago, I was dead sure LLMs could never have spatial intelligence. A year later, Astra slapped that belief right out of me 💀 There's no doubt Astra was post-trained on tons of 3D and manipulation data, and it's only going to get bigger and faster. To me, that means any domain that can be represented symbolically, with clean benchmarks for RL, is going to get swallowed by LLMs. Next to real LLM intelligence, most academic papers that add a bit of inductive bias and tune their way to SOTA are just roadkill waiting to happen. Sadly, these papers keep piling up and flooding every conference. It's inertia, plain and simple. We've been chasing SOTA for so long that it's hard to stop overnight. But make no mistake: the paradigms in a lot of areas converged long ago. What used to be research is now pure engineering, and the room for academics to add inductive biases is only going to shrink. So as a PhD student, I see two paths left. One: if you can't beat them, join them. Clean data, build infra, then go to industry and train foundation models. Two: go back to real science. Stop caring about squeezing out another 0.1% on a benchmark, and start asking why this works and that doesn't, with foundation models themselves as the object of study. As a friend put it: AI research might end up looking more and more like biology, except the organisms are silicon-based 🤣 Beyond these two, it's hard to see anything that won't get eaten by LLMs. Still, I'm pretty pessimistic. I can feel the value of human knowledge being eroded bit by bit. Everyone will get their own AlphaGo moment. What will academia even look like after this... 😭
38
123
28
1,182
167,581
Ayush Saraf retweeted
I probably watched every single @3blue1brown video from Grant Sanderson (big fan) and I am noticing this for the first time 🤯
148
169
50
6,966
413,945
Ayush Saraf retweeted
And yet everyone still uses COLMAP
Earlier, quite a few people who attended CVPR told me they came away pretty disappointed — including some extremely senior people in the field. At the time, I didn’t fully understand why. But after I attended ECCV, I finally did. There was a surprising amount of frustration at the conference, especially since it happened right after the GPT-6 release. People were openly wondering what traditional CV research is going to look like from here, and how much of it will eventually just be absorbed by large foundation models. Actually, when I gave talks about GenCeption to a few academic groups, I had already started to see that kind of frustration. GenCeption essentially leverages a large-scale pretrained diffusion model to build a single unified model that can match — and often outperform — specialized SOTA models across a surprisingly broad range of vision tasks, including DepthAnything3, D4RT, VGGT Omega, SAM3, Genmo, Lotus-2, and others. (The code is now open-sourced! genception.github.io/) Quite a few PhD students told me the work made them question the direction of their own research. Part of me was happy, because it meant GenCeption had genuinely changed how some people thought about the problem. But that feeling was quickly outweighed by how much I felt bad and empathized with the junior PhD students. Some of them seemed genuinely unsure where to go next, especially when years of work on specialized architectures or training recipes could potentially be absorbed by a sufficiently capable general pretrained model. GPT-6 seems to be just creating that same feeling on a much broader scale. It reminds me a lot of what happened with GPT-1 and BERT — tons of NLP PhDs' years of work and entire research directions perish overnight. There were even some pretty spicy takes floating around: “CV conferences are dead,” “academia is becoming disconnected from where some of the most important progress is actually happening,” “with increasingly noisy review systems and declining review quality, rejection sometimes is almost a signal of truly fundamental work,” and “conferences increasingly reward work delicately engineered around established topics or methodologies, rather than work truly at the frontier.” Maybe it's too harsh and too absolute, but the frustration behind them was definitely real. Conferences used to feel so fruitful, but this time it was hard to get genuinely excited. A lot of papers seemed to solve problems that no longer felt that important, or make progress that felt increasingly incremental. In the era of paper inflation, most people are busy trying to survive the research game, rather than thinking about how to go big after one or two years of diving deep into something. Most valuable skills such as critical thinking and taste never get trained well enough, from the beginning, all the way to the end. On the other hand, most people simply don't have enough context or a big enough picture about what problem actually matters. They get stuck optimizing for local problems that their immediate environment tells them are important. I suffered from this for a long time too, but luckily I gradually started to see a much bigger picture. My advice to junior PhD students is: when you start a project, try to collect as much context as possible before you even begin — by all means. One way is to explore the boundaries of existing models as aggressively as you can. Play with the SOTA models extensively. Keep throwing harder and harder problems at them until they fundamentally break. Understand not just what they can do today, but where their capabilities end and why. Another way is to put yourself wherever the strongest signals are. That could be a visionary academic lab, a team at an industry frontier lab, a genuinely insightful seminar, or simply a senior student or researcher who has seen a much broader landscape than you have. Expand your local context as much as possible. In an era where things are moving this fast, the information gap alone can be large enough to make years of carefully built work suddenly meaningless. Looking up and making sure you're walking in the right direction matters much more than keeping your head down and perfecting every step along the way — though, of course, being able to do both is the best :) We stand on the shoulders of giants not just to go further, but to see further — to understand which problems will actually matter in the future. If a problem is likely to be easily swallowed by large models once they have sufficient data, attention, or business interest, either avoid spending years engineering a specialized solution for it, or contribute the data and benchmarks that will help move the frontier forward faster. The problems that deserve the deepest thinking and genuinely new methods are the ones where existing models fundamentally cannot do the thing you need them to do. What I actually worry about is not that conferences will suddenly “die,” but that if they start becoming genuinely irrelevant to where the frontier is moving, they will simply fade away. And maybe one early sign of that is that a lot of people I used to want to meet aren’t really coming to conferences anymore — they’re busy with the big things happening at frontier labs, or, well, busy making money. It reminds me a little of what happened to control theory as the field matured. As many of the core problems were gradually solved, parts of the community started inventing increasingly artificial use cases, making increasingly unrealistic assumptions, and playing increasingly elaborate mathematical games within them. That possibility genuinely scares me — not because the work becomes unnecessarily sophisticated, but because it can become disconnected from problems that actually matter. I still believe that conferences will remain places people genuinely want to come to: to exchange ideas, reconnect with old friends, meet new people, and simply have fun together. We just need to keep doing the right things to make that happen :)
5
5
1
122
17,713
Ayush Saraf retweeted
A $1 Camera That Actually Works!
651
8,709
1,277
126,637
51,576,479
Ayush Saraf retweeted
the models have independently discovered diffusion lol
Tell Astra to open Paint in the browser and try to draw you
3
5
1
165
21,749
Ayush Saraf retweeted
Atlas is our scalable foundation model that can perceive, generate, reason, and interact with virtual and physical worlds. We're opening up early access in the coming weeks. Learn more and sign up here: worldlabs.ai/blog/atlas
16
28
6
638
102,328
Ayush Saraf retweeted
Replying to @Hamburgerai
2
9
307
106,744
Ayush Saraf retweeted
很喜欢这种把旅行照片收进田野笔记里的方式。 照片留住现场,旁边那枚略微缺墨的彩色橡皮章,只刻下最值得记住的轮廓。 树、穹顶、海岸都没有被讲满,反而像旅途中随手盖下的一页,安静,又很有地方感。 prompt: 请将我上传的每一张照片分别制作成一张独立的「橡皮章旅行田野笔记海报」,每张照片单独输出,不要多图拼贴。 整体采用4:3横版构图,将画面划分为左右两个区域,但不要绘制明显的分隔线。 左侧约占画面58%,忠实保留原始照片。准确保持主体身份、地形、建筑、植物、人物、空间关系、自然光影、真实质感和原有色彩氛围,仅进行克制的艺术出版物级摄影调色,并加入极轻微的细腻胶片颗粒。为适配版式可以自然裁切,但不得拉伸、扭曲、移动、替换或重新绘制主体。 右侧约占画面42%,使用温暖的米白色旧纸作为背景。纸张具有细微纤维、天然颗粒、轻微使用痕迹和哑光触感,同时保留大面积未经印刷的纸张留白,让空白成为版式的重要组成部分。 分析原始照片,从中提取最具地点辨识度的主体轮廓、建筑结构、地形走势、植物姿态、道路、水岸线或其他关键视觉关系,将其压缩成一枚小型多色橡皮章图像。 不要逐项复制照片中的全部内容。只保留能够让人一眼认出原始地点、主体和场景关系所必需的最少信息。删除人群、车辆、密集窗户、重复建筑、细碎植被、装饰构件和无关背景。 章印位于右侧纸张区域的中下部,整体只占右侧区域高度的约30%—38%,周围必须保留充足留白。章印不能放大成普通插画、完整风景画或品牌Logo。 根据原图构图决定章印的组织方式: - 标志性建筑:保留最有辨识度的外轮廓、屋顶、穹顶、拱门、塔楼或主要结构。 - 山地聚落:将建筑压缩成沿地形排列的少量阶梯状色块。 - 海岸风景:保留山体走势、聚落层级、岸线和少量断续水纹。 - 城市远景:保留主要天际线、一个标志性建筑及一两层远山。 - 自然景观:保留主要山体、树木、水岸或道路的方向关系。 - 近景遮挡物:如果其对原图叙事重要,可作为前景章印轮廓保留。 从原始照片中提取2—4种专色油墨。优先使用炭黑、深绿、砖红、赭黄、灰蓝、灰褐等经过降低饱和度的颜色,但不得强行套用固定色盘。保留原照片最有辨识度的色彩性格,仅允许一个小面积颜色作为视觉强调。 每一种颜色都要呈现为分别手工盖印的效果: 真实橡皮章雕刻纹理、手工刻痕、粗细不均的排线、轮廓缺口、断裂边缘、干燥缺墨、纸张透底、颗粒状油墨、压力不均、局部重影,以及约1—2毫米的轻微套色偏移。 不同色层之间允许出现自然错位,边缘不能数字化平滑。印迹应像真正刻制后压在旧纸上的橡皮章,而不是套用滤镜的照片、平滑矢量插画或线稿Logo。 根据照片中的地点、主题和视觉意象生成文字: 地点英文名称 No. 编号 三个简短英文关键词 西元年份 文字放在章印下方或邻近留白处,使用小型、克制、略带机械误差的打字机字体。排版应像旅行者的田野记录,而不是广告标题。确保所有文字拼写准确,不添加无关标语、品牌或装饰性文案。 整体气质像建筑师、旅行作家或自然观察者保存的田野笔记:安静、克制、触感真实、地域明确、带有手工误差和收藏感。照片负责记录现场,章印负责留下记忆中最值得辨认的部分。 Avoid:明显的中央分隔线、圆形印章、中文红色印章、邮票齿孔、蜡封、贴纸拼贴、旅游纪念品模板、平滑矢量Logo、通用城市图标、完整复制所有建筑、密集刻画、儿童手工感、卡通风格、3D渲染、塑料质感、光滑数字渐变、过度饱和、过多文字、装饰堆积,以及重绘或改变左侧原始照片。
301
891
337
11,028
5,394,462
Ayush Saraf retweeted
Winning at all costs
WATCH: A humanoid robot training for the “Robot Olympics” in Beijing runs too fast, fails to stop, slams into a safety cushion, and breaks at the waist
10
9
2
247
75,729
So cool to finally see a product for high quality 3D splat reconstruction with low effort captures! This is a foundational step towards the 3D stack
Large, consistent, and high-quality 3D worlds from simple iPhone recordings. Works for multi-room scenes, fills missing gaps consistently while maintaining high quality, and it’s super easy to use. Now in beta with partners and coming soon to the SpAItial API. Link to the scenes in the comments so you can navigate this world yourself 👇🏽
121
This will be big for 3D. Having worked on splatting for years, this capture device has the best trade off for quality v/s capture effort. They already have the user-base that would be willing to put that level of effort in!
Don't just rewatch it. Re-enter it 🌎 Insta360 X6 introduces all-new Spatial Capture. Record a place in 360, bring it into the app, and it becomes a 3D model, yours to keep. Every angle, exactly where you left it. Coming August 12. #Insta360X6 #GaussianSplatting #360camera
1
3
621
Personal update: I've joined Anthropic. Instead of teaching humans, I'll be teaching LLMs instead. GS
Readers added context they thought people might want to know
This is a parody account as indicated in its bio; the real Gilbert Strang retired from MIT in 2023 and has no affiliation with Anthropic. x.com/GilStrangMIT en.wikipedia.org/wiki/Gilbert_S… news.mit.edu/2023/gilbert-s…
99
147
180
3,159
477,694
Ayush Saraf retweeted
Today we launched Muse Image and are previewing Muse Video! 🚀 ai.meta.com/blog/introducing… 👈 Muse Image does agentic image generation! It's great at tool use and search. I highly recommend trying it out on meta.ai! Excited to continue! 🧑‍🍳🔥
1
11
2
53
9,443