Research Scientist at Bytedance Seed.
Joined July 2021
- Tweets39
- Following287
- Followers442
- Likes298
Haotong Lin retweeted
Single video in, 4D human out.
4DAnyone turns a casual monocular video into a 4DGS model.
No camera rig, no calibration, no tripod.
Project: 4danyone.github.io/
Haotong Lin retweeted
Introducing: Long-Tail Internet Photo Reconstruction (CVPR’26)
We go beyond densely captured imagery to train more general 3D foundation models for the long tail of noisy, sparse, incomplete Internet photo collections of 3D scenes. Yet, we face a data bottleneck: models need ground truth for these long-tail scenes, which classical SfM fails to provide. How do we bypass it?
We break this bottleneck with two key contributions. First, we introduce MegaDepth-X, a large new dataset of scenes with high-quality 3D supervision. Second, we propose a new way to simulate difficult image sets for training.
Project page: megadepth-x.github.io/
Haotong Lin retweeted
Excited to share our CVPR’26 work! Moving beyond dense captures to long-tail Internet photos, we introduce MegaDepth-X and sparsity-aware sampling for 3D reconstruction.
Great work led by Yuan (@yuanli16342871) and the team.
Project page: megadepth-x.github.io
Introducing: Long-Tail Internet Photo Reconstruction (CVPR’26)
We go beyond densely captured imagery to train more general 3D foundation models for the long tail of noisy, sparse, incomplete Internet photo collections of 3D scenes. Yet, we face a data bottleneck: models need ground truth for these long-tail scenes, which classical SfM fails to provide. How do we bypass it?
We break this bottleneck with two key contributions. First, we introduce MegaDepth-X, a large new dataset of scenes with high-quality 3D supervision. Second, we propose a new way to simulate difficult image sets for training.
Project page: megadepth-x.github.io/
Haotong Lin retweeted
Happy to introduce Habitat-GS, a non-intrusive extension of Habitat-Sim that brings dynamic Gaussian Splatting for photorealistic rendering and comes with hundreds of high-quality 3DGS scene assets, aiming to empowering navigation research.
Code: github.com/zju3dv/habitat-gs
Haotong Lin retweeted
In our comparison on long image sequences, Scal3R consistently outperforms DA3.
In such cases, Scal3R offers a viable alternative.
Feel free to try Scal3R: github.com/zju3dv/Scal3R
Haotong Lin retweeted
Today, we released Lyra 2.0, a framework for generating persistent, explorable 3D worlds at scale, from NVIDIA Research.
Generating large-scale, complex environments is difficult for AI models. Current models often “forget” what spaces look like and lose track of movement over time, causing objects to shift, blur, or appear inconsistent. This prevents them from creating the reliable 3D environments required for downstream simulations. Lyra 2.0 solves these issues by:
✅ Maintaining per-frame 3D geometry to retrieve past frames and establish spatial correspondences
✅ Using self-augmented training to correct its own temporal drifting.
Lyra 2.0 turns an image into a 3D world you can walk through, look back, and drop a robot into for real-time rendering, simulation, and immersive applications.
➡️ Learn more: research.nvidia.com/labs/sil…
📄 Read the paper: arxiv.org/abs/2604.13036
Haotong Lin retweeted
Spatial reconstruction is a long-context problem: real scenes come with hundreds of images. But O(N²) transformer-based models don’t scale efficiently.
Introducing: 🤐ZipMap (CVPR ’26): Linear-Time, Stateful 3D Reconstruction via Test-Time Training (TTT).
ZipMap “zips” a large image collection into an implicit TTT scene state in a single linear-time operation. The state will then be decoded into spatial outputs, and can be queried efficiently for novel-view geometry and appearance (~100 FPS)
ZipMap is not only much faster (>20× faster than VGGT), but also matches or surpasses the accuracy of all SOTA models.
Haotong Lin retweeted
papers are kind of like movies: the first one is usually the best, and the sequels tend to get more complicated but not really more exciting. But that totally doesn’t apply to the DepthAnything series. @bingyikang's team somehow keeps making things simpler and more scalable each time.
in this new version, they basically show that a strong representation encoder plus a depth-ray prediction objective is enough (you see the RAE vibes too, right?) to get solid, general spatial perception across a bunch of tasks.
people often say they hate computer vision because it’s messy--too many tasks, too many data types, too many moving parts. but that’s exactly why I love it. I think the biggest AI breakthroughs are going to come quietly from vision and then suddenly leapfrog everything else, changing how AI interacts with the real world and with us.
pretty soon we’ll realize vision is not a big list of tasks--it’s a perspective. a perspective about modeling continuous sensory data, building layered representations of the world, and inching toward human-like intelligence. and tbh we’re watching this happen every day, behind all the hype, as all these different '"tasks" slowly start to merge.
After a year of team work, we're thrilled to introduce Depth Anything 3 (DA3)! 🚀
Aiming for human-like spatial perception, DA3 extends monocular depth estimation to any-view scenarios, including single images, multi-view images, and video.
In pursuit of minimal modeling, DA3 reveals two key insights:
💎 A plain transformer (e.g., vanilla DINO) is enough. No specialized architecture.
✨ A single depth-ray representation is enough. No complex 3D tasks.
Three series of models have been released: the main DA3 series, a monocular metric estimation series, and a monocular depth estimation series.
The core team members, aside from me: @HaotongLin, Sili Chen, Jun Hao Liew, @donydchen.
👇(1/n)
#DepthAnything3
Haotong Lin retweeted
After a year of team work, we're thrilled to introduce Depth Anything 3 (DA3)! 🚀
Aiming for human-like spatial perception, DA3 extends monocular depth estimation to any-view scenarios, including single images, multi-view images, and video.
In pursuit of minimal modeling, DA3 reveals two key insights:
💎 A plain transformer (e.g., vanilla DINO) is enough. No specialized architecture.
✨ A single depth-ray representation is enough. No complex 3D tasks.
Three series of models have been released: the main DA3 series, a monocular metric estimation series, and a monocular depth estimation series.
The core team members, aside from me: @HaotongLin, Sili Chen, Jun Hao Liew, @donydchen.
👇(1/n)
#DepthAnything3
Haotong Lin retweeted
Pixel-Perfect-Depth: the paper aims to fix Marigold's loss of sharpness induced by VAE by using VFMs (VGGT/DAv2) and a DiT-based pixel decoder to refine the predictions and achieve clean depth discontinuities. Video by authors.
Haotong Lin retweeted
🚀 We release SpatialTrackerV2: the first feedforward model for dynamic 3D reconstruction and 3D point tracking — all at once!
Reconstruct dynamic scenes and predict pixel-wise 3D motion in seconds.
🔗 Webpage: spatialtracker.github.io/
🔍 Online Demo: huggingface.co/spaces/Yuxihe…
Haotong Lin retweeted
Towards Depth Foundation Model: Recent Trends in Vision-Based Depth Estimation
@realzhenxu, Hongyu Zhou, @pengsida, @HaotongLin, @ghy990324, @jiahaoshao1, Peishan Yang, Qinglin Yang, Sheng Miao, @XingyiHe1, Yifan Wang, Yue Wang, @ruizhen_hu, @yiyi_liao_, @XiaoweiZhou5, Hujun Bao
tl;dr: survey
arxiv.org/abs/2507.11540
Haotong Lin retweeted
Recently, I've been playing with my iPhone ToF sensor, but the problem has always been the abysmal resolution (256x192). The team behind DepthAnything released PromptDepthAnything that fixes this. Using @Polycam3D to collect the raw data, @Gradio to generate a UI, and @rerundotio to visualize. Links at the end of the thread
Haotong Lin retweeted
Excited to share our work MatchAnything:
We pre-train strong universal image matching models that exhibit remarkable generalizability on unseen multi-modality matching and registration tasks.
Project page: zju3dv.github.io/MatchAnythi…
Huggingface Demo: huggingface.co/spaces/Little…
Haotong Lin retweeted
Want to use Depth Anything, but need metric depth rather than relative depth?
Thrilled to introduce Prompt Depth Anything, a new paradigm for accurate metric depth estimation with up to 4K resolution.
👉Key Message: Depth foundation models like DA have already internalized rich geometric knowledge of the 3D world but lack a proper way to elicit it. Inspired by the success of prompting in LLMs, we propose prompting Depth Anything with metric cues to produce metric depth. This method proves to be very effective when using a low-cost lidar (e.g., iPhone's LiDAR), which is widely available, as prompts. We believe the prompt can generalize to other forms as long as scale information is provided.
Prompt Depth Anything offers
1⃣A series of models for iPhone lidars.
2⃣4D reconstruction from monocular videos (captured with iPhone).
3⃣Improved generalization ability for robot manipulation, e.g. Training on cans but generalizing on glasses.
4⃣More detailed depth annotations for the ScanNet++ dataset.
The first author is our excellent intern @HaotongLin.
Paper: huggingface.co/papers/2412.1…
Huggingface: huggingface.co/papers/2412.1…
Project Page: promptda.github.io
Code: github.com/DepthAnything/Pro…
Haotong Lin retweeted
(1/8) Ever wanted to create an avatar of yourself that interacts realistically with different lighting?
In our CVPR 2024 Highlight🌟paper, we present a method for creating relightable and animatable avatars from only sparse/monocular video.
Project Page:
zju3dv.github.io/relightable…