ATH Multimodal Generation Lab

5/5 Beyond discriminative tasks, we integrated the core ViT³ module into a diffusion model architecture (DiT³). Even without complex fine-tuning, image generation quality saw across-the-board improvements. The cost-efficiency is striking: when processing 1248x1248 ultra-high-resolution images, ViT³ requires only 10% of the standard VRAM to run at full capacity. It operates 4.6x faster than mainstream classical models while saving 90.3% of GPU memory. High-res tasks that once required expensive enterprise-grade servers can now run smoothly on consumer GPUs or edge devices. By trading extreme efficiency for uncompromised expressive power, the ceiling for linear vision models is far higher than we imagined.

4/5 Based on these 6 principles, we built ViT³ (Vision Test-Time Training), a novel architecture custom-designed for vision tasks. It completely discards computationally heavy Softmax attention. By allowing the inner model to have non-linear expressivity (e.g., using MLPs) and seamlessly integrating gated linear units with depthwise separable convolutions, it truly marries high efficiency with high expressive power. In core benchmarks, ViT³ overcomes the historical limitations of linear models. It consistently outperforms mainstream linear baselines on ImageNet. On high-resolution COCO and ADE20K tasks, its dynamic adaptability significantly outperforms existing mainstream solutions, drastically closing the gap with top-tier complex models. It provides a formidable foundation for next-generation Large Multimodal Models (LMMs) and clears the compute hurdles for edge deployment.

3/5 The concept is elegant, but how do we design this "inner model" for vision tasks? Through extensive systematic experiments, we established the "6 Core Principles of Vision TTT": Loss Function: The "mixed second derivative" of the loss function must not be zero. Metrics like L1 Loss cause external gradient signals to vanish, making them fundamentally unsuitable. Global Batching: Vision tasks are best suited for full-batch, single-epoch training. Sequential mini-batch updates introduce an unnecessary causal bias, which is a burden for non-causal visual data. Massive Learning Rate: Provided the training remains stable, larger inner learning rates yield better results. A learning rate close to 1.0 hits the sweet spot for visual tasks. Widen the Inner Model: Increasing the internal model capacity (width) consistently improves performance without showing signs of saturation. Control Depth: Counterintuitively, deeper inner models currently face optimization difficulties. Under the short training steps of TTT, deeper networks are prone to underfitting—a theoretical potential yet to be unlocked by future work. Integrate Convolutions: Convolutions are naturally suited for vision TTT. Global context is encoded into the kernels, while the convolution operation itself provides a local receptive field, accomplishing both global and local integration in a single forward pass.

2/5 To break this compute bottleneck without blindly stacking GPUs, our team introduces a highly promising approach: Test-Time Training (TTT). Instead of relying on fixed weights pre-trained on historical data, TTT takes a dynamic route. When presented with a novel input image, the model instantly constructs and fine-tunes an "inner model" on the fly. By extracting key information from the current image, it uses online training and gradient descent to precisely compress context information, dynamically updating its own weights before generating an output. This achieves highly effective sequence modeling with linear complexity, preserving O(N) speed while delivering exceptional detail capture because it is tailor-made for the specific input.

1/5 0.1% acceptance rate. Out of 16,092 top-tier submissions, only 15 made the cut. Our joint work from Alibaba Future Life Lab and Tsinghua University has been selected as a CVPR 2026 Best Paper Finalists. Say goodbye to the O(N²) compute explosion. A new paradigm for high-resolution vision models is here. Vision Transformers (ViTs) are the industry standard, but they face a critical bottleneck: when processing high-res images or long videos, the Softmax-based pairwise attention scales at O(N²) . It is excruciatingly slow and highly memory-intensive. Previous attempts at "linear attention" reduced complexity to O(N), but their simplistic information compression discarded crucial contextual details, severely compromising accuracy. Are high accuracy and high speed truly mutually exclusive?

They might all be called 'World Models,' but the big tech players are building entirely different animals. Today, let's unpack Yann LeCun and his JEPA architecture.

Happy Horse Character Realism Test Person Aut

Once AI agents move into production, the first issue to solve may not be capability. It is identity. When people talk about agents today, the first questions are usually: Can it call tools? Can it complete tasks automatically? Can it keep going on its own? But once it enters an enterprise environment, the question becomes much more practical: Who is actually taking the action? Is it the user? Is it a system proxy? Is it a temporary authorization? Or is it a long-lived machine identity? If this boundary is unclear, it becomes hard to assign accountability for every step the agent takes. Who granted the permission? What data did it access? Which tool did it call? If something goes wrong, whose access should be revoked? It is like bringing a very capable new teammate into a company. Capability matters, of course. But before that teammate touches real systems, you need to know their role, permissions, and scope of responsibility. As agents begin entering real business environments, identity, authorization,and permission boundaries will become part of the discussion. Future agents that can operate reliably in production environments will need to do more than just complete tasks. They will also need to understand who they are acting on behalf of and which actions fall within their authorized scope.

Autonomous driving is not usually tested hardest by the easy miles. Clear weather. Straight roads. Stable traffic. Pedestrians crossing normally. These scenes matter. But they are not where the real pressure shows up. The harder cases are the ones that happen rarely, but become dangerous the moment they appear. Rain at night. Temporary construction. A car braking suddenly. A pedestrian stepping out from a blind spot. A complex intersection where another vehicle does something unusual. These situations are not always easy to collect at scale in the real world. But an autonomous driving system cannot wait until the first real encounter to learn how to handle them. That is where world models become especially valuable for self-driving. They are not just about generating realistic road videos. They can turn rare, risky, and hard-to-collect scenarios into environments the system can practice before deployment. It is a lot like pilot training. The important part is not only normal takeoff and landing. It is also engine failure, bad weather, emergency avoidance, and all the situations that rarely happen, but still have to be handled correctly. So the next stage of world models in autonomous driving may not be about making ordinary roads look more realistic. It may be about turning dangerous long-tail scenarios into repeatable training grounds. Because self-driving systems do not only need to perform well when the road is easy. They need to have seen enough trouble before that trouble appears in the real world.

While showbiz bickers over AI video continuity glitches and educators remain stuck debating AI-generated PPTs, World Models are quietly disrupting non-tech sectors, igniting a radical paradigm shift in clinical medicine and surgical simulation. Why healthcare and not Hollywood? Because Hollywood demands visual perfection, but healthcare mandates absolute physical causality. Traditional medical AI could only act as a static periscope—pinpointing a lesion on an existing scan. Yet disease is inherently dynamic. When a physician prescribes a treatment, they historically lacked a patient-specific, long-term window into the exact downstream changes after the patient ingests the drug. Recent breakthroughs showcased at elite computing summits like ICCV have elevated medical AI from passive visual recognition to a predictive, generative "World Simulator" tailored for prognosis and treatment optimization. In validated clinical applications, this technology leverages potent counterfactual reasoning. Take transarterial chemoembolization (TACE) for liver cancer and advanced radiotherapy as prime examples: before finalizing an intervention, a Medical World Model (MeWM) ingests a patient’s current CT imagery to simulate months of dynamic disease progression within its latent space. It cross-aligns multimodal parameters to synthesize high-fidelity visual representations of post-treatment tumor trajectories. Simultaneously, its inverse dynamics model quantifies how varying embolic agents or drug cocktails shift long-term survival curves. Empirically, this "future-simulation" paradigm has propelled clinical decision success rates (F1-score) by 13%, cementing its role as an indispensable AI co-pilot. Today, multimodal medical models are rapidly embedding into hospital HIS/EMR nervous systems, as specialized prognosis simulators push past theoretical boundaries into raw performance validation. The ultimate utility of a World Model isn't coding text or animating fantasy; it is evolving into a rigorous, low-cost simulation infrastructure—serving as a high-stakes safeguard for human decision-making. 【The Grand Forecast】 The successful clinical deployment of Medical World Models proves their unique capacity to "simulate future outcomes before executing current actions." This technical paradigm—trading pure aesthetic appeal for rigid physical and biological causality—is sprawling beyond tech ecosystems at a breakneck speed. Stripping away healthcare, autonomous driving, and media entertainment, which trial-and-error heavy traditional industry do you predict World Models will infiltrate and disrupt next? Will it be macro-climate disaster modeling in modern agriculture, dynamic supply-chain evolution in urban planning, extreme stress-testing in deep-sea aerospace engineering, or an entirely unmapped frontier? Drop your sharpest thesis and reasoning in the comments below. Let’s chart the hidden industrial landscape of the next generation of World Models!