Our new 3D AI output significantly outperforms Hunyuan3D-2.5

Special thanks to @GoogleDeepMind for inviting me to try out Genie 3. I'm excited to share my thoughts on this early research prototype and also some of my live recordings below: I spent the whole day playing with the system and when it works, it is truly mind blowing🤯. It is the first neural game engine / world model I have tried that generalizes so well and has long term world consistency. Here’s a couple of examples from my live recording and some thoughts on what it means for the future of gaming, robotics, digital experiences and ASI. Where it shines: - Truly general-purpose and quick startup time. Works exceptionally well for gaming environments but also generalizes to other industrial and real-world scenarios. - It learns physics. Although there are systematic failures even for rigid body physics, it was clear to me that it can learn game engine and non-rigid physics without an underlying engine (and in limit learn from game engines via training data). - It works exceptionally well for stylized environments with characters walking around. This will have implications for concept artists, level designers and game devs. - It is way more fun than video models, indicating that there are high retention consumer experiences waiting to be built with this in the future - Photorealistic walk throughs and drone shots work exceptionally well - Global illumination and lighting works surprisingly well - Visual memory is quite powerful and the same objects approximately remain coherent under occlusion and longer time horizons Open Problems: - Physics is still hard and there are obvious failure cases when I tried the classical intuitive physics experiments from psychology (tower of blocks). - Social and multi-agent interactions are tricky to handle. 1vs1 combat games do not work - Long instruction following and simple combinatorial game logic fails (e.g. collect some points / keys etc, go to the door, unlock and so on) - Action space is limited - It is far from being a real game engines and has a long way to go but this is a clear glimpse into the future. The Future: - It is impressive enough for me to have strong conviction that this is going to disrupt the gaming industry. It is super early days and there are a lot of failures but the writing is on the wall. Lots of challenging scientific, engineering and scaling problems to be solved but it is going to happen in the next 5 years. - This is the final piece before we get full AGI and now I think we are well on our way to truly solve it once something like this is scaled up. In many ways it is more ASI than AGI but this is a matter of definitions. The fidelity and generalizability will reach human-level and quickly surpass humans - People are going to combine this with 3D AI and LLMs to build AAA games.