robertboorman

You can do some pretty fun realtime VFX once the depth output is just another (Type)GPU buffer :D I basically yoinked my old 3D fire sim (that is actually a fluid sim but do not tell anyone) and plugged it into the depth pipeline together with hand tracking Everything here is built from scratch in TypeGPU, so inference, simulation and rendering can share GPU resources and interleave passes directly. That’s 2 NN inferences, a full 3D fluid sim, relighting and rendering in one command encoder, still under 16 ms with no CPU sync in between

Thank you all for the kind words under the demo I posted :D The demo is now officially public and open source for you to play with, modify and build on top of. Feel free to tag me if you make something cool with it :) Also thank you @iwoplaza for participating in my silly video haha You can run it like any other TypeGPU example here: typegpu.com/examples/#exampl… The source code lives in the TypeGPU monorepo: github.com/software-mansion/… The repacked weights used by the example live here: huggingface.co/reczkok/depth…

Depth-aware light injection in TypeGPU I got a 448x448 monocular depth model down to ~8 ms on my M4 Pro across ~250 dispatches, which is fast enough to use in realtime :D Since the inference is written directly in TypeGPU, I can just feed the depth buffer straight into the lighting pass. It never has to leave the GPU or go through any extra synchronization/interop step Inference, lighting and draw all go through the same command encoder.