trying to feel the magic. prev: global research lead at @canva | cofounder at @leonardoai
sydney - florida - SF
Joined April 2022
- Tweets12.7K
- Following894
- Followers9.9K
- Likes26.6K
Ethan retweeted
A lesser highlighted, but super interesting part that might have more broad implications is that asymmetric init that dampens negative values slightly, I don’t know why this helps, but the best guess is early symmetry breaking providing an early bias
Ethan retweeted
I think this is one of the most fascinating projects I’ve tinkered on yet. Parameterizing our weights as a learned weighted blend of symexp and regular linear weights gives up to ~1.42x training speedup wallclock, and can be fused back into standard weights for deployment🧵
Ethan retweeted
my partner ( @torchcompiled ) and I are looking for a sublet for SF for July. craigslist is returning null, so we turn to the twitterverse
open to areas but inner city preferred. open to grouphouse or whole apt situations. pls DM if you know of any leads <3 we are clean, fun and cool. will repay in premium Graffeo coffee beans for anyone who finds a lead
Ethan retweeted
Coordinates were not meant to be discrete, you risk class imbalance and no intrinsic knowledge of number line or a loss that caters to that, just use a linear head to output 4 coordinates with an MSE loss. Fourier embed at input side. Here’s a paper we did on that
This #CVPR2026 paper from our research team is trending #1 on @HuggingFace 🤗
Meet LocateAnything: a vision-language detection model that rethinks bounding box prediction. For AI agents and robots, “seeing” is only useful if a model can pinpoint where something is fast enough to act.
Trained on 138M high-quality samples, LocateAnything decodes bounding boxes in parallel instead of one coordinate at a time, improving localization accuracy while dramatically increasing throughput for visual grounding and detection.
Project page: nvda.ws/4dKSohb
Ethan retweeted
[1/2] Most image gen models output flat pixels. Users need layers. Our MRT (CVPR'2026) closes that gap and supports:
• text → layers
• image → layers
• layers → layers
🔥 New SOTA on all three tasks
🔥 Beats Qwen-Image-Layered
🔥 10–100× faster, 50–90% less GPU memory
Ethan retweeted
Fellas be tweeting about GPT water usage whilst sitting on the toilet pooping into the same drinkable tap water that comes out of your sink.
Ethan retweeted
This is a neat result from @torchcompiled: adding a tiny cosine branch to each linear layer cuts pretraining steps by up to 32% with only ~4% extra params.
I built an interactive @marimo_io notebook to see the core idea: same MLP trained with different branch activations racing to fit a signal. cosnet consistently wins because it naturally locks onto the high-frequency details that other activations are slower to learn.
molab.marimo.io/notebooks/nb…
"NOBLE: Accelerating Transformers with Nonlinear Low-Rank Branches"
This paper shows that if you add a tiny permanent nonlinear low rank branch (with a cosine bottleneck) to each Transformer linear layer, you can teach the model hard-to-fit details much more efficiently.
This cuts pre-training steps by up to 32% and wall clock time by up to 22%, with only a small overhead.
Ethan retweeted
"NOBLE: Accelerating Transformers with Nonlinear Low-Rank Branches"
This paper shows that if you add a tiny permanent nonlinear low rank branch (with a cosine bottleneck) to each Transformer linear layer, you can teach the model hard-to-fit details much more efficiently.
This cuts pre-training steps by up to 32% and wall clock time by up to 22%, with only a small overhead.
Ethan retweeted
As part of @PrimeIntellect's RL residency program, I've been exploring how to do multi-agent RL using their current stack (from verifiers + prime-rl to lab experiments with hosted training /evals) and thinking about how it could be extended to support these abstractions natively. I've summarized my findings the blogpost below and I'll leave a few comments here, too...
Ethan retweeted
Why does this help performance? We hypothesize that the linear transform carries the majority of the burden in fitting to a target function, covering the coarser/smoother aspects (which many natural functions resemble) while NOBLE can assist in high-frequency residuals
Ethan retweeted
Have you ever gotten tired of boring plain linear layers and wanted a more complex function? We find that attaching low rank nonlinear residual functions can significantly accelerate pretraining, with an identified variant, CosNet, consistently observing 20+% wallclock speedup!
Ethan retweeted
We’re excited to launch #MagicLayers globally at Canva 🚀🚀🚀
This is currently the most advanced image-to-layer decomposition model in the world-purpose-built for design-native content.
⚡ 20–200× faster than Qwen-Image-Layer
🎨 Super strong on designs
canva.com/newsroom/news/magi…
Ethan retweeted
Transparent image generation is taking off—GPT-4o-Image just showed what’s possible for single-layer transparent image generation.
Need a high-quality multi-layer dataset? Check out our PrismLayers:
Homepage: prism-layers.github.io/
HF dataset: huggingface.co/datasets/artp…
Ethan retweeted
Code is up!
