
I'm a research scientist @Meta in Pittsburgh where I work on audio-visual modeling for photorealistic avatars.
Joined June 2020
- Tweets46
- Following171
- Followers731
- Likes130
‼️500h of 3D motion data released‼️
Our team at the Codec Avatars Lab just released a large scale dataset of 3D tracked human motion, including audio and text annotations. Check it out here: meta.com/emerging-tech/codec…
Working on your next big speech paper but still looking for a suitable dataset?
Check out EARS: 100h of full-band expressive, anechoic recordings of speech from 107 speakers with 22 different emotions, 7 different reading styles, and more.
sp-uhh.github.io/ears_datase…
Real Acoustic Fields are here!
Check out or dataset of densely captured room impulse responses paired with multi-view images! See you all on CVPR :)
project page: facebookresearch.github.io/r…
arxiv: arxiv.org/abs/2403.18821
Motion generation for photorealistic avatars? Say no more!
Check out how we animate full body avatars exclusively from audio input!
Paper: arxiv.org/abs/2401.01885
Project page: people.eecs.berkeley.edu/~ev…
Dataset + Code: github.com/facebookresearch/…
3D body models now have sound!
We demonstrate 3D spatial audio synthesis for 3D full body models. (NeurIPS 2023 Spotlight)
Paper: openreview.net/pdf?id=zQTi3p…
Dataset + Code (to be released): github.com/facebookresearch/…
**Dataset Release!**
We released high-quality 3D face data of 13 persons captured with up to 150 cameras while performing 100+ facial expressions!
github.com/facebookresearch/…
Photorealistic 3D avatars for everyone?! Check out this exciting work from our lab in Pittsburgh!
tinyurl.com/4xk25spe
Check out our @CVPR 2022 oral, presenting a new paradigm for audio-visual speech enhancement! We demonstrate high-quality results with a guarantee that noise can not bleed through. On top, the system allows for simultaneous voice conversion.
arxiv.org/abs/2203.17263
Some news here! We share the Facestar Dataset, a high-resolution, high-quality dataset with conversational freeform speech!
Also check out how we used that dataset for audio-visual speech enhancement in our @CVPR paper.
Get it here: github.com/facebookresearch/…
Diffusion Probabilistic Models for Speech Enhancement!
While many generative approaches still lag behind regression-based speech enhancement, we show that diffusion models prove to be particularly robust to domain shifts. @shinjiw_at_cmu
arxiv.org/abs/2202.05256



