It's pronounced vanilla-gee | Author of Mediabunny and Marble Blast Web. Love working on technically challenging projects that bring the best out of the web.
Darmstadt, Deutschland
Joined July 2014
- Tweets3.1K
- Following127
- Followers4.3K
- Likes6.7K
Pinned Tweet
Mediabunny 1.42.0 is now out, adding THE most requested feature: full read/write support for HLS! 🚀
It's truly a game changer for client-side media processing. Here, I'm creating 5 renditions from a 40-second video in ~10 seconds, fully client-side, without a transcode server:
Cursed fact: It turns out that basically ALL videos created with WebCodecs up to this point have audio that's slightly out of sync with the video!
This is due to a quirk in how WebCodecs works: Many audio encoders add silence padding at the start of the stream to allow the codec to converge once the real data starts. This is standard and correct behavior. The problem is that it hides the fact that it does this and never communicates how much padding it actually added. For AAC, this is usually 44 milliseconds of delay at 48 kHz (2112 samples).
Mediabunny (and also my previous libraries mp4-muxer and webm-muxer) then trust the WebCodecs timestamps and write the data into the file - leading to an output file that's slightly desynced. Thankfully, 44 ms is low enough that most people don't notice it at all, but for any serious usecase I consider it unacceptable.
The upcoming version of Mediabunny will fix this once and for all, for all codecs and containers. This means audio stays perfectly synced with video no matter how often you process it 😌
Mediabunny v1.62.0 (🚀) ships with an insane amount of fixes and improvements aimed at making Mediabunny extremely robust against adversarial and malicious inputs (files that exhaust memory or clog the CPU).
In some cases, malicious files could balloon memory usage to gigabytes and now use less than a megabyte when being processed.
My goal is that you can use Mediabunny confidently (in your web app or on your server) and know that a bad file won't freeze or crash your program. Bad inputs will simply throw early now.
See the full list of fixes here:
github.com/Vanilagy/mediabun…
Currently going over the entirety of Mediabunny to harden it against DoS attacks!
Mediabunny cannot be exploited like FFmpeg can because it runs in a JavaScript sandbox, but one can still construct strange synthetic files that either:
- Lead to heap exhaustion
- Clog the CPU for a long time (or forever)
With the next Mediabunny update: no more. I'm fixing all algorithms that are superlinear in file length, either in CPU time or memory, and making sure that all inputs either parse instantly or fail instantly - with a useful message if they are malformed or malicious.
Mediabunny is already very stable, but I want people to be confident that it can handle any untrusted garbage that one throws at it 💪
Huge shoutout to @jittervideo for becoming Mediabunny's next silver sponsor!! ❤️
I'll be working together with them to push their video performance to the next level.
This release is mostly just browser bug workarounds 💀
I am embarking on "Project Sync", a cleanup pass over Mediabunny to make sure it has sample-perfect audio/video sync everywhere.
This is currently not guaranteed: WebCodecs audio encoders shift most audio by some uncommunicated amount (known as the encoder priming delay), and Mediabunny does not compensate for it. This means that repeated conversions through Mediabunny will keep shifting the audio forward and insert silence at the start.
The good thing: I can detect this priming delay empirically by encoding a small impulse signal and then seeing by how much the encoder delays it.
Another cursed workaround. This time for a Firefox bug where the encoder shifts the output timestamps by the frame durations for no apparent reason.
I report all browser bugs I encounter, but I also immediately ship workarounds so that Mediabunny is as reliable as possible.
In fact, I found three Firefox bugs at once!
I'll report them, but it's a more frustrating process with Firefox. Reporting Chrome media bugs is always such a pleasure: they literally respond the next business day and take every issue seriously. It gives you the feeling that they really care about making a perfect browser.
Firefox issues often get ignored for months. I don't want to judge - perhaps they are overloaded with work - but it's still meh.
Wow, HORRIBLE Firefox bug: Its VideoEncoder on Windows produces invalid H.264 video!! Straight up! It produces video that it can't even play back itself 😂
This is not only pretty embarrassing but also pretty bad; it means most Firefox users are producing unplayable videos rn
An interesting potential new feature of Mediabunny would be the durable creation of new media files that survives even page reloads.
Imagine a video editor that creates low-res proxies under the hood. If the video is an hour long, the conversion may be halfway done when the tab is closed. It would be awesome if it could just resume where it left off when the user comes back, with no duplicate work! Would be amazing.
The question is if this becomes a direct feature of Mediabunny, or if I just add in the necessary tools so that one can build this themselves.