nullHawk (@null_hawk) header
nullHawk (@null_hawk) profile picture

mix of engineering and research | building voice @rumik_ai

Joined August 2023
Photo by nullHawk (@null_hawk)Photo by nullHawk (@null_hawk)Photo by nullHawk (@null_hawk)Photo by nullHawk (@null_hawk)Photo by nullHawk (@null_hawk)Photo by nullHawk (@null_hawk)Photo by nullHawk (@null_hawk)Photo by nullHawk (@null_hawk)Photo by nullHawk (@null_hawk)Photo by nullHawk (@null_hawk)
the main problem with current full duplex models is that almost everyone is iterating on the same moshi architecture which imo has some real bottlenecks. in moshi every text token is locked 1:1 to an audio frame that gets spoken right away (apart from padding and special tokens). because of this rigid lockstep the native architecture cant generate a burst of text without advancing audio time. if the model generates 20 text tokens time has to move forward 20 frames so exactly 1.6 seconds of audio gets synthesized alongside. you cant pause the audio clock to let the LLM write out a block of text. within base moshi you could in theory train it to output reasoning text or JSON tool calls while forcing the audio stream to output silence tokens. but since the clock cant be decoupled that "silent thinking" still runs at 12.5 tokens per second. a 50 token thought chain or tool call means the user sits through 4 solid seconds of awkward dead air. the other option is synthetic data where it keeps talking while producing the JSON or the thinking. but there is no data in the wild for that so you generate a lot of it and now your data is the bottleneck. the voice will only ever be as natural as the most natural TTS you can use it will always carry the quirks of the harness you built to create it and if whoever is making that data cant think through the scenarios of dyadic conversations well, you are cooked. so people came up with delegation system which is a pain in itself. you train the frontend model to be smart enough to hold a decent conversation and to hand off tool calling or thinking to a much larger, more capable model. this is engineering and its the only thing that has made the 2 or 3 full duplex models we have like openai gpt live possible at all. thinking cant be done in speech. the signal to noise ratio is low and there isnt enough speech data to give the model a good enough world model in its weights. how much data is there of people doing maths out loud? even we humans dont think in speech. yes there is "thinking out loud" but speech is not the medium humans think in. thinking or reasoning needs a dense representation like text or maybe some latent but pure speech is a fuck all dream. i saw dessa a few days ago. it seems good and sounds good but god knows how much data they trained it on and even it needs text, or some good engineering like ASR over its thinking speech, to actually interact with the real world or be even remotely useful. train on all the call centre data u want then ask it a basic question. this also reminds me of a question from @juliarturc something like: if you converted every text available into speech and trained on that, what would happen? idk it might be smart or it might not. it depends on a lot of things like whether the speech encoder you use preserves everything needed for speech understanding. codecs are good enough these days but i believe they are still a bottleneck. even if you solve that,context window is a problem and so is context degradation in speech compared to text LLMs current speech models have the memory of a goldfish. there are just too many bottlenecks but at the rate we are going we might get there next year only if a lot of people come out in the open and open source the data they are sitting on and stop giving fake demos by training moshi on their privately acquired data.
Full duplex models are having a moment now, are people using it in production? Why not? What is your top reason for not using full duplex models?
21%Cost
5%Latency
40%Control
34%Reasoning / tool call
209 votes • Final results
20
33
8
202
22,668
People have yet to realise just how powerful this is and the possibilities it unlocks for solving real consumer problems. We’ve invested deeply in voice technology, building our own speech models in-house, so this use case is a natural fit for us. @iradotso ftw
Replying to @iradotso
@iradotso I asked IRA to find me a paramedics for injection near my place. It searched and called three places and then sent me transcript of the call made and made the booking at Sanocare which were the cheapest and quickest. Crazy!! @lets_dig_deeper
2
5
1
10
991
nullHawk retweeted
(1/n) i'm putting out sneak peeks of a new neural audio codec that i've been working on recently. it matches @kyutai_labs mimi on ~1200x less data. 5,770 hours of english audio. no chinese, hindi, or japanese in training. yet it matches mimi's STOI on all four languages.
11
19
1
163
8,090
nullHawk retweeted
I built a port of Microsoft’s X++ windows only compiler for Apple Silicon While building at Lab0.ai. I couldn’t find an open-source X++ compiler port for macOS, so I built one of my own🔥 Still expanding support. Contributions welcome: github.com/lab0-ai/xppc-maco…
3
5
19
639
if you value intelligence above all other human qualities, you’re gonna have a bad time
898
3,522
1,180
24,370
9,971,105
You just cloned a 50,000-line repo. Where do you even start? I built kenly for exactly that moment — it walks you through any codebase like a senior dev giving you KT. Opens the files, highlights the exact lines, explains the flow, answers your questions. Live in your terminal
2
2
3
110
nullHawk retweeted
jensen huang shared an open letter on why open models matter. we made it brief, then gave it to the voice model we’re opening next. throughout the brief, it switches between 8 languages without losing its voice. india has never lacked voices. it’s time the models carrying them are built here, and built in the open
6
8
56
4,814
hahaha, and this will be the anthem of this event 😂
wait is this a real basecamp bangalore event???
4
2
23
3,901
nullHawk retweeted
we’re open sourcing a voice model built for how people actually speak. 20+ languages. native accents. mid-sentence language switching. Expressive yours soon.
36
44
9
610
29,574
nullHawk retweeted
less latency. more presence.
2
25
1,003
nullHawk retweeted
silk mulberry just unlocked 8 new voices. who are you picking?
1
2
1
19
1,122
nullHawk retweeted
meet ira. the ai that remembers what mattered, asks about your day, and gives the small things somewhere to go.
3
4
31
1,886
Today, we are releasing verifiers v1 — an overhaul of our environment stack for the modern era of agentic RL and evals. We decompose environments into a taskset, a harness, and a runtime. Run complex agentic tasks like coding and computer use at scale, in any harness.
34
87
39
838
275,650
nullHawk retweeted
meet silk mulberry 1.5, a voice model built to outperform expectations. - 162ms end-to-end latency - replies before you finish reading the message. - matches elevenlabs v3 on the MOS benchmark while being 95% cheaper. (₹0.40/min vs ₹9.40/min) ⁃ you don't pick a voice, you design one - describe the age, accent, pitch, emotion, whatever you want, only limit is your imagination. - mid-sentence language switching across 20+ languages with native accents, plus expressive range (whispers, singing and much more)
15
40
3
435
39,679
nullHawk retweeted
We’re sharing the next major milestone in our non-invasive brain-to-text decoder research: Brain2Qwerty v2. Building on v1, which was published today in @Nature, Brain2Qwerty v2 is the highest-performing end-to-end pipeline capable of real-time sentence decoding from raw brain signals. It advances beyond character-level performance to decoding words and semantics, enabling accuracy for overall communication. We believe this research has the potential to make a real difference for the millions of people who suffer from brain lesions or disorders that prevent them from communicating. 🧵👇
657
2,041
1,376
13,982
6,022,764
nullHawk retweeted
An Indian AI lab just dropped the world's first real-time voice model that allows you to handcraft the voice of the speaker. You choose the speaker's age, gender, emotional register, timbre, pitch, accent, and code-switching. The model is also VERY fast and affordable to run.
launching silk mulberry 1.5 one of the fastest multilingual voice models in the world it matches the best voice models in quality benchmarks (MOS) all this at more than 95% lower cost ₹0.40/min (~$0.0046/min) try now 👇
11
128
2
896
55,314
nullHawk retweeted
Replying to @lets_dig_deeper
@lets_dig_deeper Just experienced Silk Mulberry 1.5 it sounds buttery smooth and genuinely natural! I've tried dozens of voice agents, but most feel robotic and artificial. This one actually feels alive. DM'd you my detailed feedback. Would love your thoughts! 🔥
1
4
525