pulsatingGenius retweeted
Replying to @_josh_meyer_
my guess is humans mostly dont do it in speech rn as im writing this im not thinking in text per se neither in speech. its something i would call human latent and words come after.
if we really thought in speech explaining things would be easy but people find it difficult time and again to put what they think into words. and people who are born deaf think just fine without ever hearing speech.
thats why i wrote speech isnt really the medium we think in.
pulsatingGenius retweeted
our research in building duplex models gives us to debate on a lot of unanswered questions
the main problem with current full duplex models is that almost everyone is iterating on the same moshi architecture which imo has some real bottlenecks.
in moshi every text token is locked 1:1 to an audio frame that gets spoken right away (apart from padding and special tokens). because of this rigid lockstep the native architecture cant generate a burst of text without advancing audio time. if the model generates 20 text tokens time has to move forward 20 frames so exactly 1.6 seconds of audio gets synthesized alongside. you cant pause the audio clock to let the LLM write out a block of text.
within base moshi you could in theory train it to output reasoning text or JSON tool calls while forcing the audio stream to output silence tokens. but since the clock cant be decoupled that "silent thinking" still runs at 12.5 tokens per second. a 50 token thought chain or tool call means the user sits through 4 solid seconds of awkward dead air.
the other option is synthetic data where it keeps talking while producing the JSON or the thinking. but there is no data in the wild for that so you generate a lot of it and now your data is the bottleneck. the voice will only ever be as natural as the most natural TTS you can use it will always carry the quirks of the harness you built to create it and if whoever is making that data cant think through the scenarios of dyadic conversations well, you are cooked.
so people came up with delegation system which is a pain in itself. you train the frontend model to be smart enough to hold a decent conversation and to hand off tool calling or thinking to a much larger, more capable model. this is engineering and its the only thing that has made the 2 or 3 full duplex models we have like openai gpt live possible at all.
thinking cant be done in speech. the signal to noise ratio is low and there isnt enough speech data to give the model a good enough world model in its weights. how much data is there of people doing maths out loud? even we humans dont think in speech. yes there is "thinking out loud" but speech is not the medium humans think in. thinking or reasoning needs a dense representation like text or maybe some latent but pure speech is a fuck all dream.
i saw dessa a few days ago. it seems good and sounds good but god knows how much data they trained it on and even it needs text, or some good engineering like ASR over its thinking speech, to actually interact with the real world or be even remotely useful. train on all the call centre data u want then ask it a basic question.
this also reminds me of a question from @juliarturc something like: if you converted every text available into speech and trained on that, what would happen? idk it might be smart or it might not. it depends on a lot of things like whether the speech encoder you use preserves everything needed for speech understanding. codecs are good enough these days but i believe they are still a bottleneck.
even if you solve that,context window is a problem and so is context degradation in speech compared to text LLMs current speech models have the memory of a goldfish.
there are just too many bottlenecks but at the rate we are going we might get there next year only if a lot of people come out in the open and open source the data they are sitting on and stop giving fake demos by training moshi on their privately acquired data.
pulsatingGenius retweeted
Let’s go , travel anywhere with the best rates than any other platform on @iradotso
🚨CHEAPEST FLIGHTS IN THE WORLD 🚨
from today you'll find the cheapest flight in the world on @iradotso
when i say in the world, i mean it
no terms or conditions!
if you're planning to travel in the near future, do book your flights on ira
ira is invite only, ask your friends or comment below to get access!
pulsatingGenius retweeted
🚨CHEAPEST FLIGHTS IN THE WORLD 🚨
from today you'll find the cheapest flight in the world on @iradotso
when i say in the world, i mean it
no terms or conditions!
if you're planning to travel in the near future, do book your flights on ira
ira is invite only, ask your friends or comment below to get access!
pulsatingGenius retweeted
Replying to @avipat_
Exactly don’t believe everything you read people be creating fake stories out of thier ass in the name of sharing wisdom. My 2 cents on this idk much about data and all but I for one think if u have u unique and good data in some ways it’s going to sell . It’s never enough data.
pulsatingGenius retweeted
complaining used to mean forms, helplines and hold music , now i just text @iradotso and it gets all of my stuff done, she filed my petrol pump complaint with indianoil and they've taken action 🙂
pulsatingGenius retweeted
Replying to @Sentdex @basedjensen
Lol been seeing a lot of people from anthropic nd other ai labs talking about how ai might wipe us out and then the other side immediately calling all of it fearmongering bullshit. and honestly both side seems financially motivated.
I think reality is probably somewhere in the middle some risks are genuinely scary some are massively overstated and nobody really knows the probability of the really extreme scenarios.
pulsatingGenius retweeted
let’s go. the team has been cooking.
introducing rumik oss-1 , india’s first open-source expressive multi-indic speech model, trained on less than 60k hours of data.
one thing we’ve spent a lot of time on is data efficiency.
a big learning for us: for expressive speech, more data is not always better. after a point, token quality, prosodic coverage and the composition of the dataset matter much more than simply adding more hours.
we’ll share a technical report soon where we go deeper into this - audio scaling laws, what happens as you scale the dataset, where the gains start flattening, and how much high-quality data you actually need for expressive speech , but for now the weights are live , go check out
pulsatingGenius retweeted
Replying to @nrol_ling
not that trivial imo a hot dog has a right answer but emotion doesn't. what gets labelled is what listeners perceive not what the speaker felt.
and everyone treats emotion as language agnostic. i think it probably is anger should sound like anger. but every classifier i have trained on english falls apart on hindi. if the thing itself is universal. what's the model latching onto that doesn't transfer?
and also like if the thing(emotion) itself is universal the failure has to be in what we are modelling rather than what we are modelling about. i don't have a clean answer.
pulsatingGenius retweeted
Is the speaker token really what keeps a voice consistent throughout an utterance?
The answer turned out to be far more interesting than a speaker tag.
silk mulberry 1.5 is built to be versatile
with a simple voice description and utterance, it can generate natural speech across 20+ languages, with accents and emotions
this thread is a collection of sample prompts to help you explore what this looks like in practice.
mulberry can:
- respond in 162ms, the fastest ttfb on the market
- switch between two languages mid-sentence, naturally
- speak in different languages with their actual native accents, not a translated voice
pulsatingGenius retweeted
launching silk mulberry 1.5
one of the fastest multilingual voice models in the world
it matches the best voice models in quality benchmarks (MOS)
all this at more than 95% lower cost ₹0.40/min (~$0.0046/min)
try now 👇
pulsatingGenius retweeted
Someone writing small cheques into American AI labs, doing PR tours about how they “backed the future” probably shouldn’t be the loudest voice lecturing everyone on what India must do after the Fable news.
For those of us actually building models from scratch across modalities, the bottlenecks are not a breaking headline or a geopolitical event. We live them every day, data, compute, talent, inference, distribution, and relentless execution.
You don’t wake up one morning, see a model get pulled down internationally, and suddenly discover the importance of sovereign AI.
At @rumik_ai we’ve always believed in owning our stack and building foundational capabilities ourselves. Soon, we’ll be open-sourcing India’s first expressive TTS model with deep code-switching support across Hindi, Hinglish, and multiple Devanagari languages. Not because it’s fashionable, but because we genuinely believe India can build world-class AI infrastructure and models.
What the ecosystem needs isn’t more hindsight experts chasing engagement after every headline. It needs patient builders, conviction, long-term capital, and VCs who help create enduring AI companies instead of pretending to be the smartest AI researchers on Twitter.
India doesn’t have a talent problem. It has a conviction problem.
most people assume muP = safe learning rate transfer. true, but historically only across model width and depth.
when you change your batch size or total token duration, standard heuristics get you close, but they aren't exact.
a new paper by Bruno et. al completely changes this.
pulsatingGenius retweeted
Replying to @dwarkesh_sp
Related to this, I’ve noticed that when I discuss ideas with other researchers and eventually make a point that supports their view, they sometimes seem puzzled and keep debating.
I guess we have become so used to debating and defending our point that genuine agreement kinda feels unexpected 😅


















