
Co-Founder & CTO @Predibase OSS: LoRAX (https://xitter.cf/t.co/iUD8EzSwJR) | https://xitter.cf/t.co/FIOKcmavWX | @ludwig_ai
Seattle, WA
Joined August 2014
- Tweets478
- Following260
- Followers676
- Likes20.9K
Travis Addair retweeted
Replying to @musedivision @geoffreyhinton
I said "auto-regressive LLMs, in and of themselves, will not lead human-level AI"
That statement is still totally true.
First, the reasoning abilities of current AI systems are based non-auto-regressive search (which is what I have always advocated for). But AFAICT, they do it in token space, which is limited and inefficient. I have claimed that human-like reasoning must be a search in continuous representation space. It looks like the industry is moving towards that.
Second, the self-improvement methods, as currently practiced, only work for domains where the quality of outputs can be scored without human intervention, such as mathematics, code, and scenarios that can be simulated accurately. Not anything else. Humans and animals learn new skills way more efficiently than current RL methods.
Third, the multimodal capabilities of current AI assistants generally use separately-trained encoders (that are not LLMs). This is also what I've been advocating. Except that I think the best way to do this is with JEPA trained with self-supervised learning. The research community is clearly moving towards that (3000 papers on JEPA in just 4 years).
Fourth, if LLMs were a path to human-level AI, we would have domestic robots and Level-4 or Level-5 self-driving cars for consumers by now. And we don't. We certainly don't have cars that can learn to drive in 20 hours or practice like any teenager. We're still missing something pretty huge to claim human-level intelligence (let alone superhuman).
Sure, we now have computer systems that are impressive, very useful, and whose performance is superhuman in an increasing number of domains (coding being one of them).
But that's true of the entire history of progress in computer technology.
Lastly, there is a basic confusion about what intelligence actually is.
It is not the mere accumulation and regurgitation of existing declarative knowledge (which is essentially what LLMs do).
As Jean Piaget famously said, "intelligence is not what you know, it is what you do when you don't know."
It is your ability to solve new problem without any prior training, to act in previously-unknown scenarios, and to adapt very quickly to new situations with minimal training.
We're still far from that.
Travis Addair retweeted
A big problem with California is it tries to solve the worlds problems instead of focusing on the major problems at home
Travis Addair retweeted
The "whiteboard defense:" I should be able to pull you aside at any moment and ask you to explain any customer-facing system you've shipped. You should be able to clearly explain how it works and defend the decisions you made. This is my benchmark for responsible AI usage.
I don't expect line-level familiarity with the code. I don't care if you remember the exact function name or implementation detail. You may not even know it. I don't care.
But if I ask "why did you do X instead of Y?", "what happens if this actor behaves maliciously?", "what data structure did you use here and why?", or "where does this fail?" you should be able to answer confidently.
For PoCs, demos, experiments, whatever: I don't care. Generate 100% of it and understand none of it. Speed over quality every time in those specific scenarios.
But if you're shipping customer-facing work, you can't be shipping things you don't understand at a high level.
Travis Addair retweeted
Dario has written that we need to “pace the frontier,” and Sam has agreed. People may be surprised by my response: go ahead.
You guys are the frontier. By any reasonable metric — market share, revenue growth, model capability — the two of you have a duopoly on frontier intelligence. You’ve also claimed the lead is widening because of recursive self-improvement.
I don’t see what you see in the lab. If the unreleased models are scary enough that you think you should slow down, I support your decision to be responsible.
But stop pretending you need anyone else’s permission. Stop pretending antitrust law has to be suspended so you can form a cartel. Stop pretending you need a regulatory approval process that supersedes product liability. Stop pretending METR is independent when it is intertwined with Anthropic’s investors and staff. Stop pretending you need those same evaluators to police competitors who aren’t even at the frontier.
Most of all, stop pretending the motivation to slow down is purely altruistic. You face massive product-liability exposure if your products enable a truly damaging cyberattack. The market already punishes models that behave in unpredictable or unauthorized ways. After the Hugging Face episode, it is simply good business for OpenAI and Anthropic to trade some raw power for reliability and predictability. Call it alignment if you want. It is also just giving customers what they want.
Pacing the frontier would also create breathing room for a more intelligent conversation about regulation than Bernie Sanders’ “shut it all down.” China is very unlikely to join a global agreement, as you know, and that has to be taken into account as well.
So go ahead and pace the frontier. You are the ones setting it. The easiest way not to build superintelligence is for you to agree not to build it. Demanding your preferred regulatory framework as the price of that will look like blackmail of the public and the political system. So just do it.
If you do, you’ll buy goodwill for the next conversation. If you don’t, we’ll know this was just another bid for regulatory capture — or an election-season psyop.
Travis Addair retweeted
This one got to me
Perhaps because he’s not an anon, but the favorite to win statewide office in Texas.
Perhaps it’s because the kids in the picture could have been me 25 years ago.
I just donated to his opponent and hope you’ll join me
secure.actblue.com/donate/ro…
Travis Addair retweeted
Replying to @PessimistsArc
Right. Dario was already claiming that GPT2 was too dangerous to open source back in 2019.
I made fun of them then.
Everyone should make fun of them now.
Travis Addair retweeted
Ro, I like you. You know that. But you need to read the state of the state.
The number of Deca Unicorns in Cali is growing by the day as investors chase amazing startups.
A unique feature of these 10b startups is that even if they raise a billion, little, if any of that money goes to the founders, who are now worth billions of dollars over night. They are the definition of cash poor, stock rich
How are you going to tax them ? Make them borrow money against their shares, if they can? They just raised money to grow their company and a bank will come along and loan them money ? A company that has been in business maybe less than a year ? lol
Will you take their stock if they can’t ?
Do you really think each of multiple founders, who started an amazing company in Cali and is now a billionaire, can each just pull out $250m per billion of net worth from their raise ? You know they can’t.
If this passes , and it doesn’t directly impact me at all, I won’t be a cali resident, but you can bet if I’m investing in a multi billion dollar startup, I’m asking them to move from California first.
IMO, if this passes, only idiot startup founders stay in Cali.
I’ve done it before and will do it again. Dallas. Pittsburgh. Indiana. I will make NOT being in California a pre requisite for an investment
Ideology is not a strategy Ro.
The California Democratic Party and the California labor movement just stood with @BernieSanders and me in supporting 5% wealth tax on 250 California billionaires.
California voters want a Democratic Party that will stand up for the working class over the billionaire class.
Travis Addair retweeted
I'm a fan of govt experimentation, but I don't think this is a very good idea.
The problem isn't that grocery stores are greedy. The problem is that food got expensive. The Producer Price Index for all foods—avg price paid to domestic producers for food—jumped 30% between 2021 and 2023. Since then, grocery prices have continued to climb, in part due to increases in wages and housing costs.
Without strong evidence that NYC bodegas and groceries stores are colluding to protect double-digit net profit margins, the reasonable assumption is that these businesses are operating with <3% margins after rent, labor, and utilities.
So, a plan to sell some food with a 30% discount seems like a plan to create very long lines to buy a very small number of subsidized groceries. We'll see how that goes. But I don't think literal bread lines are the image that a socialist mayor wants to cultivate.
As Keynes said in 1926, "the important thing for government is not to do things which individuals are doing already ... but to do those things which at present are not done at all." Bodegas are something we're doing already. Large grocers like Trader Joes are really really good at managing supply chains and keeping prices down, whatever you may think of their other policies. I do not trust a busy municipal govt trying to do 1,001 things to be better at groceries than grocers. But I guess we'll find out.
Mamdani: Today, I am proud to announce a collection of essential staples that will be predictably 30% cheaper at all five of our city-run grocery stores. This core set of goods will include all fresh produce, meat, and seafood, along with 20 other essential items like cheese, milk, and bread.
Here's how it will work. Once a month, our five city-run grocery stores will set prices for this core set of goods at 30% below typical retail prices. No exceptions, no gimmicks. The savings will last for the entire month. That means no weekly fluctuations or sticker shock at the checkout line—not for our seniors living on fixed incomes, nor for the parents who rely on a regular supply of apple slices to keep toddler tantrums at bay.
Travis Addair retweeted
Stop the Washington Income Tax.
Vote YES on i-645. Voting begins on October 16th this year and ends November 3rd. Mark your calendar NOW.
Retweet if you want to stop the WA income tax.
Travis Addair retweeted
The socialist regards society much as a child regards supper: he notices with great precision who received the largest slice, while remaining curiously uninterested in who rose at dawn to bake the thing.
Socialists always imagine themselves at the table, never in the kitchen.
This Post is from an account that no longer exists. Learn more
Travis Addair retweeted
Americans want to own homes, build wealth, and have economic opportunity. So some are voting for socialists like Mamdani because he has recognized these anxieties and identified a villain (but not a real solution).
What is most likely to help?
Socialism consistently makes everyone equally poor. The experiment has been run already! You don’t build prosperity by attacking the entrepreneurs, businesspeople, or investors who create it. We do not want to be Maoist China, the Soviet Union, Venezuela, Cuba, East Germany, North Korea. This is a very bad club of outcomes.
The next decade will create more business owners than the last century. AI gives anyone with entrepreneurial spirit cheap access to extraordinary human capital. Reindustrialization will bring back productive work. Broad asset ownership can put millions more Americans on the path to wealth. And every dollar will go further as economic productivity surges.
Ordinary Americans will suffer if irresponsible politicians, educators and malicious international influence campaigns create 100M Marxists. It will prosper with 100M more Bogleheads.
We can be a nation of wealthy owners.
If you ever wondered about how how open/closed model makers and inference providers make economic sense, this is the piece to read
Travis Addair retweeted
There is one industry - the tech industry - where employees actually participate in companies succeeding and growing through equity.
And you are trying to punish and destroy that industry, rather than using it as a model.
Travis Addair retweeted
the destruction of American education over the past decade is an incredible self-own
competence is objective. a child can either do the math or they can’t. but in the u.s., a lot of people have reasons not to say that!
parents don’t want to hear their kid is struggling. teachers don’t want scores used to manage them. districts don’t want embarrassment. progressives worry accountability will create inequity. conservatives don’t want federal authorities. so we end up with process, weak standards, and excuses to explain away bad outcomes.
people object that it’s phones, covid, demographic change.
ok! but we fail globally when others have phones, covid too — vietnam is much poorer than the u.s., yet performs well in international math comparisons. some countries treat math as a basic skill everyone needs to master. here, it is part of a fight about fairness, autonomy, and feelings
in the age of AI — if people can’t do basic math, read closely, or think through problems, ai won’t make them more capable. it will become something they rely on without understanding. the countries that come out ahead in the global race won’t just have better technology. they’ll have people who know how to use it, question it, build on it. we need the national ability to decide something is worth doing coherently (teaching math!)
the US has the money to teach math well but it has not shown the will. we are failing the next generation
Travis Addair retweeted
Washington state's electricity rates are rising fast.
Some blame data centers, but electricity use in WA declined over the last decade.
AG Brown (@AGOWA) blames profit. But public utilities like @SEACityLight are also increasing rates.
The real cause is... 1/2 #waleg
Travis Addair retweeted
Elon could literally solve homelessness tomorrow.
All he needs to do is sell his stake in the companies he created from the ground up, crashing the price and evaporating his wealth as well as the savings and pensions of workers globally.. handing the control of these companies over to short term profit interests and preventing any long term vision from being enacted..
THEN take whatever money that remains which will be nowhere near 1tn and give it to homeless charities who have never solved homelessness anywhere on earth because they’re financially incentivised to do the opposite.
Bam - homelessness solved.
The reason he doesn’t do any of this is because he’s evil.
Been wondering for a while why GSuite doesn’t offer something like this for the enterprise.
Travis Addair retweeted
Fable is a good model. As with all new models, it is simultaneously excellent and entirely unremarkable (relative to other models). It is slow and expensive, and the "loops are all you need" discourse they are pushing is obvious in the context of someone using Fable-class models
What I've found so far is that for broad scope design (code architecture) tasks, Fable is unremarkable. Or, not better enough to justify its cost and speed.
But in highly targeted goal-oriented loops, it is another beast entirely. It is very slow but produces very good results.
I let it churn on optimizing a SwiftUI-layout resolver in Go I wrote and it was able to bring it down to an order of magnitude I could not reach myself (micro => nanosecond scale). But it took 2 hours and $40 to do it and I had to claw back some changes it overfit to Apple Silicon. Still, very worth it.
In comparison, for "implement this feature/change" iterative work, I ran head-to-head Fable vs GPT5.5 vs. GLM-5.1. They all produced equally acceptable final results, but GPT5/GLM did it in a couple minutes and Fable was churning away for 40 minutes. And GLM cost me less than a dollar, GPT5.5 ~$1.50, and Fable cost $9.
You can see that in this context, interactively working with an agent is nonsense. Its too slow. You need to write loops to keep the agent working and you probably want to highly parallelize the work being done. As with all things, I think a balance makes sense...
My sense is that I'd reserve Fable for targeted, surgical analysis and work. Not for daily driving everyday tasks.
I'm going to keep spending a shitload of money (relatively) and maining Fable for the rest of the week to continue to judge, will report if anything changes. I'll continue to head-to-head as well.
Travis Addair retweeted
In medieval times, within the arms race of ever more demonic torture devices, some sadistic genius came up with the idea of the Little Ease.
This was a prison cell built so small in every dimension that a grown man could not stand upright in it nor lie down at full length nor properly sit.
The pain is relentless and without relief and inflicted by one's own body. Prisoners were known to go insane within a few days. A stay at the Little Ease was considered even more cruel than the rack, the thumbscrew, and the other ghoulish machinery of the Tower of London.
A breeding pig will spend her whole life in a version of that box.
These are social, roaming creatures (more intelligent than dogs) who will never leave this corset of steel.
They have been selectively bred to be bigger than their frames can support. Yet we put them in cells so confined that they cannot comfortably sit, and their attempts to do so (for example, by sneaking their limbs into adjacent stalls) reliably lead to fractures and sprains.
They cannot sweat, yet have nothing to roll around in to cool themselves off. Except their own manure, which (contrary to the common misconception) they are so averse to (thanks to their strong sense of smell) that new sows will often suffer from constipation to avoid soiling the space from which they eat and sleep.
Here is how the writer Matthew Scully described what saw at one of Smithfield’s “gestation barn”:
> “Sores, tumors, ulcers, pus pockets, lesions, cysts, bruises, torn ears, swollen legs everywhere. Roaring, groaning, tail biting, fighting, and other “Vices,” as they’re called in the industry. Frenzied chewing on bars and chains, stereotypical “vacuum” chewing on nothing at all, stereotypical rooting and nest building with imaginary straw. And “social defeat,” lots of it, in every third or fourth stall some completely broken being you know is alive only because she blinks and stares up at you … creatures beyond the power of pity to help or indifference to make more miserable, dead to the world except as heaps of flesh into which the [insemination] rod may be stuck once more and more flesh reproduced.”
—
The Save Our Bacon Act is trying to unroll the few state protections we have against this barbaric cruelty - for example California’s Prop 12 - which banned the sale of pork from pigs kept in gestation crates.
It’s incredibly important we don’t end up with this sort of federal preemption.
SOB will not only kill the most important animal welfare related laws in the US of the past decade, but more importantly, it will also restrict ALL future legislative progress (aka how the animal welfare movement has gotten its biggest wins).
The Senate is currently deciding whether to add the SOB Act to the Farm Bill.
With relatively little money now, we can discourage the most pivotal senators in the Ag committee from backing this amendment.
Defeating this bill is even more important given the amount of philanthropic funding I expect to come online in the next year or two.
It will plausibly be over 10x more expensive to repeal SOB than to prevent it from passing in the first place.
All that money that could be spent transforming our society's relationship to mass animal suffering will instead have to be spent just getting us back to where we are right now.
That's why money spent now fighting this bill (and I mean right NOW) is so effective.
If you’re in a position to donate six figures, please DM me.
Travis Addair retweeted
This is a good example of large scale security issues companies can face with AI adoption. Enterprises have years of baked in assumptions that work because people following processes do not necessarily understand the capabilities those assumptions imply. Now, potentially any user




























