AI nomads. BJ/SF.
earth
Joined November 2013
- Tweets627
- Following1.9K
- Followers550
- Likes293
Cong retweeted
Early warning signs are mounting that AI is already impacting the job market in NYC.
New data just out show entry-level job postings in the city are way down in the careers most exposed to AI. Below chart shows change in NYC since 2022:
This is coming fast and we are doing almost nothing about it.
Cong retweeted
“If you're training AI to do what people have spent years and years perfecting in their careers… aren't you necessarily endangering human work?” 60 Minutes’ Jon Wertheim asks Brendan Foody, CEO of AI training startup Mercor.
“I think that you're changing human work,” Foody says. “It might not necessarily be that repetitive task that people have done in the past, but there are still so many other things that people could be able to do. And so I think that evolution is natural.” cbsn.ws/4ylJGwV
Cong retweeted
Cruise Co-Founder Kyle Vogt on Tesla’s approach vs Waymo’s:
“What I see is really Tesla as a company who kind of pioneered the end-to-end neural network approach to self-driving, which I think is the right technical bet long term.
They built (Waymo) this highly-validated, robust system that's now on public roads, and it's great. But they know that it's the wrong technical approach, and they need to move more in the direction of Tesla, of more neural networks.
It is just intractable to maintain a 3D map of every square inch of the planet and update it in real time, and then expect that every time you go somewhere, the map is still accurate on one hand, and also probably unrealistic to assume that every car built in the future is going to have these giant spinning KFC buckets on the roof.
To Waymo's credit, I think they know this, and they've started moving towards a Tesla-like approach.”
Cong retweeted
This video was a happy accident. While waiting in a car during the Netflix shoot of my ABSTRACT episode, I drew a cyclist on the side window. Then the car started moving.
On view at my current show "The Moving image" at Museum Schloss Bonndorf/Germany.
christophniemann.com/detail/…
Cong retweeted
I've spent hours and hours thinking about how AI is going to change writing. This is a 90-minute distillation of everything I've learned.
Some things I believe:
1. The combination of LLM-driven humor and image generation means that we're about to enter the golden age of memes.
2. The best writers will be fine. Robert Caro and Dostoevsky aren’t about to be disrupted by ChatGPT.
3. What are the different models like? ChatGPT is your friend who makes a lot of good points, but it’s kinda boring, Claude is your hippie friend who loves to get vulnerable but takes the whole “express yourself” thing a little too far, and Grok is your unhinged friend who leans a little too hard into tinfoil hat theories, but is always a trip to jam on ideas with.
4. People who say that AI writing is low-quality aren’t realizing that quality exists along two dimensions: (1) the absolute quality of the writing and (2) how tailored the writing is to your interests at the time.
5. I’ll tell you this: What writers are doing with AI behind closed doors is a long way ahead of what's publicly understood. I don't expect this to change anytime soon because of the social stigma associated with AI-enhanced writing. Because of that, if you want to see the cutting edge, you're gonna have to piece things together through private conversations and group chats.
6. If you want to follow what's happening in AI, remember this quote from William Gibson: “The future is here, it’s just not evenly distributed yet.” You can get a glimpse of the future by looking at how a small percentage of writers are already using AI.
7. I’m bearish on writers who are currently using AI to write for them, and bullish on writers who are currently using AI to write with them.
8. What kinds of writing will continue to be written by humans? Ones that speak to our humanity. People are interested in people. Their stories, their struggles, their emotions, their drama.
9. Almost all utilitarian writing, where the goal is to convey information, not do it beautifully, will be written by AI.
10. In some ways, AI is the end of slop. So many Google search results are slop. LinkedIn posts are slop. The way Twitter got taken over by Threadbois in 2021 was also slop. AI-generated writing is already better than all of those things, so why would you read them now?
11. AI will be tougher on writers than readers. Readers will be exposed to some slop, but the Internet will be good about filtering it out. Writers, though, are now competing against ever-improving LLMs, which are getting better and better by the month.
12. Humans will contribute with unique data or perspectives. The famous Peter Thiel interview question doubles as a good writing prompt: “What very important truth do few people agree with you on?”
13. New technologies breed new kinds of art. Ever notice how flat 13th or 14th century Medieval art looks? And how different that art looks from the Renaissance art created in the 15th and 16th centuries? Technical innovations like the camera obscura and perspective grids are behind this. Similarly profound changes will come to the writing world because of AI (credit to @jmrphy for the idea here).
14. Satya Nadella says: “The new workflow for me is I think with AI and work with my colleagues.” When it comes to discovering ideas, I've also found that jamming with an LLM is more productive than doing it with most people I know (save for a few giga-brain conversationalists).
15. Thought experiment: Will AI-writing be more like music or chess? With music, we don't care how a song is made. We just want it to be good. With chess, there's a huge market for watching human beings play even though the computers are already better. I think non-fiction writing will go the way of music. People won’t care how it was made. They’ll just care that it’s good.
16. AI has flipped the rules of tech adoption. Seasoned managers usually drag their feet with adopting new technology, but the ones I know love AI, while frontline workers struggle to see the point. My theory is that AI matches how managers already operate. Management has always been a kind of prompt engineering: set a vision, delegate, give feedback, iterate. But LLMs remove the drama that used to come with having a team. No 1-on-1s. No emotional tangles. It's like management without the headache. For frontline employees, things are different. They aren't as accustomed to setting a vision and giving feedback, so LLM prompting is a daunting and unfamiliar kind of work for them.
17. AI editors are already quite good. Sure, they aren’t as good as the world’s best editors, but they’re a fraction of the cost, they’ll instantly give you 80th percentile feedback, and they work 24/7. As a novelist recently said to me: “Paying an editor to review my novel costs me $7,000 and a 4-6 week turnaround time, whereas Claude costs me $1.25 and gets me results a few minutes later.” The edits definitely aren’t as good, but there’s a virtue to speed (and this guy isn’t a chump writer).
18. The way AI-skeptics hate on LLMs while using old models is like driving a ‘92 Honda while hating on a self-driving Tesla.
19. AI-generated fiction makes people very upset. A friend insists it’s like having sex with a robot. Doesn’t matter how good it is. It ain’t human-generated, and there’s something uniquely repulsive about that.
I’ve shared the full conversation below. It’s a solo-episode of me riffing on what I’ve learned about AI for ~90 minutes. If you’d rather watch it on YouTube or listen on Apple or Spotify, I’ve shared the links in the reply tweets.
And if you have any questions, I’ll be extra active in the replies for this episode.
Cong retweeted
Just sat down to read the DeepSeek-R1 paper. We're entering an era where compute isn't primarily for training. It's for creating better data. I expect to see the money & compute spent on data processing (generation / annotation / curation) grow to match and exceed the money & compute spent on pre-training.
People have talked about pre-training plateauing because we're "running out of data" on the internet to scrape. While that may be the case, capability improvements are going to continue full steam ahead. The improvements in intelligence are going to come not from putting in more data (scraped from the internet) but rather from putting in more compute (to generate higher-quality data). Intuitively, this feels similar to me to how people learn. You don't learn just by ingesting lots of tokens. In many cases, you learn by thinking more (I am referring to training time, but thinking more also applies at inference time).
There are many creative ways to put in more compute to get better data, and this problem will be an important research area for a number of years.
- In this paper, they train two models. Why two models? The first one (a reasoning model trained via RL) is used to generate data to train the second. This works by using the first model to generate reasoning traces and then selectively keeping only the high quality outputs (quality is judged by simply checking the results). This approach of "checking the results" works well for domains like math and coding where you can easily check the results.
- In drug development, it is super common to put compute into generating better data in two phases. In the first phase, a generative model (e.g., for protein sequences) generates a massive number of candidate drugs. In the second phase, scoring or filtering is done with a slew of predictive models which may predict structure, toxicity, solubility, binding affinity, etc. After all this work is done, you may end up with 100 data points.
- In physical domains (e.g., climate applications), expensive but accurate physics simulators exist (these simulations are run on super computers for long periods of time to simulate the physics of the atmosphere or some other system). All of that data can be used to train models, which is showing a ton of promise.
The question of "how to put more compute into generating better data" is central to progress in AI right now.
Cong retweeted
Thoughts about o3: I'll skip the obvious part (extraordinary reasoning, FrontierMath is insanely hard, etc). I think the essence of o3 is about *relaxing a single-point RL super intelligence* to cover more points in the space of useful problems.
The world of AI is no stranger to RL achieving god-level stunts.
AlphaGo was a super intelligence. It beats the world champion in Go - well above 99.999% of regular players.
AlphaStar was a super intelligence. It bests some of the greatest e-sport champion teams on StarCraft.
Boston Dynamics e-Atlas was a super intelligence. It performs perfect backflips. Most human brains don't know how to send such sophisticated control signals to their limbs.
Similar statement can be made for AIME, SWE-Bench, and FrontierMath - they are like Go, which requires exceptional domain expertise above 99.99....% of average people. o3 is a super intelligence when operating in these domains.
The key difference is that AlphaGo uses RL to optimize for a simple, almost trivially defined reward function: winning the game gives 1, losing gives 0. Learning reward functions for sophisticated math and software engineering are much harder. o3 made a breakthrough in solving the reward problem, for the domains that OpenAI prioritizes. It is no longer an RL specialist for single-point task, but an RL specialist for a bigger set of useful tasks.
Yet o3's reward engineering could not cover ALL distribution of human cognition. This is why we are still cursed by Moravec's paradox. o3 can wow the Fields Medalists, but still fail to solve some 5-yr-old puzzles like the one below. I am not at all surprised by this cognitive dissonance, just like we wouldn't expect AlphaGo to win Poker games.
Huge milestone. Clear roadmap. More to do.
Cong retweeted
Each year I ask myself these 40 questions.
This will be my eleventh year. It's still one of my favorite rituals. It helps me see how phases of my life wax and wane.
Each year since 2016, new translations come in. We're up to 30 languages! Thank you to all the contributors 🥰
Cong retweeted
昨天和新闻学生聊天，突然意识到为啥短视频是推荐算法影响的重灾区，但podcast反而成为新的深度媒体，这大概和视觉和听力的带宽有关：视觉容易被推荐、不一样的东西吸引，听觉不行、你乱推荐立刻心烦，因为带宽太小。所以podcast因为不太受算法影响，还是订阅制度、才可能逐步成为新深度媒介。
Cong retweeted
Vamos, @RafaelNadal!
As you get ready to graduate from tennis, I’ve got a few things to share before I maybe get emotional.
Let’s start with the obvious: you beat me—a lot. More than I managed to beat you. You challenged me in ways no one else could. On clay, it felt like I was stepping into your backyard, and you made me work harder than I ever thought I could just to hold my ground. You made me reimagine my game—even going so far as to change the size of my racquet head, hoping for any edge.
I’m not a very superstitious person, but you took it to the next level. Your whole process. All those rituals. Assembling your water bottles like toy soldiers in formation, fixing your hair, adjusting your underwear... All of it with the highest intensity. Secretly, I kind of loved the whole thing. Because it was so unique—it was so you.
And you know what, Rafa, you made me enjoy the game even more.
OK, maybe not at first. After the 2004 Australian Open, I achieved the #1 ranking for the first time. I thought I was on top of the world. And I was—until two months later, when you walked on the court in Miami in your red sleeveless shirt, showing off those biceps, and you beat me convincingly. All that buzz I’d been hearing about you—about this amazing young player from Mallorca, a generational talent, probably going to win a major someday—it wasn’t just hype.
We were both at the start of our journey and it’s one we ended up taking together. Twenty years later, Rafa, I have to say: What an incredible run you’ve had. Including 14 French Opens—historic! You made Spain proud... you made the whole tennis world proud.
I keep thinking about the memories we’ve shared. Promoting the sport together. Playing that match on half-grass, half-clay. Breaking the all-time attendance record by playing in front of more than 50,000 fans in Cape Town, South Africa. Always cracking each other up. Wearing each other out on the court and then, sometimes, almost literally having to hold each other up during trophy ceremonies.
I’m still grateful you invited me to Mallorca to help launch the Rafa Nadal Academy in 2016. Actually, I kind of invited myself. I knew you were too polite to insist on me being there, but I didn’t want to miss it. You have always been a role model for kids around the world, and Mirka and I are so glad that our children have all trained at your academies. They had a blast and learned so much—like thousands of other young players. Although I always worried my kids would come home playing tennis as lefties.
And then there was London—the Laver Cup in 2022. My final match. It meant everything to me that you were there by my side—not as my rival but as my doubles partner. Sharing the court with you that night, and sharing those tears, will forever be one of the most special moments of my career.
Rafa, I know you’re focused on the last stretch of your epic career. We will talk when it’s done. For now, I just want to congratulate your family and team, who all played a massive role in your success. And I want you to know that your old friend is always cheering for you, and will be cheering just as loud for everything you do next.
Rafa that!
Best always, your fan,
Roger
