Kevin (@KevinJSharp) profile picture
Joined May 2018
Very cool!
In a recent blog post, I claimed that nobody has yet tried to “evolve” Vibrio natriegens in the laboratory to make them grow even faster. But I was totally wrong, and it warrants a correction. For context, V. natriegens is a microbe that doubles every 9.8 minutes in highly enriched growth media, or every ~30 minutes in more “minimal” media, with just some sugar and salt. In my recent blog, I explained that these cells divide faster than any other known organism because they make a large number of ribosomes quickly. (It has nothing to do with the time required to copy a genome): “V. natriegens has at least a dozen ribosomal RNA operons, or gene clusters encoding ribosomal RNA molecules, in its genome… [Also,] these ribosome genes are located next to strong promoters, or genetic sequences that recruit RNA polymerase enzymes. In other words, Vibrio devotes more of its genome to ribosomal genes, and has also evolved a stronger start signal for those genes, meaning the cell makes ribosomal RNA much more frequently, and in higher numbers, than other microbes.” At the end of the blog post, I proposed some experiments to make Vibrio natriegens grow even faster. Perhaps we could make its ribosomes smaller, such that each one takes less time to create. Or, alternatively: “…we could take a more agnostic approach and just let evolution take its course, albeit in an accelerated way…Perhaps we could run a Richard Lenski-esque experiment, in which V. natriegens’ cells are grown in a robotic bioreactor and flooded with glucose every few hours…If we repeat this lots of times, some microbes may evolve to grow even faster… Or maybe not; V. natriegens may already be quite close to the theoretical cell division time limit. These experiments haven’t been done yet.” This last claim — that nobody has yet done these experiments — was totally wrong, as Adam Feist, bioengineer at UC San Diego, explained to me by email. Feist is a co-author on two studies that have done basically this exact experiment! In one study, they used an automated laboratory evolution (ALE) robot to evolve E. coli cells to grow faster. Their goal was to see whether E. coli could grow as fast as V. natriegens. (Answer: No.) In a second study, they did much the same with V. natriegens to see if they could “break” its speed limit. (Answer: No.) For the first study, the researchers took six different E. coli strains, including BL21, K-12 MG1655, Crooks, and others. They grew each strain in three triplicate flasks, filled with growth media containing just a minimal amount of sugar and salts, and incubated them at 37°C. Every time the cells reached a certain density, they transferred a tiny amount of cells into a fresh flask with the same exact conditions. And they repeated this again and again for 900 generations of cell growth. The beauty of this experiment is two-fold: First, it’s super simple and easy to replicate. Every time cells are transferred into a fresh flask, microbes evolve to grow faster such that they can take over the nutrients in that flask. So this is artificial selection in a tube. And second, they froze the cells at regular intervals so they could sequence them, measure their gene expression, and so on to really figure out how the cells were evolving over time. All six strains started with different growth rates. MG1655 doubled every hour, whereas the Crooks strain doubled every 47 minutes. After hundreds of generations, though, all the strains converged to a similar doubling time: About 40 minutes. All six strains also took similar paths toward the faster growth rate; the same genes kept getting mutated again and again across the flasks. Genes involved in eating and breaking down glucose were mutated to become more efficient. Ribosomal genes were also massively upregulated across strains, as were genes involved in making amino acids and nucleotides. This all makes sense. To compensate for this faster growth, the cells also “broke” a handful of genes. They mutated stress response genes, for example, because they no longer needed them in their stable, consistent environment. And they shut down motility genes (it’s super “costly” for a cell to make a flagellum, and why would they need one in a shaking flask anyway?) Even after all this convergent evolution, though, the key thing to remember is that none of the E. coli strains approached the division time of V. natriegens. All of them maxed out around 40 minutes per cell division in this “minimal” growth media. In the second study, then, they wanted to see if they could use the same principles to make V. natriegens grow faster. So they set up 10 identifical flasks, each with some M9 growth media and a bit of salt and sugar, and inoculated the cells. Every time the V. natriegens cells hit a certain density, a small amount were transferred to a fresh tube. This was repeated for 1,000 generations, but there was no increase in cell division rates. The cells did acquire mutations, though. Nine of ten flasks mutated a stress response gene (much like the E. coli did) and most flasks made mutations to an ion transporter protein. But despite those changes, the cells had a median division time of 28 minutes at the start and end of the experiment. So what happens now? Well, for starters, I think we need to run more of these experiments, albeit with some different starting parameters, to see if we can make V. natriegens divide faster after all. Feist proposed, in his email, that we should insert more copies of ribosomal genes into V. natriegens and E. coli, say, and then repeat the evolution experiments with both of them. Perhaps the cells just didn’t have enough time to duplicate these genes on their own but, if we added them artificially, their speeds would go higher. This seems like quite an easy experiment to do. But without running the experiment, we won’t actually know if the abundance of ribosomal genes alone is sufficient to speed up division times, or if we also have to cut down the size of ribosomes themselves to make cells grow faster. Let’s get busy!
24
This is brilliant! Validating the Einstein test that many (including Denis Hassabis and me) have proposed for LLMs.
I trained an LLM from scratch on pre-1900 text to see if it could come up with quantum mechanics and relativity. While the model is too small to do meaningful reasoning, it has glimpses of intuition. When given observations from past landmark experiments, the model can declare that “light is made up of definite quantities of energy” and even suggest that gravity and acceleration are locally equivalent. I’m releasing the dataset + models and leave this as an open problem to the research community. I also include what this project has taught me about intelligence in a mini essay linked below. 🧵(1/n)
3
4
61
9,522
Interesting exercise…
Another Deep Research thread. Perhaps you'd agree that this is the kind of thing that it's supposed to be able to do. The document it produced for me is rife with errors, nonsense, and, for lack of a better word for this, lies. Here's a link. chatgpt.com/s/dr_6823e309a95…
28
Noticing myself adopting a certain rhythm in AI-assisted coding (i.e. code I actually and professionally care about, contrast to vibe code). 1. Stuff everything relevant into context (this can take a while in big projects. If the project is small enough just stuff everything e.g. `files-to-prompt . -e ts -e tsx -e css -e md --cxml --ignore node_modules -o prompt.xml`) 2. Describe the next single, concrete incremental change we're trying to implement. Don't ask for code, ask for a few high-level approaches, pros/cons. There's almost always a few ways to do thing and the LLM's judgement is not always great. Optionally make concrete. 3. Pick one approach, ask for first draft code. 4. Review / learning phase: (Manually...) pull up all the API docs in a side browser of functions I haven't called before or I am less familiar with, ask for explanations, clarifications, changes, wind back and try a different approach. 6. Test. 7. Git commit. Ask for suggestions on what we could implement next. Repeat. Something like this feels more along the lines of the inner loop of AI-assisted development. The emphasis is on keeping a very tight leash on this new over-eager junior intern savant with encyclopedic knowledge of software, but who also bullshits you all the time, has an over-abundance of courage and shows little to no taste for good code. And emphasis on being slow, defensive, careful, paranoid, and on always taking the inline learning opportunity, not delegating. Many of these stages are clunky and manual and aren't made explicit or super well supported yet in existing tools. We're still very early and so much can still be done on the UI/UX of AI assisted coding.
453
1,035
262
12,163
1,217,570
A thread on some of the fascinating noncoding genetic discoveries from 2024. I've written long form posts and discussed on The Genetics Podcast about these stories. I'll share links to all below with a brief summary.
4
37
2
110
16,430
Clearly our UK electoral system is grossly unfair and no longer appropriate. I say this even though I an quite happy with the outcome in this case! Let’s hope Starmer heeds @Peston ‘s advice!
I said just after the exit poll dropped at 10pm last night that we were experiencing the most significant general election result since Margaret Thatcher's victory in 1979.  I haven't changed my view, though 1979 and 2024 could not be more different in the explanations for why they are defining moments.  Thatcher's victory mattered for how she changed everything about the economic stewardship of the UK. She remade the landscape for generations.  We don't know if Keir Starmer will do that. His manifesto is long on big ambitions to restore growth and prosperity and short on practical detail. But last night is as important because it is the confirmation of some huge and important trends. Perhaps the most important is that political tribalism focused on Labour and Tories is dead, that political allegiances have fragmented, and that huge numbers of voters didn't vote at all and wish a curse on all houses. That may seem odd to say when Starmer has won by such a landslide margin. But he did so on the back of less than 34% of the vote, only marginally up on Jeremy Corbyn's 32.1% haul in 2019, when Boris Johnson required just under 44% of the vote for an 80 seat majority. Starmer is being carried into Downing Street on the shoulders of the LibDems, the Greens and - most especially - Reform, all of whom rode the crest of the national mood, which was to evict the Tory Party.  The combined vote share of Labour and Tories, less than 60%, is lower than it has ever been - though of course the 20 percentage point drop in the Conservative share is the big shift. At the time of writing, there are still a couple of seats left to declare. But here are the numbers that should focus all our minds.  The actual numbers of votes cast for the Conservatives have halved - yes halved - to around 7m. Labour's tally of around 10m may end up being less than what it picked up on that disastrous day in 2019.  And Reform have received more than 4m votes, for its four seats, about a half a million more than the LibDems, whose reward was more than 70 seats, and double the 2m received by the Greens (on a great night for them, and which yielded four seats). There is much to say about all this. First, that this is the least proportional distribution of seats in modern electoral history. If you believe that the configuration of the Commons should roughly reflect the revealed preferences of voters, this is not a fair result. Second, the turnout of circa 60% means around 80% of the British electorate did not vote for Labour. which is remarkable. Starmer did not win by a landslide of public opinion towards his party.  He won because of fury with the Tories and because - thanks to a brilliant logistical campaign organised by Morgan McSweeny - his votes were concentrated in the right places. He would be wise however to govern with humility, and perhaps consensually, rather than with an arrogance that his unassailable position in parliament will inevitably make tempting. There is so much more to discuss, not least what the SNP's collapse means for the independence cause in Scotland, and whether it is now clear that discontent with the EU was always the symptom of a much more profound fissure in politics rather than the cause. But I'll pick all that up in coming days. For now, let's take a deep breath and acknowledge that this is a result that will start an important debate about whether our electoral system remains fit for purpose.
33
Philip hits the nail on the head. "You're having your kitchen refitted... the bloke's a little fella & his trousers are too short.. " #LBC
147
1,386
242
5,628
701,192
Incredible anecdote by @danielboffey in the Guardian A Labour MP tells a story about meeting a voter who said he wasn’t voting for him because Labour was taxing condoms
286
1,405
229
6,185
676,221
So Sunak didn’t just lie about the tax rise, he was TOLD in advance of #itvdebate NOT to say it What was that about “integrity?”
178
3,554
220
9,674
466,591
This letter is very tricky for the Conservatives.
EXCLUSIVE The chief Treasury civil servant wrote to Labour two days ago saying that the £38 billion/£2,000 tax attack “should not be presented as having been produced by the civil service” He said he had reminded ministers of this
258
1,563
124
4,625
350,048
Very much agree with this and much of what @ylecun has been sating on these subjects recently.
General intelligence, artificial or natural, does not exist. Cats, dogs, humans and all animals have specialized intelligence. They have different collections of skills and an ability to acquire new ones quickly. Much of animal and human intelligence is acquired through observation of -- and interaction with -- the physical world. That's the kind of learning that we need to reproduce in machines before we can get anywhere close to human-level AI.
31
Have you ever wanted to train LLMs in pure C without 245MB of PyTorch and 107MB of cPython? No? Well now you can! With llm.c: github.com/karpathy/llm.c To start, implements GPT-2 training on CPU/fp32 in only ~1,000 lines of clean code. It compiles and runs instantly, and exactly matches the PyTorch reference implementation. I chose GPT-2 to start because it is the grand-daddy of LLMs, the first time the LLM stack was put together in a recognizably modern form, and with model weights available.
252
1,777
329
12,360
1,661,932
Good point very well made on the distribution of rare variants.
The use of the phrase “ancestry-specific variant” is increasing, particularly to describe rare Single-nucleotide variants (SNVs). But these alleles are not ancestry-specific. They have not yet been found elsewhere, but they will be. 1/n
42
Funny to see the PCA vs UMAP debate start to take on a culture war angle. In reality, the latter simply doesn't identify basic population structure correctly. Here's an example with three-way admixture: admixed individuals get mapped almost arbitrarily.
13
46
4
331
47,975
1/n: There are some academic papers that are so brilliantly and so accessibly written and so universal in scope that they transcend disciplines and stand as timeless testaments to both great thinking and great writing. Here's a short personal selection:
97
1,306
137
7,967
1,439,587
This toy example may change how you look at tSNE & UMAP. They find clusters in the most uniform data possible.
42
163
38
1,131
542,642
GitHub Copilot and LLM chatbots are causing a big drop in StackOverflow traffic. What's fascinating about it is that these systems work largely thanks to the user-generated content from StackOverflow. Is this feedback loop sustainable?
186
599
88
4,279
956,580
No economist will ever come up with a better description of why being poor is so expensive than Terry Pratchett. Such a great quote.
296
20,995
1,171
58,224
“our system actually rests upon the collective who are trying their best, not upon the exceptional few.” How true of everything, and yet, how seldom acknowledged.
a profound article on the art of living a peaceful and satisfied academic life (very much applies to industry life as well). highly recommend this!
62