The @kevinroose book The AGI Chronicles is really good. The Google chapter should be required reading for everyone working in a corporate environment on what not to do. Many of the details were known, but seeing them all together is so painful. By the mid-2010s, AI is the official focus of the company. They have virtually all the best researchers, the most compute, all the data, the cloud and consumer business to plug it into, and founders who are obsessed with it. They have not one, but two, internal, at-scale teams (Brain and Deepmind) pursuing AI, one in Europe, one in the US. The two teams hate each other and don’t collaborate. Google introduces the phrase “Large Language Model”, and invents the transformer architecture, and techniques like mixture of experts and distillation for neural networks. By 2020 Google has an internal LLM called Meena that was trained on an internet sized text corpus and rolled out as a prototype to the company. It’s immensely popular. The researchers behind it write a memo “Meena Eats Google” laying out the competitive threat from LLMs and calling it a trillion dollar opportunity if they launch it publicly. The project is immediately enmeshed in interteam politics with search feeling that a chatbot shouldn’t be allowed as it will confuse people who also enter questions into search. The phrase “Meena Eats Google” is considered very hostile by Google’s many teams working on AI and it is renamed to “Meena Eats The World”. The researchers are all caught up in a pre-existing world view of classical ML, and each researcher is pursuing a pet paradigm. They all criticize LLMs for not using their techniques. Teams working on making AI more politically correct call LLMs “stochastic parrots” and don’t want models trained on real world data as that contains disparities and could perpetuate racism and sexism. The Meena effort is criticized as sexist for giving the AI a female name, and a memo is written demanding that no one refer to Meena as she. Ultimately the talented people get frustrated and leave for Google, and, as it comes into existence, Anthropic. Google isn’t out of the race, but somehow managed to fumble the biggest lead imaginable and now has to play catch up to formerly broke no-name startups with research teams who are all ex-Google. Oofda.