trey (@Comparativist) header
trey (@Comparativist) profile picture

#FloridaMan, lifelong educator, lapsed scholar (HKU PhD). 20 years in the Pearl River Delta. writing about systems, AI, infrastructure, history, and theory

Canton
Joined December 2008
Photo by trey (@Comparativist)Photo by trey (@Comparativist)Photo by trey (@Comparativist)Photo by trey (@Comparativist)Photo by trey (@Comparativist)Photo by trey (@Comparativist)Photo by trey (@Comparativist)Photo by trey (@Comparativist)Photo by trey (@Comparativist)Photo by trey (@Comparativist)
Amazingly beautiful.
Chomsky was wrong. Researchers from UC Berkeley and Google DeepMind cracked one of the biggest mysteries in AI. For years, scientists have been finding impossible geometric shapes hidden inside the brains of LLMs. Calendar months arrange themselves into literal circles. Historical years bend into smooth, rippled timelines. Cities organize themselves into actual 2D maps of latitude and longitude. Everyone assumed this proved massive transformers were spontaneously inventing complex "world models" out of thin air. They were wrong. This paper proves you don't need a massive LLM for any of this to happen. Even basic, decade-old word embeddings like word2vec produce the exact same circles, timelines, and maps. The geometry isn't coming from the AI's architecture. It’s coming from a hidden mathematical property inside human language itself. Researchers call it translation symmetry. In natural text, how often any two months appear near each other depends almost entirely on the time gap between them. January and February share the same statistical relationship as July and August. When a neural network trains on data with that kind of symmetry, the math forces it to learn Fourier representations, literal sine and cosine waves. The circles? Those are the long waves. The strange "ripples" researchers found on AI timelines? Just higher harmonics. Here is the craziest part. Rresearchers tried to break it. They deleted every single sentence in the training data where two months appeared together. No "January" next to "February." No "March" next to "April." And the circle formed anyway. Why? Because the geometry is a collective phenomenon across the entire language. Even if two months never touch in a sentence, they are both pulled into orbit by thousands of other words tied to the same seasons, weather, and events. The rest of human language forces the shape into existence.
5
7
166
17,637
Unbelievably funny that the banks had to tell SpaceX “no we will not lend to you if you shoot the collateral of the loan into space.”
“.. lenders have expressed concerns that if the group later decided to shoot the valuable chips that underpin their loans into space, it would prevent them from reclaiming and selling the semiconductors in the event of a default ..” @FT $SPCX ft.com/content/33b5643c-2957…
18
133
11
1,843
40,038
This is one of my favorite scientific figures in computational social science: the Division of Democrat and Republican Party members of the U.S. House of Representatives from 1949–2012. You can see our history in a figure resembling mitosis. journals.plos.org/plosone/ar…
25
119
12
699
48,259
Xi Jinping logró algo imposible, matemáticamente hablando: que una mesa redonda tuviera una cabecera.
176
528
61
17,160
2,126,183
Not to be insensitive, but the Space Shuttle isn’t a good example of a safety culture. It’s an example of a culture where warnings were ignored by management at the cost of lives.
When I flew the Space Shuttle, we didn’t just hope things would go right. We tested systems and had a plan if things went wrong. We’ve spent decades managing risks in space travel, defense, and other high-risk industries. But right now, AI companies are policing themselves without any real safeguards. My framework puts that proven expertise to work to make AI safe.
56
158
4
3,530
63,963
Han chinese urge to impose monoculturalism has no bounds
It never ceases to amaze me at how liberal democrats globe types just eagerly throw Indian under the bus as being intrinsically inferior to the Chinese just to make sure no blame gets attached to the democratic system. But let’s do a thought experiment shall we? Freaky Friday style. Let’s say it was a Ghandi-style pacifist Democrat that somehow got a hold of China and an Indian Mao that took control from the British. Let’s just focus on language. I find it hard to believe a China that was democratic from the start would have been able to institute mandarin as the universal tongue. Probably something like what happened in India would have happened to China. Mandarin could be put in place as the official language while each region would have its own dialect as the co-official language for that region. It would have proved a terrible barrier to national unity and prosperity. Regional governments would be strong. Regional inequality terrible. Xinjiang, Tibet and Inner Mongolia would easily be pried away. China would have remained the tray of loose sand with no coherency. More prosperous provinces might even agitate for autonomy. I can easily see it turning out worse than India today in fact. Meanwhile, if India had a strongman period that did nothing but broke caste definitively and imposed Hindi as the official language, it would almost certainly have a brighter future today.
6
4
75
2,262
This was a deliberate attempt to humiliate a head of state. She's said publically that she has chronic rheumatoid arthritis for more than 10 years.
韓国のメディアも高市首相の、APEC中国でのタラップの落差を報道
69
186
6
5,182
399,588
China builds world’s biggest lazy susan: diners need 8-foot spoon to reach giant plate of Shenzhen ‘mystery pink slurry’ specialty.
2
10
494
I'd never read the original proclamation by the Indians of All Tribes group that occupied Alcatraz for 19 months in 1969-70. It's pretty funny!
2
18
2
98
5,177
trey retweeted
"State Chairman"
2
10
1,474
imagine telling someone from the 80s that one of these guys was a socialist and the other one was the capitalist
503
4,771
301
80,028
2,590,183
本届APEC大会结束了,如果让我选一张最能代表本次会议的照片,我就选这张,再过几十年,一看照片就知道时间地点,应该害羞的,并不是日本。
1,058
4,423
374
34,180
2,033,427
If you're still concerned about citation hallucination by LLMs, you're about 2 years behind here.
80
292
45
3,385
274,450
早八上习概就是困
Trump listens to Chinese President Xi Jinping's speech at the APEC summit.
64
325
56
6,691
226,457
But the leaders actively sidestepped difficult topics on the sidelines of the Asia-Pacific Economic Cooperation (APEC) leaders' meeting. At one point, they ignored a question about Jensen Huang and Nvidia (NVDA) and whether the ousted CEO was "loyal to the United States." China then barred that Bloomberg correspondent from the next photo-op, according to reporters at the event.
3
2
6
6,852
In the spring of 2023, Anthropic invited me to spend several weeks at their headquarters ahead of the launch of Claude 2, sitting in on meetings and interviewing dozens of employees to write the first major profile of the company in the NYT. I just reread that story, and went back through my notebook in light of Anthropic's IPO. A few things that stuck out: -- Anthropic was very small, very recently. When I visited, they had 160 employees in a small office on Sansome, a chatbot nobody had ever heard of, and seemed to have no interest in running a business. I wrote in my notebook that they seemed like "ambivalent capitalists"; their newly hired sales guy told me that the only reason he needed to make money at all was so that the safety researchers could save the world. -- Things were not going well for them: their biggest shareholder (SBF) had been arrested for fraud, and OpenAI was eating their lunch with ChatGPT. (I later learned that they also had an expensive model training failure in 2023.) This was 3.5 years before they went public as one of the most valuable companies in the world, with 5,300 employees and 14x annual revenue growth. I guess they figured out the capitalism thing! -- They were obviously and sincerely terrified of AI. Claude 2 was a very dumb model by today's standards, but every discussion I had there revolved around existential risk. Researchers told me they lost sleep over their fears of AI takeover, and made terrifying predictions of how soon AGI would arrive. It didn't seem fake or manufactured, but I didn't really get it! I wrote in my notebook: "I have never encountered a company that so badly needs a Klonopin." (I stand behind this, although I also need one now.) -- Dario was in Italy when the NYT photographer wanted to take his picture, so his team rented an office in some random Italian town and staged it to look like his actual office, and they took the photos there. It had a rotary phone (?) and panic-room red lights, which I never really understood. -- The Ants I talked to mostly struck me as sharp and thoughtful, although I did end up wondering whether they were too safety-obsessed to run an actual business. (I jotted down: "I do not know why you would invest in a company like this.") But I was struck by their earnestness, especially because I'd just spent several years covering social networks that didn't worry about the harms of their products at all. Anyway, I'm feeling sort of conflicted about the Anthropic IPO. (As are a lot of people, including some at Anthropic!) I don't know if being public makes an AI company more or less likely to follow through on its safety commitments, and I know the company has changed a lot as it has grown. But on some level I'm also just like, damn, what a crazy story. Here's the gift link, it's sort of fun to look back on the ancient runes of July 2023. nytimes.com/2023/07/11/techn…
45
220
24
1,654
194,482
Given the surge in interest in Shenzhen, here’s a thread of some of my reporting over the last six years beyond the tall buildings and skyscrapers. First, on former factory workers in a rock band singing about the struggles of that phase of Shenzhen. reuters.com/article/business…
9
6
1
26
2,425
trey retweeted
Explaining to the foreigners that the pro-China anti-democracy mayor of Auckland is a right-wing possibly corrupt bigot who is also a centrist YIMBY doing a broadly productive job as Auckland mayor and mainly working with the council's left block.
Mayor of Auckland says democracy has "had its day" and it is time to try another form of government while lauding China's one-party system. Asked what his alternative to democracy would be, Wayne Brown, who recently visited China, said: "What's working? China's working. You know, in terms of economic growth and prosperity" u.afp.com/SFmk
56
252
35
4,686
109,157
Once you begin to grasp the unholy & feverish pace at which the anthropology of the human race has been forcibly reconfigured on a radical level, you cannot un-see it. The way of life we now consider totally compulsory and inescapable was invented less than a century ago. It bears no resemblance whatsoever to 99% of historical human ways of life. We are in completely uncharted territory and most of us simply do not appreciate the full weight of that fact. In fact, it appears we'd rather not think too much about all that.
Photos from Tierra del Fuego of Selkʼnam (Ona) men in traditional costumes. A world that no longer exists.
133
1,095
46
12,257
341,187