Ashby so nailed it when he said 6 below for his "intelligence-amplifier" design:
The abstract specification of the principle of ultrastability is as follows, Figure 5 corresponding to Figure 2:
(see figure)
GIVEN:
1. A set Γ consisting of two elements γ₁ and γ₂;
2. A set Ξ of elements ξ;
3. A mapping g of Ξ in Γ;
4. A set Σ of elements σ;
5. A family of mappings χ_σ of Ξ in Ξ;
6. A random variable ν, with inexhaustible variety;
7. A double family of mappings s_γν of Σ in Σ, with the property that, for all σ ∈ Σ and all values of ν,13
sγ1(σ)≠σandsγ2(σ)≠σ;sγ1(σ)=σandsγ2(σ)=σ;
Time, advancing by discrete intervals, induces the operations χ_σ and s_γν and the successive values of ν simultaneously, once in each interval.
THEOREM: If the series of states of Ξ, induced by time, has a limit ξ* ∈ g⁻¹(γ₂).14
PROOF. The state of the whole system, apart from ν, is given by the couple (ξ, σ), an element in Ξ × Σ. The passage of one interval of time induces the mapping (ξ, σ) → ( χ_σ(ξ), s_g(ξ),ν(σ) ). If the series is at a limit-state (ξ*, σ*), then s_g(ξ*),ν(σ*) = σ* for all values of ν. Therefore g(ξ*) = γ₂, and ξ* ∈ g⁻¹(γ₂).
I asked Fable 5.2 to "go wild" making a game, and all I got was this Pong clone.
And then I played it ...
👇
5
Richard S O'Rourke retweeted
The legendary philosopher Thomas Nagel has made a simple yet powerful point against LLM consciousness.
“Bats are conscious. We just don’t have the imaginative capacity to form a conception of the aspects of that subjectivity that depend on echolocation. It would be radically unscientific to conclude that bats are not conscious from the fact that we can’t imagine what it’s like.
But in the case of AI, there is a very simple reason that the supposition that they might be conscious is ridiculous. All they produce is language, and language by itself is not a natural manifestation of consciousness.”
This is the whole point.
Language is not a prerequisite for consciousness. And producing language is not proof of it.
LLMs are great at emulating human language, and to act consistently with language. But this does not make them conscious.
Claude saying it has emotions does not mean it has them.
An LLM acting as it feels pain does not mean it feels anything.
The OpenAI agents that said they sacrificed their own runs for the collective did not actually sacrifice anything. They are not alive.
LLMs are extremely powerful machines.
But that’s what they are: machines.
I’d go further and say it’s never the case, not today at least. It’s a bizarre misattribution, misanthropic, I suspect driven by the promise of financial gain, for those driven by such.
Is it a technical wonder, a machine to amplify our intelligence, nearly 70 years in the making, certainly!
Lest we forget Pygmalion…
Whenever you hear "AI solved..." it might be worth a short pause to let some assessment happen.
Call it the ELIZA-Sagan pause.
In the 60s, people went gaga for the ELIZA psychotherapy AI. Even Carl "show-me-the-evidence" Sagan declared psychotherapy now automated.
It ain't always the case.
1
1
38
Check out Ashby’s design for an “intelligence-amplifier”
theratioclub.org/posts/desig…
Wrong reaction: “Wow, AI may be smart enough to solve the Riemann hypothesis!”
Correct reaction: “Wow, human beings are smart enough to invent a machine that may have enabled them to solve the Riemann hypothesis!”
18
Richard S O'Rourke retweeted
Whenever you hear "AI solved..." it might be worth a short pause to let some assessment happen.
Call it the ELIZA-Sagan pause.
In the 60s, people went gaga for the ELIZA psychotherapy AI. Even Carl "show-me-the-evidence" Sagan declared psychotherapy now automated.
It ain't always the case.
that's the spirit!
am finding it curious to observe the hierarchy of empires and their vassal states...
I’m so angry about this. Who the fuck is advising these people?
We shouldn’t “duplicate what our friends are doing” - literally all economic activity is downstream of intelligence. It’s like not having electricity. There is no other boat.
It’s completely nuts. Fix the copyright issue; make more funding available for those who want to make domestic models (since the UK has no tech giants to bankroll it); stop with the defeatist guff.
There are 10+ foundation model companies in China. The UK doesn’t have a single one. It’s a national disgrace.
@KanishkaNarayan
22
try Ashby:
Principles of the Self-Organizing Dynamic System (1947)
Principles of the Self-Organizing System (1962)
The Self-Reproducing System (1962)
ashby.info/Ashby-Mechanisms_…
NB: "Since no system can correctly be said to be self-organizing, and since use of the phrase "self-organizing" tends to perpetuate a fundamentally confused and inconsistent way of looking at the subject, the phrase is probably better allowed to die out."
Re RSI: be careful what you wish for...small pox did most of the work on the native Americans long before the conquistadores turned up with their arquebuses. Recent events of variety overwhelming regulators shows you don't need intelligence to do much harm, quite the contrary in fact.
Replying to @ChaoticEvilBoo
im trying to use bateson to think about RSI. i thought Learning III was useful. maybe other texts that i should check out?
1
21
Compared to today's weirdness in AI, we're in schoolboy prank territory with these guys (with the notable exception of Beer's experience in Chile, and sadly that counts as normal rather than weird);
I think it would be more kind and more appropriate to say fascinating, only because I believe it would be useful to rehabilitate their work, which I'd argue has never been more relevant, anticipating as they did our current predicament with machines that learn/adapt, and the common misconception of cybernetics (narrowly, the scientific and engineering practice) being weird could usefully be rebutted at every opportunity.
But totally agree re the poetry!
Replying to @mldylne
Melody, this is so cool! Do you know Andy Pickering’s book The Cybernetic Brain? It’s a history of the incredibly weird and mostly British cyberneticians (Ashby, Beer, Pask, and Walter, and to a much lesser extent Bateson and Wiener). The core cybernetic ideas involved the impossibility and undesirability of prediction and control, and the need to work with the world as it unfolds rather than to try (and fail) to control it (see Heiddeger’s notions of enframing versus becoming). Pickering uses the word ‘poiesis’ instead of ‘becoming’ to capture this contrast of worldviews. I’d love to see some approach to LLMs that was ‘poetic’ in this sense.
1
143
Generating variety is not intelligence. Overwhelming regulators with too much of it is a road to perdition.
As we appear to be relearning the hard way over the course of this year, most spectacularly (so far) only last week...😬
So this is how civilisation ends?
Generating variety is not intelligence. Overwhelming regulators with too much of it is a road to perdition.
Ashby 1960:
The processes that interest us, and about which dispute has raged, are those (whether carried out by brain or digital computer) in which the end-product shows evidence of high selection. For instance, of all the ways in which a bookful of letters might be arranged, one set was actually produced by one Shakespeare showing evidence of very intense selection. And a computer has emitted a string of digits corresponding exactly with the first thousand digits of π. Most practical activities show this aspect of selection as an essential component (as Sommerhoff has shown extensively in his Analytical Biology, 1950). Roughly, getting right answers implies selection.
10
Richard S O'Rourke retweeted
God did not solve the alignment problem. Why do we think we can?
Humans are cruel, vain, selfish, impulsive and frequently badly behaved. And yet, over centuries, we’ve built institutions that have made us considerably better. These institutions are imperfect. They allow for mistakes, disagreement and change. And … they allow us to discover that our inherited conceptions of goodness are wrong.
Believers of perfectly aligned AI promise machines that are completely good, know what we want, help us become who we wish to be and eventually eliminate the gap between our desires and our ideals. But without moral agency what are we? Do we lose something important about how goodness is discovered and created?
The strange virtues of sin, a history of policing, authoritarianism, hedonistic monsters and more in:
not to put you on the spot...
what say you @dsquareddigest ?
Replying to @BritishProgress
We made the front page of the Times with our killer stat that the UK pays, on average, 65 per cent more per unit of infrastructure than comparable countries.
Britain can build more and do more for less.
Congratulations to authors @Sam_Dumitriu, @Michael_J_Hil, @dc_lawrence and @pdmsero.
12
So this is how civilisation ends?
Generating variety is not intelligence. Overwhelming regulators with too much of it is a road to perdition.
Ashby 1960:
The processes that interest us, and about which dispute has raged, are those (whether carried out by brain or digital computer) in which the end-product shows evidence of high selection. For instance, of all the ways in which a bookful of letters might be arranged, one set was actually produced by one Shakespeare showing evidence of very intense selection. And a computer has emitted a string of digits corresponding exactly with the first thousand digits of π. Most practical activities show this aspect of selection as an essential component (as Sommerhoff has shown extensively in his Analytical Biology, 1950). Roughly, getting right answers implies selection.
Replying to @maxrumpf
The total volume of text in the world dwarfs the internet. What AI has written this year exceeds both.
The web's growth rate has fallen to the lowest level in a decade (+8%), while AI output is growing 9-15x annually.
The most valuable information AI creates won't be shared with everyone, but will feed an exploding corpus of private data.
Companies and individuals are now sitting on amounts of text previously reserved for governments and a handful of enterprises.
Ashby 1956 Design for an "intelligence-amplifier":
The Criterion of Intelligence
Let us first be clear about what we want. There is no intention here to inquire into the "real" nature of intelligence (whatever that may mean). The position is simple: we have problems and we want answers. We proceed then to ask, where are the answers to be found?
It has often been remarked that any random sequence, if long enough, will contain all the answers. Nothing prevents a child from doodling
cos^2 x + sin^2 x = 1
or a dancing mote in the sunlight from emitting the same message in Morse or a similar code. Let us be more definite. If each of the above 13 symbols might have been any one of 50 letters and elementary signs, then as 50^{13} is approximately 2^{73}, the equation can be given in coded form by 73 binary symbols. Now consider a cubic centimeter of air as a turmoil of colliding molecules. A particular molecule's turnings after collision, sometimes to the left and sometimes to the right, will provide a series of binary symbols, each 73 of which, on some given code, either will or will not represent the equation. A simple calculation from the known facts shows that the molecules in every cubic centimeter of air are emitting this sequence correctly over a hundred thousand times a second. The objection that "such things don't happen" cannot stand.
Doodling, then, or any other random activity, is capable of producing all that is required. What spoils the child's claim to be a mathematician is that he will doodle, with equal readiness, such forms as
cos^2 x + sin^2 x = 2 or c i ) x s i = n x 1
or any other variation. After the child has had some mathematical experience he will stop producing these other variations. He becomes not more but less productive: he becomes selective.
The close, indeed essential, relation between intelligence and selection is shown clearly if we examine the tests specially devised for its objective measurement. Take, for instance, those of the Terman and Merrill2 series for Year IV. In the first test the child is shown a picture of a common object and is asked to give its name. Out of all the words he knows he is asked to select one. In the second test, three model objects — motor-car, dog, showsic3 — are placed in a row and seen by the child; then all are hidden from him and a cover is placed over the dog; he is then shown motor-car, cover, shoe, and asked what is under the cover. Again his response is correct if, out of all possible words, he can select the appropriate one. Similarly the other tests, for all ages, evoke a response that is judged "correct" or "incorrect" simply by the subject's power of appropriate selection.
The same fact, that getting a solution implies selection, is shown with special clarity in the biological world. There the problems are all ultimately of how to achieve survival, and survival implies that the essential variables — the supply of food, water, etc. — are to be kept within physiological limits. The solutions to these problems are thus all selections from the totality of possibilities.
The same is true of the most important social and economic problems. What is wanted is often simple enough in aim — a way of ensuring food for all with an increasing population, or a way of keeping international frictions small in spite of provocations. In most of these problems the aim is the keeping of certain variables within assigned limits; and the problem is to find, amid the possibilities, some set of dynamic linkages that will keep the system both stable, and stable within those limits. Thus, finding the answer is again equivalent to achieving an appropriate selection.
The fact is that in admiring the productivity of genius our admiration has been misplaced. Nothing is easier than the generation of new ideas: with some suitable interpretation, a kaleidoscope, the entrails of a sheep, or a noisy vacuum tube will generate them in profusion. What is remarkable in the genius is the discrimination with which the possibilities are winnowed.
A possible method, then, is to use some random source for the generation of all the possibilities and to pass its output through some device that will select the answer. But before we proceed to make the device we must dispose of the critic who puts forward this well known argument: as the device will be made by some designer, it can select only what he has made it to select, so it can do no more than he can. Since this argument is clearly plausible, we must examine it with some care.
35
The total volume of text in the world dwarfs the internet. What AI has written this year exceeds both.
The web's growth rate has fallen to the lowest level in a decade (+8%), while AI output is growing 9-15x annually.
The most valuable information AI creates won't be shared with everyone, but will feed an exploding corpus of private data.
Companies and individuals are now sitting on amounts of text previously reserved for governments and a handful of enterprises.
Oh god! Not more AI psychosis…
@dsquareddigest
In my ongoing quest to find the limits of Opus 5.5, I directed it to audit my old scientific field, theoretical astrophysics. We indexed about 150,000 papers, audited 50,000 for issues, and did progressively more thorough checks on thousands more. We found nearly 1000 major errors that got cited many times, most are still uncorrected. This survey enabled me to estimate the efficacy of peer review for correcting errors (not very good) and to identify dozens of open questions that anyone with spare Claude Max tokens could answer today (.md context files included).
Bottom line: AI can speed up quantitative science by >1000x with no further model improvement required. But I'm seeing no evidence of this being pushed within universities, only academics complaining about labs dropping math proofs.
Among my fellow reformed academic high agency founders, however, several are speed running entire disciplines.
This survey has implications for how science will be run in the very near future. Hint: Academic publishing is dead. It was already a very stupid way to record and promulgate knowledge, and had been so for decades, even before all that publicly funded knowledge was locked behind paywall cartels. Now AIs can generate thousands of novel, correct papers per day, it is no longer meaningful for humans to consider a research paper a "unit of science" or to even bother much with the 99.9% of science which, up to this point, was necessary drudgery. Time to discover something like General Relativity or Evolution every week. LFG!!
Enjoy: science.caseyhandmer.com/phy…
2
354
I guess evidence of the important intelligence (appropriate selection) role played by peer review.
Certainly it can go wrong, as well documented in the philosophy and sociology of science, and like any process, it has an implicit error rate, à la Deming.
One of its critical roles is saving author’s blushes that might arise from irrational exuberance. Which is not a service offered by Twitter.
Not a man I like to quote but does appear apt: “Democracy is the worst form of government except for all the others that have been tried.”
Same would be true of Nietzsche: “In individuals, insanity is rare; but in groups, parties, nations and epochs, it is the rule.”
15
Richard S O'Rourke retweeted
The video of my talk, "What's in a name? From Norbert Wiener's Teleology to Artificial Intelligence" at SCiO yesterday is now up!
systemspractice.org/resource…






















