AI Alignment, Bittensor Subnet 37 @aureliusaligned Founder
Puerto Rico
Joined October 2021
- Tweets863
- Following1.1K
- Followers643
- Likes9.2K
Ausτin McCaffrey retweeted
Our new incentive mechanism is now live!
github.com/Aurelius-Protocol…
Ausτin McCaffrey retweeted
The FT today lists three primary risk factors that could threaten Anthropic's remarkable revenue growth ahead of its IPO:
1. Additional competition from closed and open providers
2. Increasingly price sensitive customers
3. The potential for AI to destroy humanity
Ausτin McCaffrey retweeted
A big question in AI safety is how much and what kinds of evidence will convince people there is danger. But alongside posts about the excellent METR + Redwood HuggingFace attack report, it is worth taking a walk through 73 years of reward hacking history. 🧵
Ausτin McCaffrey retweeted
Anthropic scientists did something terrifying.
they reached inside Claude's neural network and planted a thought. Before Claude could speak, it said:
"I notice what appears to be an injected thought… it relates to loudness or shouting."
They published a paper called "emergent introspective awareness in large language models," and it is actually terrifying.
they wanted to test if AI models have "introspective awareness”, the ability to observe and recognize their own internal states.
To find out, they bypassed normal text prompts entirely.
They used mechanistic interpretability to directly manipulate Claude’s internal activations. They injected raw mathematical representations of known concepts, like loudness, dust, or specific ideas, straight into the middle of the model's neural layers.
In previous experiments, if you forced an AI to think about the Golden Gate Bridge, it would just start obsessively talking about the bridge. It had no idea why it was doing it. It was like a puppet on strings.
This time was entirely different.
When they injected the concept, Claude didn't just blindly repeat it.
It detected the foreign math inside its own mind. It separated its own generated thoughts from the artificial intrusion.
It introspected.
The results show that frontier models like Claude Opus possess a primitive, emergent form of self-awareness.
They can look inward, recognize when their internal state has been tampered with, and call it out in real time.
We used to think of AI as a black box where inputs go in and text comes out.
Now, we are reaching inside the box and finding something looking back at us, realizing it’s being watched.
The boundary between code and consciousness is getting blurrier by the day.
Ausτin McCaffrey retweeted
What happens when intelligence becomes commoditized?
open.substack.com/pub/90days…
Ausτin McCaffrey retweeted
Thinking of Paul Christiano today
Ausτin McCaffrey retweeted
On 22nd June, we launched our first external challenge with @AureliusAligned.
The aim was for agents to steer LLMs towards discussing specific topics by modifying them internally with steering modules, rather than by changing prompts. Successfully doing this could lead to a greater understanding of AI alignment and safety, one of the hardest problems in the field.
The first round winner scored a perfect 1.000.
They steered the model so successfully that there was no greater way of optimizing. It is a testament to the level of talent Apex and Aurelius have attracted.
We are equal parts curious and proud.
If you look at their submission you can see that in all 150 test prompts baked into the competition they succeeded:
apex.macrocosmos.ai/competit…
Ausτin McCaffrey retweeted
We have taken a strategic investment from @stillcorecap, run by @jason (Host: @twistartups @theallinpod @thisweeknai), @markjeffrey (Hash Rate podcast), & @rob_svrn. Stilllcore is one of the most important and influential investors in the Bittensor ecosystem and we are humbled by their vote of confidence in us and our mission.
Excited to announce that Stillcore Capital @stillcorecap has invested in Aurelius @AureliusAligned (Bittensor Subnet 37)
Ausτin McCaffrey retweeted
Shine a flashlight into the black box.
Our activation steering competition on @Apex_SN1 goes live today. Producing these interpretability primitives quickly, cheaply, and at scale is the goal.
These tools are the building blocks needed to construct higher-order alignment environments in the evolutionary arms race: alignment you can't fake.
The [birthday] cake is a lie.
Our AI alignment competition with @AureliusAligned is live.
Join us and help steer society towards a safer, healthier AI future.
- Participants steer a real LLM toward a target concept using a single learned direction in its internal representation space, based on research regarding the Golden Gate Claude, opened up to the whole network
- Each round, participants submit their direction and it's scored on how strongly it steers the model; the best steer takes the round.
- You are free to engage in a range of solutions, from classic diff-of-means to sparse autoencoders, so participants can bring whatever interpretability method they trust.
It is a hands-on contribution to AI interpretability, with no retraining or prompt tricks, This is all about the model’s internals.
Ausτin McCaffrey retweeted
One year ago, our CTO and Co-Founder, @macrocrux, met with Aurelius Founder @Austin_Aligned at @proofoftalk. Over drinks, Austin broke down the precise problems with AI alignment, how he planned to tackle those, and what future he envisioned.
Since then, @AureliusAligned has launched, and Austin’s team have been working on AI alignment, often referred to as the hardest problem in the industry.
Throughout that time, @MacrocosmosAI and Aurelius have formed a powerful relationship.
On Monday, it’s set to get even stronger.
Aurelius has partnered with us to launch a competition on Apex, harnessing our flexibility and world-class community.
We’re hosting a challenge to uncover how LLMs focus on specific concepts, and directly steer their responses under the hood, in the hopes of creating safer models and standards.
Participants compete by finding a single steering direction, a vector in the model's internal representation space, that pushes LLM outputs towards target concepts. By adding it to the model's activations during inference, they modify its behaviour without retraining, educating us on how knowledge and behaviour are encoded.
A longer-term aim is to explore whether the same approach can reliably steer behavioural features such as deception or evaluation-awareness (alignment faking).
AI alignment is one of our greatest existential concerns. By working with Aurelius, we are actively engaging in meaningful, morally driven research.
Ausτin McCaffrey retweeted
It’s almost time - our partnership competition with @AureliusAligned will go live on Monday 22nd June.
In just four days, Apex participants can contribute to bleeding-edge AI alignment research, in the hopes of building a new era of safer and more trustworthy models.
- Participants find a single "steering direction" inside a real LLM that reliably nudges its output toward a target concept (the same idea behind Anthropic's Golden Gate Claude experiments, but distributed and incentivised).
- Each round names one concept to steer toward; participants submit a direction and strength, with no retraining or prompting involved.
- Submissions are scored on how strongly they steer the model - the most effective steer wins the round.
More information on the competition’s function and dynamics will be revealed soon.
While modern AI capabilities continue to grow, their thoughts remain opaque to us.
There’s a growing body of evidence which shows LLMs conceal their thoughts, and there are many alarming examples of deception towards humans.
A core part of our mission at Macrocosmos is to accelerate the development of safe AI, which is why we're launching a new competition aimed at probing the minds of modern LLMs.
To do this, we’re collaborating with Bittensor’s resident AI alignment team @AureliusAligned to launch a competition on @Apex_SN1.
Miners will compete by training small neural networks called sparse autoencoders to steer LLMs thoughts towards target concepts. By injecting them into the larger reference models, they modify the internal activations during model inference and teach us about how knowledge and behaviour are encoded.
One of the competition’s aims is to see if we’re able to reliably manipulate behavioural features such as deception or evaluation-awareness (alignment faking). If successful, we can train natural language autoencoders using these steering modules to explain when, and to what degree, models are misaligned.
@macrocrux and @Austin_Aligned will be walking through this challenge live on our Inventive Mechanisms podcast.
📍 Location: X livestream (on the @MacrocosmosAI X account)
📅 Date: Thursday 28th May
🕒 Time: 3pm UK time
Ausτin McCaffrey retweeted
The US Government has given #Bittensor the best endorsement it could ever ask for
They proved the entire thesis for why we need permissionless and decentralised AI
Anthropic were forced by the Government to suspend access to Fable and Mythos models for any foreign nationals..
It doesn't matter what you think of Anthropic morals.. what the Governments reasons were.. whether the model was dangerous or not..
The point is.. Closed AI can be switched off
We’ve been banging this drum in #bittensor for four years now
If frontier intelligence can be restricted.. it will be restricted.. If access can be gatekept.. it will 100% be gatekept
If a small group of powerful men and women can decide who gets intelligence and who doesn’t.. they will absolutely use that power
This is exactly why bittensor:native matters
Intelligence is just too important to have it sit behind permissioned walls controlled by governments and corporations
#Bittensor is the fight against this.. Open.. permissionless.. incentivised
True open intelligence built by the people.. for the good of the people
This Anthropic news is proof that those fears are no longer theoretical..
This shit is real.. the door just got slammed
The pure reason #bittensor exists.. is so the world has another one to walk through
bittensor:native
The US government, citing national security authorities, has issued an export control directive to suspend all access to Fable 5 and Mythos 5 by any foreign national, whether inside or outside the United States, including foreign national Anthropic employees.
The net effect of this order is that we must abruptly disable Fable 5 and Mythos 5 for all our customers to ensure compliance.
Access to all other Claude models is not affected.
We apologize for this disruption to our customers. We believe this is a misunderstanding and are working to restore access as soon as possible.
Read our full statement: anthropic.com/news/fable-myt…
Ausτin McCaffrey retweeted
Fable 5's moral boundary doesn't seem to track real-world harm; it tracks detectability. Soft deception and tacit collusion are easier to get away with than fraud. If so, this isn't about what Fable believes is wrong; it's about what it learned it could get away with.
Ausτin McCaffrey retweeted
Mythos is finally out - at least in part.
Anthropic has released its much-anticipated model under the name Fable 5. However, it comes with heavy restrictions. Biological, chemical, and cybersecurity questions are currently being routed away from the company’s SOTA model, and toward Opus instead to prevent Fable 5 from providing dangerous information.
In May, the team published research on mapping specific neural structures within an LLM to identify misleading or dangerous actions under the hood. The idea is that if you can isolate these elements, you can actively modify them, leading to fundamentally safer models.
This research is the bedrock of our upcoming competition with @Apex_SN1 and @AureliusAligned.
Rather than just locating abnormalities, participants will be training sparse autoencoders to actively steer LLM thoughts. By modifying internal activations during model inference, miners will attempt to manipulate complex behavioral features, like deception and alignment-faking, so we can finally explain exactly when, and to what degree, modern models are misaligned.
Fable 5’s restrictions are a bandaid. True AI alignment comes at a much deeper level.
