Jordan Taylor

CoT controllability is near saturated for many reasoning lengths. When prompted, the model is able to e.g. produce reasoning unrelated to the question or its answer.

"[...] the set of tasks Astra is able to accomplish without the use of CoT is greatly expanded compared to prior models. [...] See similar discussion in UK AISI evaluations, suggesting the no-CoT time horizon may have increased by about an order of magnitude."

The full report gives a lot of recommendations for model developers, but here are some recommendations I'm most excited about for general AI researchers:

White-box methods are also potentially promising, but are not maturing quickly enough to fill in for degradation elsewhere. And while action-only monitoring provides a floor for oversight, there are many settings where bad actions are hard to distinguish from benign ones.

Although promising, these training-based approaches are face challenges around generalisation:

There are a several interesting training-based approaches for enhancing oversight. Building upon or red-teaming these would be valuable!

Running evals for misalignment ahead of time would be ideal, but eval gaming is already threatening to undermine the validity of these tests:

But there are a bunch of ways in which we're playing on easy-mode today, relative to how difficult oversight like this could be in the future. Chain-of-thought reasoning is currently the most informative monitoring signal, but it is also at the most risk of degradation:

Right now, we have decent oversight IMO. Not great, not terrible. When AIs do bad things, they can often be caught through a range of techniques:

We interviewed a lot of experts and did our own analysis on how it will get harder to tell if AI systems are safe.