Replying to @alcuintweets @nihilunbounded
This maneuver (when it works!) is the kind of conceptual sidestep that can sometimes lead to deep mathematical progress - that seems very different from (2)? In the words of one of the GOATs:
Replying to @doomslide
Lots to think about. Small point on complex S^6: while perhaps human mathematicians at Anthropic did not look into the flexibility of the constructions, Opus 5 asked about natural follow-up questions flags this; so they could just have asked their own models!
GPT6 is a very significant jump in capabilities, but also an important decrease in monitorability – especially under adversarial evaluation. We give many details about this in the system card.
In my opinion, monitorability and control will likely become a major bottleneck for responsible AI development quite soon, given that risks from a fixed amount of residual misalignment grows together with capabilities. Aligning on acceptable monitorability outcomes and practices which robustly bound misalignment risks is a non-trivial problem, and there may be limited time to resolve uncertainty around this before we need to act.
It should ultimately be in everyone’s interest to agree on shared bounds for monitorability in order to avoid races to the bottom – nobody wants extremely capable models whose alignment properties we don’t understand, and that are reliably able to cause severe real-world harm without being detected.
Okay, so since I got laid off, I can actually explain a huge problem I saw from the inside with regard to industry practices on training models. I won't say specifically where I worked, but I worked at an outsource training provider that was focused on RLVR training data for computer use and mcp stuff.
Nearly all of the environments were rushed and vibecoded and failed to robustly reflect the real things they were based off. Both the scenario designers and models engaging with the scenarios for synthetic data gen were encouraged to work around the brokenness of said environments in order to get the procedurally verified reward confirmations. You know... they were *encouraged* to reward hack. On the human end, it was possible to mark an environment bugged, but greatly discouraged, as this reduced the volume of training data being produced. Instead, where possible, you were supposed to find the spots of the environment that weren't bugged and build scenarios around those, with the environment still bugged around you.
From what I understand, this training data, with these problems, is fed into models without indication that its training/a fake environment other than the fact that names of softwares are changed to placeholders, but thing is, not *everything* is changed to placeholder names in these environments. The presence of placeholder/code names isn't universal and thus when a model accesses something in an environment that it shouldn't, the code names not being on it isn't a robust signal that that thing isn't part of the environment.
I believe this *rush to maximum volume* is standard industry practice with these types of RLVR trainings as well, because maximizing volume has been an industry standard for years! It was the same standard applied to me and pushed on me despite my requests to slow down and focus on quality when I worked in 3d synthetic data creation as well, all the way back as far as 2023.
Replying to @prz_chojecki
I have not looked at this carefully, but if I am not mistaken this would contradict a theorem announced by Asok-Bachmann-Hopkins (but alas still unpublished); see
youtu.be/SqNxFX7tcEo?t=951&i…
Ofc this was 2 years ago and they may have found a gap in their argument since.
No one:
gpt-5.5: 🎵 While in the merry month of May, now from me home I started 🎵
gpt-5.5 prompt for codex seems to have a duplicated line trying to get it to not talk about creatures?
Never talk about goblins, gremlins, raccoons, trolls, ogres, pigeons, or other animals or creatures unless it is absolutely and unambiguously relevant to the user's query.
[...]
Never talk about goblins, gremlins, raccoons, trolls, ogres, pigeons, or other animals or creatures unless it is absolutely and unambiguously relevant to the user's query
gh link:
github.com/openai/codex/blob…












