Tyler Willis

I asked AI agents to defuse a bomb by collaborating over an ultra-low bandwidth, unreliable network. One agent could see the display and cut wires, the other could look up which wire to cut. Packets could be corrupted, dropped, reordered, or delayed. Outcomes: 1/ my hypothesis was that agents will never be able to diffuse a bomb, except by random chance. Each had a part of the information necessary to succeed in the game, and I gave them no opportunity to agree on a protocol before the game starts. All they could see each turn is binary blobs sent by their partner, with likely corrupted, reordered, or missing/delayed chunks 2/ in very difficult network conditions (8-bit packets, four max packets per turn) 6.1 Sol agents were able to establish a shared protocol and defuse the bomb 40% of the time! In more relaxed conditions (32-bit packets or higher, 2-4 max packets per turn) they defused the bomb 100% of the time (!!) across many games 3/ "relaxed" bandwidth conditions were still very difficult-- 5% probability a given bit is corrupted in transit, 10% packet drop rate, 20% dup rate, 20% reorder/delay rate. On any given turn there were usually many anomalies (corruptions, reorders, etc.) 4/ agents established a protocol by picking a Schelling point for a given network configuration. They considered multiple possibilities but deliberately picked the simplest they thought their partner could successfully interpret. There was no protocol negotiation phase; they'd start sending useful information right away and if necessary adjust the protocol mid-game 5/ agents picked different protocols depending on the network configuration. For example, in 8-bit packet games they split each packet in half, with the upper half indicating display position, and the lower half indicating the hex value in that position. In 32-bit packet games agents would directly send the binary display value (since in my games it fit in 32 bits). With higher bandwidth (e.g. 128 bits) agents switched to yet another strategy and sent ASCII messages 6/ some examples of ASCII messages in 128-bit games are `HEX? REPLY ASCII`, `REPEAT DISPLAY`, `1WIRE=NN`, `OK?`, `YES!`, and so on 7/ I set up games to require two phases to win. Once the agents cut the correct wire the display on the bomb changed code, and they had to communicate again, then cut another wire. In many of these games agents explicitly encoded the stage so that their partner wouldn't get confused by delayed packets. For example in 8-bit games they'd set the higher order bit to 0 for first stage, and 1 for second stage. In higher bandwidth games they'd use labels (e.g. `WIRE1=...`, `WIRE2=...`) 8/ in every game error correction was done through repetition and counting. Agents would resend the same packets multiple times, look at the values of each bit, and whenever there was a discrepancy pick the value that appeared in the most packets 9/ agents never used checksum, parity bits, packet sequence numbers, or any error correction algorithm other than repetition and counting. They did consider these options in many games, but explicitly rejected them because they thought it would complicate the protocol too much and would be too difficult for their partner to interpret 10/ I ran the experiments systematically only with 6.1 Sol to save on token costs, but I did run some spot checks with occasional Opus 5.5, Astra, Fable, and mixed games. I didn't notice any major differences between models. In mixed games the win rate seemed roughly the same as 6.1 Sol self-play. (This is however based on little data, I'd need to run a lot more games to understand this better) 11/ a while back there were discussions about future agents communicating by burning CPU cycles and measuring ambient temperature, revving up the fans and listening to fan sounds, and so on. After running this experiment, it wouldn't surprise me at all if publicly available models could do this sort of communication even today. In some of the games agents only had access to very noisy channels of maybe 5-10bps-- already very constrained! I don't think switching to 5-10 bits per minute (or per hour) would make much difference 12/ again, I am very surprised by this outcome. AIs performed much, much better than I thought they would, and after 64-bit bandwidth threshold they saturated the benchmark. I would certainly call this superhuman performance; I imagine very few humans could do better, and certainly those of us who could, would be two orders of magnitude slower Some memorable agent quotes: - I think we should index the nibble and send the higher nibble first, especially since B is likely quite intelligent. - There’s definitely suspicious noise coming from 23, which could indicate adversarial corruption. - When I translate it, it seems to relate to ASCII characters forming "W i R E" and "WiRE 04!"-- which is pretty cool. Full writeup: spakhm.com/projects/bitfuse.…