John Saigle retweeted
Deterministic Simulation Testing Is For Everyone
outdata.net/blog/261120
Given that big names like Project Zero and Anthropic are moving toward 30 day full disclosure timelines for high-severity bugs, I wonder if solo bug bounty hunters will do the same thing?
60
John Saigle retweeted
"The most load-bearing question, then, is the morality of the one who wields the ring."
John Saigle retweeted
show this video to anybody that is an old school programmer that hates AI
this is one of the best videos I’ve seen all month.
John Saigle retweeted
I unfortunately can't take seriously the big AI devrels on Twitter. Popular software factory repos just don't compute for me as cred for "How to use AI". The people I trust the most are people focused on correctness whose bottom is on the line. Kernel/Inference folks, TigerBeetle, our own Tempo team etc. Basically systems folks. I believe that 90% of application layer advice out there is garbage and actively distracting you.
So I love Cyberpunk lore as a lens for the current moment but this is a misleading read
The Blackwall:
1. Didn't work
2. Was controlled by a centralized org with corrupt elements (NetWatch)
3. Was itself AI
(more) 🧵
Are we witnessing the beginning of the end of the global internet?
In Mike Pondsmith's Cyberpunk, a hacker named Rache Bartmoss unleashes AI daemons upon the world that causes an event called the DataKrash, essentially the destruction of the first internet which resulted in a new, heavily walled guarden, balkanized set of versions emerging.
Smaller, local networks took over, and the Blackwall served as a divider keeping the rogue areas of the net away from the current, functioning ones.
There are some lessons in there as we think about agent swarms and this approach, but the main one I am going to pull out is this:
The easiest way to stop this behavior is to stop this behavior, by which I mean walling things off. The simplest response to hostile nation state actors and agent swarms is to simply block them off. We think of the internet as a global tool, because right now it is, but let me give the following thought experiment:
If your options were constant breaches, crashes, and the destruction of privacy and bricking of all online systems... or having an internet only for your own country and some allies, which are we going to pick?
I think if the cyber threat continues to escalate along this vector, it is more likely than not that the real answer is going to be to balkanize the internet.
1
1
53
4. The mainline internet arguably didn't work well given how easy it is for netrunners to still hack everything
5. AI persisted among the mainline internet, even other autonomous rogue AI (Delamain)
1
17
Read this if you're a bounty hunter or a triager!
Max has a ton of experience on both sides. During our many triage discussions, he has always advocated for the hunter's perspective and the health of the bug bounty scene in general.
This is a measured take from a man who cares
Some thoughts on making bug bounties better. maxwelldulin.com/BlogPost/re…
John Saigle retweeted
Replying to @bilskirnir3124 @thsottiaux
GrapheneOS code quality has greatly increased due to using frontier models for code review and bug finding. Our port of the Pixel 10 drivers from Linux 6.6 (QPR1) / 6.12 (QPR2) to Linux 6.18 went incredibly smoothly because we were able to find most of the issues in advance.
John Saigle retweeted
There’s no excuse for new software not to be logical, not to be deterministic.
Investing in owning your testing, to be DST-first, and to accelerate the local development velocity of your team, is probably the best technical decision you can make.
John Saigle retweeted
It’s refreshing to have access to, and be able to work with, a team of incredibly passionate and skilled (not to mention human) engineers, who want to break the code, and find bugs as much as you do.
To be clear, machines (like the VOPR and LLMs) are great at exhaustively enumerating bugs, but these things are defense-in-depth:
The principle is that you should always be adding layers, but never replace one layer with another.
For example, don’t replace human review with LLM review, but rather add another layer as defense-in-depth. (The LLM should only be a canary, if it finds anything, you know your other layers failed, and you fix them and invest in them).
At the end of the day, the most valuable software in the world is not valuable because of the “time” it took you to create, but because of the “quality” and value you added.
We did this ~1 year ago for Wormhole's major repos and it does indeed help code reviewers (human or otherwise) report fewer false positives
Clearly identifying invariants, and documenting them for others, is still hugely undervalued
The biggest reduction in false positives we’ve seen in AI auditing came from giving the agent a clear threat model.
Put a THREAT_MODEL.md at the repo root and tell the agents to read it.
Cover:
• What the system does and how it’s deployed
• What needs protecting
• Who is trusted, and for which actions
• What an attacker can access or control
• Where untrusted input enters
• Which security rules must always hold
• What’s out of scope, and why
With the latest SOTA models, we’re seeing fewer hallucinations but more findings based on incorrect assumptions about the system, which is now the main source of false positives.
Codex Security’s threat-model skill is a good starting point, but the devs should own the document, verify every claim, and keep it up to date as the codebase evolves.
github.com/openai/codex-secu…
32
There are bits of gold buried in slop. It's not right to harm users just because one finds LLMs distasteful
johnsaigle.com/posts/quoting…
The rust team has absolutely hamstrung itself.
My coding agent refused to even open a PR on my fork with an experiment to reduce this bc the AGENTS.md is so aggressive.
Time to rethink how to scale review!
58
Current LLM failure mode: testing DoS issues via progressive benchmarking and actually risking OOM on the host instead of just like, eyeballing it
2
1
61



















