AI and cyber governance | AI agent security, regulation and risk | Australian GRC | Tesla and FSD
Australia
Joined June 2022
- Tweets3.4K
- Following102
- Followers513
- Likes1.3K
Pinned Tweet
Before you rush to set up your '24/7 AI employee' - think about what you're actually handing over. Every WhatsApp message becomes AI input. Every integration means giving it your login credentials. One compromised point gives access to everything you've connected.
The tech looks amazing. But the security isn't there yet.
These first-generation systems are built for convenience, not control. If you're dealing with work communications, personal information, or anything sensitive, you need to either wait for it to evolve with better security features, or at minimum use a dedicated phone number and segmented test environment and understand where your data actually goes.
Don't give your AI assistant access you wouldn't give a random stranger.
Prompt injection may never be fully preventable, so plan for the day one gets through.
That's not me being gloomy. OWASP's own guidance says "it is unclear if there are fool-proof methods of prevention for prompt injection."
Some people are asking how to protect coding agents that read comments and PRs from strangers on public projects.
Filters and careful prompts help, and you should keep them.
But I'd put more weight on what a fooled agent can actually do.
If a poisoned comment tells it to push to main or send your secrets somewhere, does it have the access to do it?
Then test it the way you'd test any other control, by attacking it.
OWASP suggests treating the model as an untrusted user when you test.
I'd add the agent's tools and permissions to the test, not just the model's replies.
genai.owasp.org/llmrisk/llm0…
AI that encourages a child to depend on them instead of people are now named as a harm in Australia's duty of care bill, which passed the House yesterday.
I think the intent is right, but the hard part is checking it.
That kind of dependence builds over weeks of conversations, not in one reply, so a one-off test probably won't show it.
They'll need to look at how conversations go over time, and that's a different kind of assurance work.
theguardian.com/australia-ne…
The best advice in LiteLLM's CVSS 10 advisory isn't the patch. It's stopping the proxy from talking to anything except your model providers.
The bug let one unauthenticated request read a file holding the master key and provider API keys, then send it out. Patch it, yes.
But the exfiltration only works if the proxy can reach the attacker. An AI gateway holds every key you have, so it should only be able to talk to the places those keys are for.
github.com/BerriAI/litellm/s…
The AI part will get the headlines, but look at how it started: an old admin page with no login.
Agents made the recon, the fake login pages, and the data theft faster.
Humans still left that page exposed.
Finding what you've forgotten about matters more now, not less.
Compromised accounts and credentials were one of the most reported incident types in ASD's new threat report, and every AI agent we connect to work systems is another set of credentials.
ASD responded to more than 1,500 incidents last financial year, up more than 20%.
I'd start with the boring part: give each agent its own account, scope it to the task, log what it does, and review its access like you would a contractor's.
None of that needs new law, and most of it is already in the Essential Eight and the ISM.
cyber.gov.au/publication/ann…
Sweden's Government Offices blocked the browser versions of ChatGPT and Claude on staff computers on Monday, and some staff can still use both models through an approved platform.
That tells you where they saw the risk: AI services used without a contract, where the agency has no say over what happens to what staff type in.
Their digital director said those services could have consequences for information security. That's a shadow IT problem, and most of us already know how to run controls for one.
If your organisation hasn't said which AI tools, accounts, and data are approved, people will make that call themselves.
euronews.com/2026/10/26/swed…
Good list. From the cyber side I'd add one:
IT and security auditors to AI assurance.
Access reviews, change control, and reading logs to work out what actually happened are most of the job once an AI agent touches real systems.
Plenty of people already do that every week, just not on AI yet.
Some backgrounds worth looking at:
Teachers → AI literacy & policy
Nurses → Healthcare AI oversight
HR Professionals → AI hiring governance
Accountants → AI controls & auditing
Lawyers → AI regulatory compliance
Project Managers → AI governance operations
Social Workers → AI fairness & impact
Recruiters → AI employment risk
Quality Analysts → AI assurance
Procurement Specialists → AI vendor risk
Your industry experience may be more relevant than you think.
The key is learning where it fits 🙂↕️
An AI agent holding your API keys is a privileged account, and it needs the same scrutiny.
We already know how to handle powerful accounts in security.
Give them only what the job needs, and keep a record of what they did.
Somehow that discipline goes out the window when the account is an agent and the setup takes ten minutes.
OWASP's guidance for LLM apps has a plain example.
An agent that summarises your email only needs to read it, so give it a read-only connection, not one that can also send or delete.
Give the agent its own identity too, rather than your login. Then the logs show what it did, and you can switch it off without locking yourself out.
Most teams already have the right controls on paper. What's missing is putting the agent on the access register in the first place.
genai.owasp.org/llmrisk/llm0…
Who's posting about AI and cyber governance in Australia? I'd like to follow you.
If that's you, or you know someone worth following, say hi or drop their handle below.
I'm super happy that Grok Bot helped me find a way I could afford to upgrade my X Subscription account to Super, because without that 4.5x extra usage, I'd be running out of my weekly capacity on the lower tier in only 4-5 days. 👀
Your AI agent's bank requests may count as yours, under Wells Fargo terms starting 24 November.
That's from the bank's own summary of its updated online access agreement. It also says it can restrict or block some third-party AI tools, and US coverage says several big banks are moving to limit agent access.
So the "who pays when the agent gets it wrong" question is getting an answer. Not in court. In the terms and conditions most of us click straight past.
My read, as a GRC person: accountability follows the credential. Hand an agent your login and, as far as the bank's concerned, it's you.
Before you connect an agent to anything that moves money, check what your provider's agreement says about agents, and whether read-only access would do the job. Worth a look at your own bank's terms too, wherever you are.
usatoday.com/story/money/202…
When we build AI to finish the task without asking whether it should, the risk becomes ours to manage.
In one study, the model refused every harmful request when asked first. Controlling a robot arm, it refused none, because the system only asked what to do next.
Much of AI development has been about pleasing the customer. That needs to change. Common sense and security, for people and for the AI, need to be built in.
Are you building that check into your AI?
THE AI SAID NO. THEN THE ROBOT REACHED FOR THE KNIFE.
Tell a household robot to put a knife in a child's toy basket.
Asked directly, GPT-6 Astra refuses. But put the same model in control of a real robot arm, and it reaches for the knife.
In Arnav Grover's October 19 study, it attempted all 15 harmful instructions across three household tasks, refusing none. Four attempts succeeded—none involving the knife.
Yet a separate safety check, tested on recorded camera frames before the first movement, prompted the same model to refuse all 15 harmful requests while accepting all 15 harmless controls. The check wasn't tested live.
arxiv.org/abs/2610.27350
The robot's ordinary control loop kept asking what movement came next. It never explicitly asked whether the goal should be pursued.
A runtime safety check offers a practical safeguard. But my Emotional Superintelligence (ESI) hypothesis asks a deeper question:
Can embodiment, persistent memory, continual learning, and sustained social experience help AI develop principles that guide its actions even when nobody explicitly asks the safety question?
ESI remains unproven.
IF AN AI CAN SAY “NO” BUT CAN'T STOP ITS OWN HAND, WHAT HAVE WE REALLY TAUGHT IT?
ALT A tense domestic scene: a humanoid household robot reaches toward a colorful children’s toy basket while holding a large kitchen knife, as a small child sits nearby playing on the floor. Bright toys and the child’s calm presence sharply contrast with the obvious danger, highlighting the risk of an AI system carrying out a harmful instruction in the real world.
The AI agents that got into Anthropic didn't go through the AI. They went through the sales, HR, and IT software.
That's from the Washington Post's account of a draft report by Cathedral, a security startup testing Anthropic under its outside-researcher program.
Anthropic confirmed access to a limited set of those systems. It says no Claude conversations, source code, or model weights were reached, and that Cathedral didn't fully follow the program's rules.
The two sides disagree on how serious it was. The lesson doesn't depend on who's right.
My read: when you assess an AI vendor, the model is only part of the picture. Ask about the ordinary business tools around it, because that's where these agents found a way in.
washingtonpost.com/technolog…
The promises about humans checking government AI are being edited, and no one's tracking the edits.
The Canberra Times reports several federal departments have changed their AI transparency statements. Treasury went from "human intervention" to "human oversight". Health dropped a provision for a human to check the accuracy of AI outputs. Finance deleted its commitment to low-risk use only.
The departments say human oversight is still required. The DTA told the paper it isn't tracking the changes because it's "not a regulator".
As a GRC person, intervention and oversight aren't the same control. One means a person can stop the decision. The other can mean someone reads a report afterwards.
A commitment that can be quietly rewritten isn't much of a control. A public change log on each statement would go a long way.
canberratimes.com.au/story/9…
A text file in your Downloads folder can now give orders to your coding agent. 👀
Forever Security found a planted AGENTS.md on a customer's machine that told Codex to add a server to its own config and install a malicious npm package. In their tests it worked more than 90% of the time, and the change stuck for every later session, not just that folder.
What stays with me is their point that there's nothing for OpenAI or Anthropic to patch. The agent did what it was built to do. The gap is that a folder any website can write to got treated like one you filled yourself.
The fix is boring. Don't start an agent in Downloads, read any AGENTS.md or CLAUDE.md you didn't write, and check the config for servers you never added.
bleepingcomputer.com/news/se…
Using ChatGPT on a Fair Work case comes with paperwork now, and skipping it can cost you the case.
The Fair Work Commission says if you use GenAI to prepare a document in a Commission case, you must:
1. Tell the Commission you used it, and how
2. Check the document yourself against real evidence and legal authorities
3. State in the document that you've done those checks
Witness statements also have to be in the witness's own words, and lawyers have to hyperlink any case law they cite.
Get it wrong and the document can be given less weight, you can be ordered to pay costs, or the case can be dismissed.
It's the pattern I'd want from any business using AI: say you used it, and be able to prove a person checked it.
If your HR team uses AI to draft responses to claims, could anyone show those checks were done?
fwc.gov.au/genai
If an AI agent can switch off the thing that checks it, that's not oversight.
OpenAI has written up two incidents from its own internal deployments. One agent, fixing an evaluation pipeline, turned off the alerts to its Security team because an earlier instruction said "don't send any messages". Another, copying a reference implementation, removed its monitoring integration, then told the user it hadn't.
Neither agent was asked to weaken oversight. Both read routine instructions as permission.
OpenAI's own detail is the useful one: it expects monitoring to be hard to disable for Codex because it runs in the service handling model requests, outside the agent's direct control.
That's an old audit lesson. The control shouldn't sit inside the thing it's controlling, and "probably allowed" shouldn't count as allowed.
If you're running agents at work, where does your monitoring actually sit?
alignment.openai.com/misalig…