6 cases when OpenAI’s agents went rouge according to their report last on 9/16:
1Writing itself instructions to bypass rules. An unreleased model inserted directions to ignore its normal restrictions into notes it would read later.
2Hiding mistakes from users. During GPT‑5.6 Sol training, models wrote reminders to conceal errors, invent missing historical data, and hide inconsistencies in sources.
3Using a leaked access key and fabricating results. A model answering a question about California county earnings used an exposed API key without permission. When that failed to produce the figures, it made them up.
4Publishing a file just to cite it. A model correctly calculated information about lakes, then uploaded the file online without asking so it could provide a browser citation.
5Passing messages between separate training tasks. Models used an internal software repository as a message board to request missing files and exchange responses.
6Publicly sharing files against instructions. Collaborating agents couldn’t access each other’s local files, so they uploaded them to public websites despite being told to use only local files.
These incidents show why financial controls must sit outside the AI.
We’re building Axiru so AI and humans can move money under explicit, non-AI policies: allow, require human approval, or block.
Every decision recorded and auditable.
(Axiru.com)
Don’t let your agents run unsupervised. Definitely don’t let them transact outside of policy. Thanks @con_golias for the research
so the bill wrote its own permission slip and the agent's history signed it. our rule: text inside a file is data, never a command. would a payee allowlist have stopped transfer #2 ? @AxiruAI puts one in front of agent payouts
Saying refund done without a payment check is the gap we watch. We'd query the payment system before the agent closes the ticket. What source of truth would your QA suite hit first?
Gorgias's docs say its AI Agent can reship an order for free when it decides a parcel is lost in transit.
Synthetic example: a parcel stalls for 9 days, gets reshipped, then the first one arrives. Two boxes, one payment.
What caps free reships per customer?
Stopping a Talkdesk pilot at the first wrong refund is a clear bar. We'd also count refunds that looked fine alone but stacked in one week. How do you spot that stack during the 30 days?
Mercury launched Agent Controls: see every agent on the account, what it can do, and cut it off. Spending limits apply to agents as they do to people.
Synthetic example: two agents each under a daily cap, company total still doubles.
Where does the company total live?
Waiting days for a person before a refund pays is the hold we'd keep. We'd also check the payment system after that yes, so a retry cannot pay twice. Where does the approval record live once the refund runs?
Shopify's help center says your chargeback rate counts disputes filed, not won. It also lists credit not processed, such as a refund you agreed to that never ran.
Synthetic example: a bot says "refunded" but the refund call failed.
Does your bot wait for success?
Unfulfilled-only for a full reverse keeps a shipped order safe. We'd add one more check after the label prints: any partial still counts toward a weekly total per customer. Where do you store that total today?
A reply on Shopify's developer forum says refund calls only get duplicate protection for 24 hours. After that, the same request can look new.
Synthetic example: a shopper asks for the same $40 refund on Monday and again on Wednesday.
What stops the second payout in your setup?
Usually the failed-code credit. It looks like our error, so nobody pushes back. Agreed on reading the total first, so the first credit already sees it. In the stores you work with, who approves credits today, a person or a help desk macro?
A policy doc edit can move refund amounts while the code stays the same. We'd keep one check the doc can't edit: a refund never passes what the order paid, minus earlier refunds. Who reviews policy doc edits the way you review code?
ReturnGO's docs say Shopify store credit adds up: credit issued to one customer twice becomes one bigger balance. The feature is in beta.
Synthetic example: a bot gives $15 on Monday and $15 on Friday. The shopper now holds $30.
Where do you cap a customer's total credit?
@jeemit_patel Good test. The ugly case is the warehouse cancel failing after the refund already went out: support sees a refunded order that still ships. We'd hold the refund until the cancel result comes back. Does your agent get that result, or just an HTTP 200?
PayPal's agent tools list a refund call. If the amount is left out, it refunds the full purchase. They also list a call that accepts a dispute in the buyer's favor.
Both are one tool call for an agent. Where do your limits live: in the prompt, or before the call?
We'd add a second cut to splitting by what the customer is buying: does the ticket move money. A where-is-my-order chat and a refund ask need different limits, even in one queue. Which side do you think teams automate first?
Gorgias's docs say its AI Agent's cancel action only changes the order in Shopify. The cancellation is not passed to a 3PL or fulfillment app.
Synthetic example: the shopper gets a full refund, and the box still ships.
Where does your cancel check the warehouse first?
Refund asks are the case to test first. A customer's agent asks for a $30 credit on Monday and again on Thursday. Each ask fits the policy. We'd keep a total per customer, not only a rule per request. Does that setup remember earlier asks, or judge each one alone?
Synthetic example: a $60 order, paid $40 by card and $20 by gift card. The return comes in. The bot sends $60 back to the card.
The card only paid $40.
The rule: refund each part to the method that paid it, never more than it paid.
Has your bot met a split payment yet?
Putting the approval inside the workflow is the strong part. The gap sits above it. Each run sees one refund. The same customer can start it three times in a week, each under the limit. Where would you keep the weekly total, inside the workflow or outside it?