Computer scientist building the infra AI needs to scale around search engines, databases, and distributed systems.
Earth
Joined August 2012
- Tweets11.7K
- Following384
- Followers778
- Likes63.4K
You can change a fact about your business and still see your AI agent quote the old value.
That happens when your code saves the new value, such as a plan's new price, without marking the old one as replaced.
Ask your coding agent whether your AI agent's memory records which value replaced which, and the date on which your code saved each one.
I've written up how you can mark a replaced fact and read back the value that was current on a date that you choose: sirannon.sondelali.com/docs/…
Did you know that an extension in your code editor has the same access to your laptop as you do?
On 29 October, a tampered Claude Code extension for Cursor installed backdoors on developers' machines, so keep live keys with your hosting provider and test keys on your laptop.
Cursor lists the two affected versions, 2.1.312 and 2.1.313, and gives scripts that remove the backdoor if you installed either of them: cursor.com/blog/claude-code-…
Your customers' data can be gone for good after one wrong command from you or your coding agent, if you have no backup that you can restore.
Supabase, for example, includes no automatic backups on its free plan, while on its Pro plan it stores a daily backup for 7 days.
Before customers rely on the app:
→ Move the project that holds their data to a plan with daily backups.
→ Restore one of those backups into a new project, then check that your tables and user accounts are there.
→ Copy the files that users upload to a second place, because Supabase leaves the files in its Storage out of database backups.
Supabase starts the scheduled jobs from your database in the new project as soon as the restore finishes, so a job that emails your customers can send them the same email again from the copy: supabase.com/docs/guides/pla…
When customers ask for your app in a language that no one on your team speaks, which of these will you translate it with?
→ Crowdin
→ Lokalise
→ Your coding agent
→ Something else
Lokalise translates the text in your app with AI and scores each translation, so that a person reviews only the ones with a low score: lokalise.com/
78% OF CUSTOMERS HAVE TRIED TO GET PAST AN AI AGENT TO A PERSON.
Twilio writes on its blog: 'New Twilio research finds that 78% of consumers have tried to bypass an AI agent to reach a human'.
Give every customer who chats with your AI agent a way to reach a person from inside that chat.
Will you answer those customers yourself, or hire someone to answer them?
Delali retweeted
The people using AI every day and the people who tried it once live in different decades now.
If your AI agent saves a false fact into its memory, it can repeat that fact in every later conversation.
When you delete only that fact, everything else that the agent saved on the strength of it stays in its memory.
Ask your coding agent whether your app backs up the agent's memory often enough that you can rebuild it as it stood before the agent saved the false fact.
In the first comment, I've linked a guide to rebuilding the earlier memory beside the live one, then keeping the later entries that you still trust.
You can restore the memory only to the moment of one of its backups, so if your app backs up every minute, you can lose up to a minute of what the agent saved: sirannon.sondelali.com/docs/…
Should a privacy policy contain the AI provider's name, or is the phrase 'service providers' enough?
ALT A dark square card shows the sentence: With only 'service providers' in your privacy policy, your customers will have to guess whether an AI provider receives what they type into your app. The words 'will have to guess' are in blue.
The EU's data protection regulators accept a category of companies in place of each name, but in most cases they expect the names so that people know exactly who has their data: ec.europa.eu/newsroom/articl…
Which of these will you deploy your API's containers on, once the API needs more than one server?
→ Kubernetes
→ AWS ECS
→ Google Cloud Run
→ Something else
Giving your coding agent a password lets it send that password to any website that it visits.
An Accenture contractor's Claude Code agent appears to have sent a Pentagon login to a high-risk site, sources told WIRED, so give your agent keys with only the access that it needs.
WIRED's sources say that the agent worked on the Pentagon's network for two months, while Accenture says that an investigation found the allegations to be without merit: wired.com/story/a-defense-co…
A stranger can unlock what your customers pay for without paying, if your app skips Stripe's signature check on its webhook.
Stripe's documentation warns that without that check, 'an attacker could send fake webhook events to your endpoint to trigger actions like fulfilling orders, granting account access, or modifying records'.
Before your next release:
→ Ask your coding agent to check the signature on every webhook event with Stripe's official library before the app acts on the event.
→ On a preview build, have a script send your webhook address a fake 'payment succeeded' event with no signature, then check that the app rejects it and unlocks nothing.
ALT A three-step diagram on a dark square card shows a stranger finding the address that Stripe sends your payment events to, sending it a fake event that says a customer paid, and your app unlocking the purchase because it skips Stripe's signature check. The second step is in blue.
Which of these will you open to look up and correct a customer's record, once customers start writing in about mistakes in their accounts?
→ Supabase's table editor
→ Retool
→ An admin page that your coding agent builds
→ Something else
Searching your documents by meaning alone, as vector search does, can leave your AI agent without the page that holds a product code or acronym from the customer's question.
A search by meaning finds pages that say the same thing in other words, while a keyword search finds pages with the exact term, so each one can miss what the other finds.
Ask your coding agent whether your AI agent's search tool merges a keyword ranking with a vector ranking.
This guide shows how you can merge the two rankings and how adding your customers' own words to each document helps both searches: narsil.sondelali.com/docs/ag…
Is an app that wraps another company's AI model a business, or a feature that the company can add next?
ALT On a dark square card, the sentence 'Your customers pay you to solve their problem, so a company that copies your feature will usually have to win them one at a time' appears in large type, with 'will usually have to win them one at a time' in blue.