Gaurav

everyone shouting 'idempotency keys' is failing the actual MTS interview question—you specifically asked what happens when the external API DOES NOT support idempotency the 3 distributed systems patterns for surviving agent timeouts against non-idempotent legacy APIs: 1. the fundamental rule: timeout is an unknown in-flight state: a timeout is the Two Generals' problem: the TCP packet could have failed on ingress, OR the bank/vendor executed the mutation and only the response dropped if an agent blindly retries a non-idempotent POST, you guarantee duplicate side effects (double charges, duplicate records) the state machine must transition to IN_DOUBT, never straight to RETRY 2. deterministic read-before-retry (synthetic idempotency probes): even if the vendor has no Idempotency-Key header, their payload usually accepts custom metadata or reference fields embed a deterministic token into the request: reference = hash(agent_id, task_id, step_nonce) on timeout, the agent NEVER retries the POST directly—it issues an out-of-band probe: GET /transactions?reference=REF_42 - if the record exists: adopt the remote ID, mark succeeded, proceed - if 404 after an established quiet window: only then is it safe to retry 3. the Saga pattern & asynchronous reconciliation queues: if the vendor has neither idempotency nor deterministic reference queries, synchronous resolution is mathematically impossible the agent must persist the intent to a dead-letter / reconciliation outbox and yield an out-of-band worker attempts reconciliation or issues a compensating reversal (void / cancel) before unlocking the agent's workflow the true MTS architecture answer: never let agents call raw un-idempotent third-party APIs directly. you place an internal egress proxy in front that shims synthetic idempotency and transaction locks