A deadline that turns "maybe" into "no"
Put a timeout on every outbound call is standard advice, and it is wrong for about half of them. On a payment or a text message, a deadline converts "we do not know" into "it failed" — and the retry does the damage.
Having found one request with no time limit on it, we went looking for the others. Twenty-seven places in our system talk to somebody else's computer. Eighteen of them had no deadline at all.
The obvious move is to add one everywhere and call it a morning. We added seven and deliberately left eleven, and the reason is worth more than the fix.
A deadline is an opinion about an unknown
When you give up waiting, you have not learned that the thing failed. You have learned that you do not know. Those are different, and a program has to turn one into the other to carry on.
For some calls that is free. Asking a provider "is this key valid" has no consequence — give up, say "could not reach them", and a person presses the button again. Asking for an access token is the same: getting one twice costs nothing.
For others it is the whole ballgame. If a card terminal has taken the money and is slow to say so, a deadline records a failed sale. If the caller then retries — and retries are exactly what failure handling is for — you have charged somebody twice. Same with a text message: the provider has queued it, your customer will receive it, and your system has decided it did not send and sends another.
Where we drew the line
Bounded, because giving up cannot cost anything: the four checks that ask a provider whether a key works, the nightly job that checks whether a hardware link still resolves, and the two calls that fetch an access token.
Left alone, with the reason written next to each one rather than a note to do it later: the card adapter, the payments API, the two message paths, email, push notifications, the two AI calls, and one shared helper that does reads and writes through the same door.
Those last ones are not a backlog. They need a design, not a number. The card one, for instance, needs to read the sale back before it believes its own timeout — and a card sale legitimately waits on a human finding their bank card, so the budget is tens of seconds, not three.
The question worth asking your supplier
Not "do you have timeouts". Everybody says yes. Ask what happens on their side when a payment request times out: do they report a failure, or do they go and look? The difference shows up in your bank account, on a Saturday, once.
For the avoidance of doubt on our own side: HelchPOS does not process card payments. Card is recorded on the till after being taken on a separate terminal. The code discussed here is built and unproven, and the site says so everywhere else too.