Engineering · Commission · Money

A rate saved in pieces

Commission bands were cleared and rewritten one at a time. Anything reading in between saw a rate with no bands, and charged a different number.


A market can charge its kitchens a commission that changes with turnover: ten per cent up to a point, eight above it, six above that. Those are bands, and somebody types them into a screen and presses save.

Saving them meant: update the header, delete every existing band, then insert the new bands one by one. Four separate writes, each one landing on its own.

The window

Between the delete and the last insert, the rate exists with no bands attached to it. Anything reading in that moment — the payout run, a statement, a charge being calculated — sees a plan with an empty band list.

And here is the part that makes it a money problem rather than a glitch: the code that reads it does not treat that as an error. If a plan has no bands, it falls back to the plan's flat percentage and charges that. Which is exactly right for a plan that genuinely is flat, and that is why it would never have complained. A commission charged at the wrong rate, by a function behaving precisely as designed, during a window a few hundred milliseconds wide.

The second fault is worse and quieter: if one of those inserts had failed, the rate stays that way permanently, with the header already committed.

Found before anyone could be charged by it

We want to be exact about the harm here, because overstating it would be its own kind of dishonesty. We checked the live database: five commission plans, every one of them flat, and no bands anywhere in the system. So no market has yet lost a band to this, and nobody has been charged the wrong amount. The fault was real, the exposure was not — yet. It would have become live the first time somebody used the screen for the thing the screen is for.

All four writes now go together, as one operation that either completely happens or completely does not. There is no moment when a rate exists without its bands.

How we went looking

We had fixed one case of this shape a few releases earlier — opening hours, deleted and rewritten the same way — and asked whether it was the only one. A search found nineteen places in the system that delete rows and insert rows into the same table. We read all nineteen by hand. Eighteen were fine, for four different reasons: some are tidy-ups rather than replacements, some write a single row, some build their statements for a caller that groups them properly already, and some had been fixed.

One was this. Nineteen candidates, eighteen of them innocent, is not a result you can automate — a check that cries wolf eighteen times in nineteen is a check people learn to skip. Some things have to be read.


Try it on tonight’s service.

Nothing to install, no card. Not better by the weekend? Close the tab.