Ask your reports how many rows they read

We spent four days going through every report in our own product asking one question of each number on it: how many rows did this actually read? It turned up five wrong figures. This post is what we learned, written for anybody choosing or living with hospitality software rather than building it.

The shape

Almost every report screen shows two things: a headline figure, and the rows behind it. The rows need a limit or the page takes ten seconds to draw and nobody uses it. That limit is correct, sensible, and usually years old.

Then, later, somebody needs the headline figure. The rows are already fetched, sitting right there. Adding them up is one line. And the report now quietly answers a different question from the one on the label — not "what did we take today" but "what did we take in the most recent five hundred orders of today".

The tell is that both halves are individually right. Capping the list is right. The arithmetic is right. Only the join between them is wrong, and no review of either half finds it.

Why hospitality is where it hides best

Three reasons, and they compound.

The figure is right until you are successful. At 200 orders a day a 500-row cap never bites, so the screen is genuinely correct for the whole period you are learning to trust it. The first day it lies is a good day.

Every figure is plausible. Takings vary enormously by day and weather. There is no number a cash-up screen could show you that would make you stop and say that cannot be right, which is exactly the check that catches ordinary bugs.

The error is one-directional. A truncated sum is always lower than the truth. It cannot make a good day look bad, only a bad one look fine — so it never triggers the investigation that would find it.

More things are totals than you think

This was the useful realisation. It is not only sums. Every one of these is a total wearing a different hat, and every one breaks the same way when fed a sample:

• A "worst of". Show it the newest fifty counts and it names the worst of fifty, not the worst there was.
• A "which are missing". Anything it cannot see counts as missing — which is how a settled year turns into an unpaid one.
• A breakdown by group. Whole groups disappear rather than showing wrong figures, and the quiet ones go first.
• A count of things needing attention. The badge that says "3" when it is 3 of 40.

Four questions to ask of any system

1. Pick your busiest day and count something by hand. Cash orders on one till. Compare it with what the report says the total was built from. This takes ten minutes and it is the whole test.
2. Look for round numbers. A list that stops at exactly 100, 200 or 500 rows has a cap. That is fine. The question is whether the headline above it came off the same fetch.
3. Does anything ever say "showing X of Y"? A system that never says it either never truncates or never admits it.
4. Widen the date range and watch what changes. Ask for a year instead of a month. If a total goes up by less than the underlying volume did, or if a name disappears from a breakdown, you have found one.

What we did about ours

Every calculation now reads what it needs completely — usually by counting in the database rather than by fetching rows, which is both correct and faster than the sample it replaced. Lists stay capped, because a person is not reading 620 rows. And where a list is capped, the screen says so: showing 500 of 620, newest 50 of 89.

There is also a test that reads the report code itself and fails the build if a total is ever wired to a capped fetch again — because the honest lesson from four days of this is that the mistake is natural. It is what a reasonable person writes when the rows are already in front of them. It will be written again, and something other than good intentions has to catch it.

What we do and do not claim is set out plainly on the about page, and what any of it costs is on pricing.