Blocked is not dead, and it is not ok either
Our nightly check of the shop links called three working links dead and one bot wall fine. A status code has two answers. The question needed a third.
Our shop page links to twenty-one pieces of hardware on a retailer’s site: printers, tablets, a label machine. A link can rot. The retailer can move a path or block a region, and a shop page full of dead buttons is worse than one with fewer items. So a job checks one link a night, round-robin, and writes a row saying what it found.
For a long time that row was a coin toss. The check asked one question of the response, whether the status code was an error, and a status code cannot answer the question being asked. Measured against the live links one afternoon, four for four:
- Three printer links answered 503 with an identical 1,427-byte page. The job called all three dead. Every one of them opens perfectly in a browser.
- The tablet link answered 200 with a 2,288-byte page. The job called it fine. The page was a bot wall asking us to prove we were human, and it contained no product at all.
The retailer throttles a server-side fetch with a 503 and challenges it with a 200. Read as a status code, the first is a failure and the second is a pass, and both readings are wrong.
A third word, not a better threshold
The fix is not a smarter rule about which codes count as errors. It is a third verdict. We were not allowed to look is not a pass and it is not a failure. It is no information, and saying so is the only thing that makes the other two words mean anything.
So the check now answers one of three ways. No answer, a 429 or a 503 is blocked. Any other error is dead. A 200 whose body carries the fingerprints of a bot wall is blocked too, and only a 200 with none of them is ok. The row stores the word, and the log line is built from the same word so the two cannot drift.
What it changes for the person reading the log
The question is any shop link dead? is now answered by looking for the word dead, never by a non-200 status. A run of blocked rows means the retailer would not talk to a server that night, which is their choice and tells us nothing about the links. The prices on that page are a separate matter: they carry the date a person last checked them, because no job of ours can read a price through a bot wall, and refreshing the date is not checking.