Notes  / 

When sixty devices fail together, it was your monitoring

An availability number is only as honest as the rules behind it. Here are the rules on a dashboard that sits on a monitoring-centre wall, and why each exists.

A client dashboard shows availability per station, per camera group, per microwave link. Those percentages end up in a monthly report and, eventually, in a conversation about an SLA. So the rules behind them are written in the code, printed on the page, and short enough to argue about. These are the current ones.

  • Down means ten minutes. One missed poll is not an outage.
  • Mass failure is an artefact. If sixty or more devices fail in the same five-minute slot and recover within fifteen minutes, the monitoring blinked, not the network. Excluded from availability, visible to staff in the raw view.
  • Never polled means unverified. A device the poller has never reached is not down, and it is not up. It is excluded until it has been seen once.
  • A category with nothing up and no polls for an hour is unknown. Its time is not counted at all, in either direction.
  • Three station switches within fifteen minutes is one event. It is reported as a shared upstream fault, not three site faults.
  • Down for more than thirty days is a different list. It is folded off the overview so that the overview still means something.

Each rule came from a specific morning where the number was wrong in a specific way. The point is not the thresholds, which will move. The point is that the thresholds are visible, so when the number is questioned the conversation is about the rule, not about trust.

Availability   Dashboards   LibreNMS   · Terawiz engineering notes