Top shelf would be noticing an anomaly in behavior for a node and then interrogating it to see what’s wrong.

Automatic load balancing always gets weird, because it can end up sending more traffic to the sick server instead of less, because the results come back faster. So you have to be careful with status codes.