Sites 2,3 and 4 showed dropouts last night around 09:15 UTC (02:15 MST). Data from 3 and 4 resumed about 1/2 hour later, site 2 is still out.
As of this morning we're still not getting data from 2. A host at the IP address returned for isfs2.dyndns.org responds to pings, but reports "connection refused" when ssh-ing to the special port we use. It probably isn't our system at that address.
From the system log (/var/log/isfs/messages) at site 3, the router_check.sh script detected at 09:32 UTC that it couldn't ping eol or google, and successfully power cycled the router and modem. This happened just once. Things worked like they should!
Site 4 is more complicated. The cellular connection is frequently going up and down - several times an hour, which has been going on for the last serveral days. The router detects the connection problem itself and resets the modem, the router_check.sh script is not powering things down.
I think we should try an external antenna at site 4.