inaccessible via ssh or http, but pings

[Lyttek]

hair-pulling problem:

4 machines in a rack all connected to an unmanaged switch. One feed line to the switch for upstream internet. All machines on same subnet.

All 4 machines running centos 4.4 with CSF firewalls.

Symptom: machine becomes unresponsive to http or ssh traffic; can still ping machine.

However, I can ssh into a responsive machine and then ssh into the 'unresponsive' machine. It's as if all ip traffic is blocked while allowing icmp traffic, but upstream of the switch.

 

[kkargel]

hmm.. can you say firewall? It sounds like there is some port blocking going on somewhere..

 

[Lyttek]

Well, that's the thing... it doesn't actually seem to be blocking the port, as I can access it from other machines... nothing is listed in csf.deny and even if it were, it would block all traffic from an IP address, not just all traffic except ICMP.

Also, it usually seems to strike two machines at the same time, and then they both clear up at the same time.

The datacenter isn't reporting any DDOS traffic...

Any ideas on testing for other protocols when it happens? UDP for instance?