If your hardware is local to you then you can turn off ssh. Its only used to get a "console" on the system. We have all of our machines in-house so we do not allow SSH for anyone or anything unless the Cpanel team wants to get into our boxes for some tech support.
You should never turn off SSH as it is the only way your system can be repaired if it has problems.
However, you can do something very simple by moving the SSH port from 22 to another, higher, port which prevents many of the attacks. You would then use that higher port to access SSH if necessary.