Mistakenly ran cpan -u

[Spork Schivago]

Hi,

I was trying to update some modules and ran cpan -u. I didn't realize at the time that this was a bad idea. It failed, saying something about running out of memory.

Now, I get an e-mail and it's the same e-mail. It just says:

Can't locate Carp.pm in @INC (@INC contains: /usr/local/bandmin /usr/local/lib64/perl5 /usr/local/share/perl5 /usr/lib64/perl5/vendor_perl /usr/share/perl5/vendor_perl /usr/lib64/perl5 /usr/share/perl5 . /usr/local/bandmin) at /usr/local/bandmin/IP/Acct.pm line 16.
BEGIN failed--compilation aborted at /usr/local/bandmin/IP/Acct.pm line 16.
Compilation failed in require at /usr/local/bandmin/bandmin line 13.
BEGIN failed--compilation aborted at /usr/local/bandmin/bandmin line 13.


I think this is probably related to me running cpan -u. Does anyone know how I can undo that and restore the cpan modules that cPanel depends on? I've ran /scripts/upcp --force but that didn't fix the problem. Any help is greatly appreciated. Thank you!

 

[Spork Schivago]

I wonder if I should restore the server. I should have made a more recent backup but I've been a bit busy and just didn't have time. Wish I could find a way to back it up using my Linux server. I tried using rsync but that just doesn't seem to work properly. I had to connect many, many times. It just kept on hanging and getting disconnected. Then, there were a lot of things it didn't back up.

I just noticed /scripts/upcp --force fails a bunch with the same error message. It really seems like I messed things up here.

 

[cPanelMichael]

Hello,

You should not have to restore the server or reinstall cPanel. Try running the following command to see if it helps:

/scripts/perlinstaller Task::Cpanel::Core

You can also open a support ticket using the link in my signature if you want us to take a closer look. Ensure you post the ticket number here should you decide to do so.

You may find the following documents helpful should you want to learn more about using Perl in cPanel and WHM:

Guide to Perl in cPanel - Software Development Kit - cPanel Documentation
Guide to Perl in cPanel - Perl Environments - Software Development Kit - cPanel Documentation

Thank you.

 

[Spork Schivago]

Hello,

You should not have to restore the server or reinstall cPanel. Try running the following command to see if it helps:

/scripts/perlinstaller Task::Cpanel::Core

You can also open a support ticket using the link in my signature if you want us to take a closer look. Ensure you post the ticket number here should you decide to do so.

You may find the following documents helpful should you want to learn more about using Perl in cPanel and WHM:

Guide to Perl in cPanel - Software Development Kit - cPanel Documentation
Guide to Perl in cPanel - Perl Environments - Software Development Kit - cPanel Documentation

Thank you.

Unfortunately, that command failed. I've now started a system restore. I've backed up important stuff and there isn't much there, so afterwards, I'll update and try to fix the problem that cPanel's support fixed for me the other day.

This is what the command failed with (after about 45 minutes or so of running):

Successfully installed Encode-Detect-1.01
Installing /home/sporkschivago/perl5/lib/perl5/x86_64-linux-thread-multi/.meta/Encode-Detect-1.01/install.json
Installing /home/sporkschivago/perl5/lib/perl5/x86_64-linux-thread-multi/.meta/Encode-Detect-1.01/MYMETA.json
! Installing the dependencies failed: Module 'String::CRC32' is not installed, Module 'Parse::RecDescent' is not installed, Module 'Spreadsheet::WriteExcel' is not installed
! Bailing out the installation for Task-Cpanel-Core-11.36.004.
Expiring 3 work directories.
164 distributions installed
Perl Expect failed with non-zero exit status: 256

All available perl module install methods have failed

 

[cPanelMichael]

Unfortunately, that command failed. I've now started a system restore.

Note a restore is generally not required to resolve this type of issue. Feel free to update us with the outcome when the restore process is complete.

Thank you.

 

[Spork Schivago]

Note a restore is generally not required to resolve this type of issue. Feel free to update us with the outcome when the restore process is complete.

Thank you.

Thank you cPanelMichael.

I have trouble with my memory and because of this restore, I remember now backups / restores don't work. I think csf / lfd have been blocking GoDaddy. I was able to get a list of IPs that I should whitelist. So I guess at least something good came from this.

Out of curiosity, any idea why the command you gave me failed? Also, just so we're clear, I shouldn't ever run cpan -u to update the cpan modules, right? Because cPanel depends on certain ones and updating them all might break cPanel, is that right? Thanks!

 

[cPanelMichael]

Out of curiosity, any idea why the command you gave me failed? Also, just so we're clear, I shouldn't ever run cpan -u to update the cpan modules, right? Because cPanel depends on certain ones and updating them all might break cPanel, is that right?

Hello,

It's possible the system Perl binary is broken on your system. Feel free to open a support ticket if the issue continues and we can take a closer look. Generally, running "cpan -u" should not break the system. There's a document here that explains the different Perl environments on a cPanel server:

Guide to Perl in cPanel - Perl Environments - Software Development Kit - cPanel Documentation

Thank you.

 

[Spork Schivago]

Thanks! I did a system restore and upgraded to the next tier so now I got 2GB of RAM and 60GB hard drive space. So, I'm updating cPanel, CSF, and fixing the stuff cPanel support fixed a few days ago for me (a permissions problem). I get an e-mail that says:

lfd on franklin.JetBBS.com: WHM/cPanel root access alert from 184.168.224.94 (US/United States/p3plvertigo01.prod.phx3.secureserver.net)

Time:    Tue Jul 12 21:54:21 2016 -0400
IP:      184.168.224.94 (US/United States/p3plvertigo01.prod.phx3.secureserver.net)
User:    root

The bold is the subject line. This stuff always worries me. Earlier, I asked GoDaddy for a list of IPs that they owned that I should whitelist so csf / lfd doesn't block them accidently. That IP address isn't in the list. Does that mean someone hacked into my server at 21:54 today? It doesn't say invalid login, it just says root access alert. I can't see why this wouldn't be in the addresses that I should whitelist. I believe this was right when I refreshed the host stuff on GoDaddy's site. This is probably just their GUI accessing my domain, right? I wish they'd give us more info on this kind of stuff though. Last time I saw something like this, I blocked it and then backups broke. What do you guys think? Ignore it or try to investigate further?

 

[Spork Schivago]

I searched the net and it does seem that IP address is for the backup server. I've contacted GoDaddy again asking for a complete list of IPs that should be whitelisted. The restore went well but upgrading the software to the newest stuff, not so well...

I'm going through the logfile and I see a bunch of these. I didn't copy them all nor did I copy the whole line because of how much was there. I can provide more info if you need:

 [Tue Jul 12 22:30:06.006124 2016] [core:notice] [pid 20989] AH00094: Command line: '/usr/local/apache/bin/httpd'
        [Tue Jul 12 22:30:06.006107 2016] [mpm_prefork:notice] [pid 20989] AH00163: Apache/2.4.18 (Unix) OpenSSL/1.0.1e-fips mod_bwlimited/1.4 configured -- resuming normal operations
        [Tue Jul 12 22:30:01.213601 2016] [:error] [pid 30524] [client 127.0.0.1] ModSecurity: Warning. Operator GE matched 5 at TX:inbound_anomaly_score. [file "/usr/local/apache/conf/modse$
        [Tue Jul 12 22:30:01.211800 2016] [:error] [pid 30524] [client 127.0.0.1] ModSecurity: Warning. Operator EQ matched 0 at REQUEST_HEADERS. [file "/usr/local/apache/conf/modsec_vendor_$
        [Tue Jul 12 22:30:01.211660 2016] [:error] [pid 30524] [client 127.0.0.1] ModSecurity: Warning. Operator EQ matched 0 at REQUEST_HEADERS. [file "/usr/local/apache/conf/modsec_vendor_$
        [Tue Jul 12 22:29:37.580481 2016] [:error] [pid 30523] [client 127.0.0.1] ModSecurity: Warning. Operator GE matched 5 at TX:inbound_anomaly_score. [file "/usr/local/apache/conf/modse$
        [Tue Jul 12 22:29:37.579262 2016] [:error] [pid 30523] [client 127.0.0.1] ModSecurity: Warning. Operator EQ matched 0 at REQUEST_HEADERS. [file "/usr/local/apache/conf/modsec_vendor_$
        [Tue Jul 12 22:29:37.579114 2016] [:error] [pid 30523] [client 127.0.0.1] ModSecurity: Warning. Operator EQ matched 0 at REQUEST_HEADERS. [file "/usr/local/apache/conf/modsec_vendor_$
        [Tue Jul 12 22:28:51.004921 2016] [core:notice] [pid 20989] AH00094: Command line: '/usr/local/apache/bin/httpd'
        [Tue Jul 12 22:28:51.004901 2016] [mpm_prefork:notice] [pid 20989] AH00163: Apache/2.4.18 (Unix) OpenSSL/1.0.1e-fips mod_bwlimited/1.4 configured -- resuming normal operations
        [Tue Jul 12 22:28:36.425743 2016] [:error] [pid 25397] [client 127.0.0.1] ModSecurity: Warning. Operator GE matched 5 at TX:inbound_anomaly_score. [file "/usr/local/apache/conf/modse$
        [Tue Jul 12 22:28:36.425090 2016] [:error] [pid 25397] [client 127.0.0.1] ModSecurity: Warning. Operator EQ matched 0 at REQUEST_HEADERS. [file "/usr/local/apache/conf/modsec_vendor_$
        [Tue Jul 12 22:28:36.425005 2016] [:error] [pid 25397] [client 127.0.0.1] ModSecurity: Warning. Operator EQ matched 0 at REQUEST_HEADERS. [file "/usr/local/apache/conf/modsec_vendor_$
        [Tue Jul 12 22:25:01.323559 2016] [:error] [pid 25401] [client 127.0.0.1] ModSecurity: Warning. Operator GE matched 5 at TX:inbound_anomaly_score. [file "/usr/local/apache/conf/modse$
        [Tue Jul 12 22:25:01.322356 2016] [:error] [pid 25401] [client 127.0.0.1] ModSecurity: Warning. Operator EQ matched 0 at REQUEST_HEADERS. [file "/usr/local/apache/conf/modsec_vendor_$
        [Tue Jul 12 22:25:01.322149 2016] [:error] [pid 25401] [client 127.0.0.1] ModSecurity: Warning. Operator EQ matched 0 at REQUEST_HEADERS. [file "/usr/local/apache/conf/modsec_vendor_$
        [Tue Jul 12 22:23:46.468467 2016] [:error] [pid 25400] [client 127.0.0.1] ModSecurity: Warning. Operator GE matched 5 at TX:inbound_anomaly_score. [file "/usr/local/apache/conf/modse$
        [Tue Jul 12 22:23:46.467704 2016] [:error] [pid 25400] [client 127.0.0.1] ModSecurity: Warning. Operator EQ matched 0 at REQUEST_HEADERS. [file "/usr/local/apache/conf/modsec_vendor_$
        [Tue Jul 12 22:23:46.467583 2016] [:error] [pid 25400] [client 127.0.0.1] ModSecurity: Warning. Operator EQ matched 0 at REQUEST_HEADERS. [file "/usr/local/apache/conf/modsec_vendor_$
        [Tue Jul 12 22:21:14.347852 2016] [:error] [pid 25399] [client 180.76.15.137] ModSecurity: Audit log: Failed to create subdirectories: /usr/local/apache/logs/audit/myusername/2016$
        [Tue Jul 12 22:21:14.346867 2016] [:error] [pid 25399] [client 180.76.15.137] ModSecurity: Warning. Match of "pm AppleWebKit Android" against "REQUEST_HEADERS:User-Agent" required. [$
        [Tue Jul 12 22:20:54.099624 2016] [:error] [pid 25398] [client 180.76.15.26] ModSecurity: Audit log: Failed to create subdirectories: /usr/local/apache/logs/audit/sporkschivago/20160$
        [Tue Jul 12 22:20:54.098777 2016] [:error] [pid 25398] [client 180.76.15.26] ModSecurity: Warning. Match of "pm AppleWebKit Android" against "REQUEST_HEADERS:User-Agent" required. [f$
        [Tue Jul 12 22:20:01.713270 2016] [:error] [pid 25400] [client 127.0.0.1] ModSecurity: Warning. Operator GE matched 5 at TX:inbound_anomaly_score. [file "/usr/local/apache/conf/modse$
        [Tue Jul 12 22:20:01.712056 2016] [:error] [pid 25400] [client 127.0.0.1] ModSecurity: Warning. Operator EQ matched 0 at REQUEST_HEADERS. [file "/usr/local/apache/conf/modsec_vendor_$
        [Tue Jul 12 22:20:01.711919 2016] [:error] [pid 25400] [client 127.0.0.1] ModSecurity: Warning. Operator EQ matched 0 at REQUEST_HEADERS. [file "/usr/local/apache/conf/modsec_vendor_$
        [Tue Jul 12 22:18:45.481452 2016] [:error] [pid 25399] [client 127.0.0.1] ModSecurity: Warning. Operator GE matched 5 at TX:inbound_anomaly_score. [file "/usr/local/apache/conf/modse$
        [Tue Jul 12 22:18:45.477300 2016] [:error] [pid 25399] [client 127.0.0.1] ModSecurity: Warning. Operator EQ matched 0 at REQUEST_HEADERS. [file "/usr/local/apache/conf/modsec_vendor_$
        [Tue Jul 12 22:18:45.477176 2016] [:error] [pid 25399] [client 127.0.0.1] ModSecurity: Warning. Operator EQ matched 0 at REQUEST_HEADERS. [file "/usr/local/apache/conf/modsec_vendor_$
        [Tue Jul 12 22:13:41.671789 2016] [:error] [pid 25398] [client 127.0.0.1] ModSecurity: Warning. Match of "pm AppleWebKit Android" against "REQUEST_HEADERS:User-Agent" required. [file$

This is the Apache log I believe and was displayed when Apache was restarted. How can I fix these dang warnings? Another thing, what's the best way to have stuff setup you think? I believe I got something setup, like suphp or something, I dunno which, but it creates log files, a lot of them, in directories like this:

[Tue Jul 12 22:21:14.347852 2016] [:error] [pid 25399] [client 180.76.15.137] ModSecurity: Audit log: Failed to create subdirectories: /usr/local/apache/logs/audit/myusername/20160712/20160712-2221 (Permission denied) [hostname "jetbbs.com"] [uri "/"] [unique_id "V4WlmmjudWkAAGM3uJEAAAAD"]

Before when I had it running, that myusername directory would be filled with a ton of subdirectories, all with the same date / time format for their name. It made searching for problems real hard. I wish I knew more about this stuff so I could set it up so all those subdirectory log files just went to one file, like /usr/local/apache/logs/audit/myusername/big_log_file

 

[Spork Schivago]

cPanelMichael,

I think I got my server back up and running to the way it was again. I'd like to try running cpan -u again but I wanted to know if you knew how I could make a backup of all the modules first, so if something goes wrong again, I can just simply restore a directory with the modules or something. Any suggestions? Thanks!

 

[cPanelMichael]

I think I got my server back up and running to the way it was again. I'd like to try running cpan -u again but I wanted to know if you knew how I could make a backup of all the modules first, so if something goes wrong again, I can just simply restore a directory with the modules or something. Any suggestions? Thanks!

Could you let us know the specific reason why you want to run this command so we can let you know if it's the right command to run for that particular situation?

Thank you.

 

[Spork Schivago]

Could you let us know the specific reason why you want to run this command so we can let you know if it's the right command to run for that particular situation?

Thank you.

Yup,

When I worked at my last corporation, we got hacked. It could have been a lot worse. It was just a defacement but if the hackers had realized they had access to servers with many credit card numbers, social security numbers, names and addresses to match, along with monthly and annual salaries, things would have been much, much worse. We shut down everything and started investigating. The hackers got in because we didn't keep our servers updated. The main IT guy felt if it wasn't broke, don't fix it. There were services that were running that were exploitable. Patches were made the same day the exploits were found, years ago, we just never updated.

I see people trying to get into my server fairly often for some reason. I run csf / lfd. I get sometimes 3 e-mails saying people were blocked for doing bad stuff, on a bad day, maybe 30 e-mails. Some of them are just port scans and might not actually be attacks. However, my server detects known security scanners and on a regular basis, my server is scanned by these from unknown attackers in other countries.

I noticed a lot of these modules had new versions. Even if known exploits don't exist yet, I just like to keep all my stuff up-to-date. I wanted to create some crontab entry that would just update it on a daily basis or something. I felt there's no real need to have out-dated software on there. If there's a newer version, why not update it to the latest and just keep it updated?

 

[Spork Schivago]

I should add it wasn't just one exploit that they used. They used some remote buffer overflow to get into the system, but that didn't give them a root shell, then they actually used another exploit once they were in the system to get a root shell! We had a bunch of exploitable stuff on the server, it was horrible. It was all automated. It was some college in some other country, so the FBI couldn't do much to go after them because our country didn't have diplomatic rights with them or something like that.

They had this automated system (the hackers) that just crawled the internet and scanned sites. Then, it'd create some database of exploitable servers and automatically try to get in. It was some hacking contest. We know this because they uploaded a program onto the server that communicated back with their servers. We did some research on the IP address it was connecting back to. They had this list on their server of successful hacks, what was done, and how long it took the server people to realize they were hacked. They had awards for most number of servers hacked and then one for the longest time the hack was in place without being noticed. I was the one who discovered the defacement but only because I was writing some software that connected to our website. I noticed it maybe 2 hours after we were hacked. If it wasn't for that, we probably wouldn't have known until a customer contacted us, which would have been real bad, because we did business with big banks.

So, the only reason I want to run that command is to update the various Comprehensive Perl Archive Network modules to the latest version, just to get the latest bug fixes, etc. I have made a new backup of the server and I'm tempted to just run the cpan -u command again, if it shouldn't affect cPanel at all. I have more RAM now so I don't think I'll run out like before.

Thanks!

 

[cPanelMichael]

Hello,

You may also find this StackOverflow thread helpful:

How do I update all my CPAN modules to their latest versions?

Thank you.