Cron Job Error messages suddenly started

magicalwonders

Well-Known Member
Nov 21, 2012
112
2
18
cPanel Access Level
Root Administrator
I hope someone can help me with a cron problem I'm getting.

I have a script that requires three cron jobs which I've been running since 2008 without problems. In 2012 I switched to VPS managed hosting and in July 2013 I installed the latest version of script.

On Monday night this week I started receiving emails reporting the following error each time one of the cron jobs runs -

Error, do this: mount -t proc none /proc
The following command for cron was set to run on the hour and at 25 minute intervals. The error is produced each time it runs.

*/25 * * * * /usr/bin/perl /home/mysite/public_html/cgi-bin/arp3/arp3-popreader.pl 1>&2 > /dev/null
My hosting support have not been able to help and want to point the finger at the script. My position is that the script has remained unchanged since 31 July 2013, and has worked without problem until this week.

My server is running CENTOS 5.10 x86_64 virtuozzo / WHM 11.40.1 (build 10)

The command line for the cron works fine when executed from a browser, so I know my script is working and produces the expected output.

The other two crons for this script are not generating any error. I've tried deleting cron job, rebooting server and re-establishing cron job, plus changing cron times, but the errors messages keep coming!

The function of the script is to empty the "catch all" mailbox of my autoresponder whenever the script is run.

Here's a copy of it. I've blocked out my mailbox password, but that is the only change I've made.


Code:
#!/usr/bin/perl
 
###################################################
# AutoResponse Plus (tm)                          #
# Copyright ECom24 Ltd 2000 - 2013                #
# All rights reserved                             #
# autoresponseplus.com                 #
###################################################

$ARP3_CGI_PATH = "/home/magicalw/public_html/cgi-bin/arp3";

$mailHost = "mail.magicalwonders.com";
# On a CPanel server this will be mail.yourdomain.com

$mailUser = "arplus\@magicalwonders.com";
# Remember \@ if this is an email address
# Example: you\@yourdomain.com
# On a CPanel server this will be whatever+yourdomain.com

$mailPassword = "xxxxxxxxxxx";

#######################################
# CHANGE NOTHING BELOW THIS LINE      #
#######################################
print "Content-type: text/html\n\n";

exit if scalar(split "\n",`ps | grep 'arp3-popreader.pl'`) > 1;

use Net::POP3;
$pop = Net::POP3->new($mailHost);
if ($pop) {
    $lastDate = "00000000000000";
    if (open (LOG, "<$ARP3_CGI_PATH/temp/pop.last")) {
        $lastDate = <LOG>;
        close(LOG);
    }
    %months = (Jan=>"01",Feb=>"02",Mar=>"03",Apr=>"04",May=>"05",Jun=>"06",Jul=>"07",Aug=>"08",Sep=>"09",Oct=>"10",Nov=>"11",Dec=>"12");
    @dozen = ("00", "01", "02", "03", "04", "05", "06", "07", "08", "09", "10");
    $count = 0;
    $msgs_number = $pop->login($mailUser, $mailPassword);
    for ($msg = $msgs_number; $msg > 0; --$msg) {
        $header = $pop->top($msg);
        $currDate = "00000000000000";
        foreach $line (@{$header}) {
            if ($line =~ /(?:(?:Mon|Tue|Wed|Thu|Fri|Sat|Sun),)? (\d+) (\w+) (\d+) (\d\d):(\d\d):(\d\d)/) {
                $d = $1;
                $d = $dozen[$d] if (10 > $d);
                $m = $months{$2};
                $currDate = "$3$m$d$4$5$6";
                last;
            }
        }
        if ($currDate > $lastDate) {
            if (!$count++ && open (LOG, ">$ARP3_CGI_PATH/temp/pop.last")) {
                print LOG $currDate;
                close(LOG);
            }
            open (CAPTURE, "|/usr/bin/perl $ARP3_CGI_PATH/arp3-emailcapture.pl");
            $pop->get($msg, *CAPTURE);
            $pop->delete($msg);
            close(CAPTURE);
        } else {
            last;
        }
    }
    $pop->quit();
    print "Count: $count\n";
}
I'm a bit lost as to how the script can be suddenly causing the issue. It's been doing it's job for a long time without a problem.

I was hoping that the error message "Error, do this: mount -t proc none /proc " might actually mean something useful and point towards what the problem is?

A search on the forum regarding cron errors revealed a thread that mentions an error log at /var/log/cron
I've downloaded this, and attached a copy to this thread, but I can't see anything that explains the problem.
Having said that, I don't really know what I'm looking for! It's not really my field of expertise!

If anyone could throw any light on this, or what my next course of action might be, I'd be really grateful. In words suitable for a newbie please! :)

Many thanks,

Myles
 

Attachments

cPanelMichael

Administrator
Staff member
Apr 11, 2011
47,913
2,205
363
Hello :)

Do you experience any difference in behavior if you grant full shell access to the account instead of jail shell or no shell access?

Thank you.
 

magicalwonders

Well-Known Member
Nov 21, 2012
112
2
18
cPanel Access Level
Root Administrator
Hello :)

Do you experience any difference in behavior if you grant full shell access to the account instead of jail shell or no shell access?

Thank you.
Ah, you've kind of lost me with that question. Although I've got VPS hosting, I purchased managed hosting, in the hope that someone else would have that expertise. I really am a newbie with how this stuff works!

Is this something I can turn on or off in my WHM? I have full root access, so prepared to give it a go. If it's something that needs entering at a command prompt, I may need some extra advice! :)
 

cPanelMichael

Administrator
Staff member
Apr 11, 2011
47,913
2,205
363
You can manage shell access for an account via:

"WHM Home » Account Functions » Manage Shell Access"

The reason I mentioned this is because since cPanel version 11.38, there are changes to how cron jobs function when assigned jail shell or no shell. It's documented here:

VirtFS (Jailed Shell)

Thank you.
 

magicalwonders

Well-Known Member
Nov 21, 2012
112
2
18
cPanel Access Level
Root Administrator
You can manage shell access for an account via:

"WHM Home » Account Functions » Manage Shell Access"

The reason I mentioned this is because since cPanel version 11.38, there are changes to how cron jobs function when assigned jail shell or no shell. It's documented here:

VirtFS (Jailed Shell)

Thank you.
OK thanks, I'll take a look at that, and experiment a bit!

At the moment I've deleted the cron job as the error messages were starting to drive me crazy! Lol. I'm going to put it back on without the /dev/null and see what email that produces. Then I'll play with shell access. :)

Thanks for the advice. :)
 

magicalwonders

Well-Known Member
Nov 21, 2012
112
2
18
cPanel Access Level
Root Administrator
I tried running the cron with email output, and am now getting the following emails -

Content-type: text/html

Error, do this: mount -t proc none /proc
Count: 1
I've looked at shell access, and all my accounts are currently set to shell disabled. If I select Normal Shell or Jailed Shell, I get the following warning -

Package conflict: Account is on package webhost1_Alpha
[Set this account to have no package] [Keep this account on its package (Not recommended)
Not sure how to proceed? If I select "Set account to have no package" I assume there will be no disk quota or bandwidth retstraints?
 

cPanelMichael

Administrator
Staff member
Apr 11, 2011
47,913
2,205
363
That warning appears because you are modifying a setting for the account that differs from the setting applied to it's package. Since this is only a temporary change to see if it makes a difference in the error you are receiving, you can select "Keep this account on its package". Remember to change the value back to no shell access after you have tested. Note that changing it to "Jail Shel" would not result in different behavior compared to no shell access.

Thank you.
 

magicalwonders

Well-Known Member
Nov 21, 2012
112
2
18
cPanel Access Level
Root Administrator
That warning appears because you are modifying a setting for the account that differs from the setting applied to it's package. Since this is only a temporary change to see if it makes a difference in the error you are receiving, you can select "Keep this account on its package". Remember to change the value back to no shell access after you have tested. Note that changing it to "Jail Shel" would not result in different behavior compared to no shell access.

Thank you.
Good news...... I think!

When I changed the shell access to "normal shell" for that account, the cron output produces no error -

Content-type: text/html

Count: 0
Now I suppose I have to change the shell access back to "No shell" which I guess will bring back the error? Any advice on what I need to do next to fix the issue?

Many thanks,

Myles
 

cPanelMichael

Administrator
Staff member
Apr 11, 2011
47,913
2,205
363
You could either leave regular shell access enabled for the account (some users prefer to leave it disabled for security purposes). Or, you could browse to "WHM Home » Server Configuration » Tweak Settings" and enable the following value under the "System" tab:

"Jailed /usr/bin mounted suid"

I believe this is the correct option based on the script you provided, but let us know if it does not work. Note that this can also decrease security, so it's a matter of preference if you decide to grant shell access to the individual account or enable the above option for all accounts.

Thank you.
 

magicalwonders

Well-Known Member
Nov 21, 2012
112
2
18
cPanel Access Level
Root Administrator
Thanks for the assistance with this. I've made the changes which produced the following message on "save".

Your changes have been saved.

Restarting cPanel daemons...done.

Updating your system to reflect any changes...

While trying to rectify your configuration for autodiscover_proxy_subdomains an error was encountered:
Updating Jailed /usr/bin mounted suid from Off to On.
Jailed /usr/bin mounted suid was updated.
Updating Jailed /proc mount method from Mount limited /proc (RHEL/CentOS 6)+, Full /proc (RHEL/CentOS 5/xenpv) to Mount limited /proc (RHEL/CentOS 6)+, No /proc (RHEL/CentOS 5/xenpv).
Jailed /proc mount method was updated.

Done.
Unfortunately, it doesn't seem to have impacted on the errors, which are still being generated when the cron runs.
I'm guessing I didn't need to reboot for changes to take effect?

Anything else I might try?

Myles
 

cPanelMichael

Administrator
Staff member
Apr 11, 2011
47,913
2,205
363
Feel free to open a support ticket so we can take a closer look and determine which setting would work in this particular instance. You can post the ticket number here so we can update this thread with the outcome.

Thank you.
 

magicalwonders

Well-Known Member
Nov 21, 2012
112
2
18
cPanel Access Level
Root Administrator
OK, I've created a ticket for this problem. ID 4543413

I've added the SSH access for the IP addresses provided and installed SSH key on server, and authorised it.

Not really sure how to adjust firewall settings to allow remote connections?
 

magicalwonders

Well-Known Member
Nov 21, 2012
112
2
18
cPanel Access Level
Root Administrator
What firewall are you running?
I'm not really sure, but cPanel support have been able to access my server now, so it looks like I don't need to action anything in this regard. :) Their initial investigation is pointing towards some sort of security breach! :(
 
Last edited:

magicalwonders

Well-Known Member
Nov 21, 2012
112
2
18
cPanel Access Level
Root Administrator
I'm happy to report that this matter has now been resolved. :)

It seems there was some malware on the server. :( Here's an extract from the support ticket....

"....It appears that your server has been compromised with a malicious payload designed to sniff for and steal server passwords. Everything that we know about this payload and how to identify it can be found here:

http://go.cpanel.net/checkyourserver ....................."

"................A proper OS Reload and restore of your server (as per any other root-level compromise) will address this issue and allow you to resume work as usual on your server............"
My host has now reinstalled the OS and re-established my accounts. A bit of tweaking and everything is working good again! :)

It's probably a good job I persisted investigating the issue. Had I listened to some of the support staff at my host company, they would have had me ignore the issue! :( Not the best advice I've ever had. :)

Thanks for all the help folks! :)

Myles
 

cPanelMichael

Administrator
Staff member
Apr 11, 2011
47,913
2,205
363
I am happy to see the issue is now resolved. Thank you for updating us with the outcome.