cron problems

[Dan Lester]

OK, I'm in "cron hell". I have asked cPanel to create a simple cron job that just appends one file to another, once a day, at 11:55pm. Thusly

55 23 * * * cat ~/access-logs/subdomain.myaccount.net >> ~/subdomain.myaccount.net/totalarchive

This is just to make a running access-log called "totalarchive" out of the daily access logs.

That command works fine, when I give it over ssh. It doesn't work at all when I schedule it with cron. So the command is executable. BTW, I have asked that cron notify me by e-mail when it does that command. I never get notified. Not even an error. So evidently it doesn't get executed because it isn't even trying to execute it!

My host providers are clueless: "Cron jobs aren't our responsibility!" Sez Level 2 support. Duh?

So my question is ... is this a cPanel issue, or a provider issue? I there somewhere in cPanel that you can simply turn off cron jobs? Help!

 

[rpvw]

Have you tried running it as a script ?

Try creating a file eg cronjob.sh (don't forget to make the permissions executable) with something similar to the following content and call that via cron

#!/bin/sh
cat ~/access-logs/subdomain.myaccount.net >> ~/subdomain.myaccount.net/totalarchive

 

[Dan Lester]

I have not tried that, and probably should. Is there some reason why cron would run a script and not a shell command?

 

[rpvw]

Generally, a shell command in cron would need to have fully qualified path to the executable, so you might make it work using /bin/cat (if that is where your cat executable resides).

You may well still need to define the full path to an executable inside a shell script - much will depend on who (eg system admin root user, or non-privileged user) is calling the cron, and what access they have been allocated.

You should be able to determine where the cat command is (and if it is run as an external executable, or from inside a shell) by running $ type cat

$ type cat
cat is /bin/cat

You can confirm there are no external executables by using $ which cat

$ which cat
/bin/cat

(Try the above commands with other instructions like echo and help instead of cat to see the difference)

 

[cPanelMichael]

Hello,

Let us know if the previous post helps.

Thank you.

 

[Dan Lester]

My cat is all /bin/cat. So you mean I have to command in cron,
/bin/cat ~/access-logs/subdomain.myaccount.net >> ~/subdomain.myaccount.net/totalarchive ?

 

[rpvw]

Cant hurt to try it - it will either work or you will have to go down the shell script route

You may be able to increase the verbosity of the logged output by adding -vvv to the command.

(You are changing your ~ to the fully qualified paths of where your access-logs files are residing ? Cron may not be running from the folder root of those files)

 

[Dan Lester]

Well, this just gets snarkier and snarkier.

I tried it with a script. My script file was /home1/memyselfani/job.sh, which contained fully qualified addresses ...

#!/bin/bash
cat /home1/memyselfandi/access-logs/subdomain.myaccount.net >> /home1/memyselfandi/subdomain.myaccount.net/test1

and the cron job I invoked was

bash /home1/memyselfandi/job.sh

You'd like to believe at the scheduled time, a file test1 would appear in the folder subdomain.myaccount.net and I'd get an e-mail telling me it happened. Nope. Nada.

Oh, when I run that bash command from the shell, it works perfectly.

WTF???

 

[Dan Lester]

OK, now I just tried a simple (enhanced) cron job

/bin/cat /home1/memyselfandi/access-logs/subdomain.myaccount.net >> /home1/memyselfandi/subdomain.myaccount.net/test1

No joy. I'm just baffled. No errors, no responses. Nothing.

 

[rpvw]

Does your created file test1 have any content?
Is content concatenated into the test1 file after multiple manual runs in shell?

I would have thought that the first part of your cat command should refer to a file to read (or to a wildcard pattern to read more than one file) but you seem to have it only referring to a folder /subdomain.myaccount.net (Be careful with wildcard matches and writing to the same folder - you can get in a run-away loop. Better to use a pattern match)

It is entirely possible that running the command under a users shell will create a zero byte file (since no input has been defined) called test1, but under cron will error out and do nothing.

 

[Dan Lester]

The problem is that once the scheduled time for the cron job is reached, there is no test1. test1 never happens. When I run the command in the shell, it all works the way I want it to, and I get a test1 file with the right access-log content in it. The cat command does refer to a file to read. That file is named subdomain.myaccount.net, and it is sitting in the access-logs folder. I can cat, more, or even vi that file. It's a file. Really.

 

[rpvw]

I think I can see where this is going.

If I am not mistaken, access-logs is a symlink to /usr/apache/domlogs/<username> which in turn is symlinked to /etc/apache2/logs/domlogs which is symlinked to /var/log/apache2/domlogs

On my servers the ownership of the files are root, and the Group is the user. In order that a cron call will execute the command as you intend, it may have to be run as root.

There has been some discussion about whether a file run by cron should be named filename.sh or just filename and be executable. Personally I have various executable files named filename.sh run under root, but have never tried to run them as a user.

There is a handy quick reference at 6 Reasons Your cron Job is Not Running - and further reading at Why is my crontab not working, and how can I troubleshoot it?

 

[Dan Lester]

That's very interesting. In fact, that file in access-logs IS owned by root (but it is in my group, which is the user). root has rw permission on that file, and I have r permission on it. That's why I can view the file in a shell script. So cron doesn't have the same privileges that I do?

Geez, does that mean that I have NO cron capabilities for access-logs, but I'm just free to manually access them?

Thank you for the interesting reference links. Need to study those.

 

[rpvw]

One (there are many) of the biggest irritations of the cron shell is that it runs it's own shell in a very restricted environment. This shows up as the command you want to run working perfectly if you execute it from your shell, but then it stamps its foot and pouts petulantly (much like politicians) and refuses to do anything when you try and have it run under cron.

I usually side-step the issues by running any problematic cron job as root - but this is certainly not the correct answer, and anyway, you may not always have that option !

 

[Dan Lester]

Oh goodness. I certainly don't have the option of running as root. I'm using a shared server. I don't even have WHM access. I had no idea that cron couldn't do everything I could do manually.

So if I can't use cron to schedule certain shell scripts, what other options might I have?

 

[rpvw]

If you have PHP exec() available (not a great idea as this is ofton used by hackers to inject code), you might be able to create a PHP file that can be executed by cron and that then calls the shell script you want to execute.

PHP: exec - Manual
PHP: shell_exec - Manual

and you might want to read exec() disabled when PHP-FPM enabled if you are using FPM

Alternately, since your user can read the files, you may be able to use PHP file_get_contents() Function and then write the string to your output file located in the users home folder with their full RW permissions.

 

[Dan Lester]

Well, I'll see what I can do. Thank you for your insights. I am *stunned* that there are such restrictions on cron usage. One would think that whatever you're able to do in the shell, you should be able to do with cron. So this is an OS deficiency, rather than a cPanel deficiency. One wonders what motivated the OS developers to make it so screwy. Doesn't sound like a security issue.

 

[Infopro]

This is just to make a running access-log called "totalarchive" out of the daily access logs.

Can you provide any more details as to what you're hoping to do exactly? Are you experiencing some issue that you'd like to track the logs for or something?

 

[Dan Lester]

Pretty simple. This implementation of cPanel, or the underlying Linux, produces DAILY access logs, and overwrites them, starting at midnight. So on Tuesday, I don't get to look at the logs from Monday. I'd like to review the logs for a given WEEK. My Mac Apache 2.3 produced weekly logs automatically. You set that up in /etc/newsyslog.d/apache.conf as root. So, not being able to access that file in my cPanel, what I was trying to do was, at the end of each day, append the daily log to a master log file. At the end of the week, that master log file would be moved to a safe place, so a new one could be started. I have a service I do each week, so my operational time scale is one week. In principle, that should have been a trivial job in cron. In reality, evidently it is not.

Ideas?

 

[Infopro]

cPanel, or the underlying Linux, produces DAILY access logs, and overwrites them, starting at midnight.

Are the access logs of no use to you?




You might consider getting your own VPS with full root access. Then you'll have plenty of logs to look at.