Restoring AWstats data from log backup question

ca2236

Well-Known Member
Feb 2, 2018
228
22
18
Nebraska
cPanel Access Level
DataCenter Provider
Hi @cPanelLauren,

This is a follow up from the conversation here:

Hello,

It is possible but it's a bit of a process. Here is how we do it:

extract the old log file copy the run the stats program manually against it, then move the resultant file to the correct flle name for that month:

Code:
cd /home/cPUser/tmp/awstats

cp /home/cPUser/logs/domainname.tld.bkup-Mon-20YY.gz

gzip -d domainname.tld.bkup-Mon-20YY.gz

/usr/local/cpanel/3rdparty/bin/awstats.pl -config=/home/cPUser/tmp/awstats/awstats.domainname.tld.conf -LogFile=/home/cPUser/tmp/awstats/domainnamed.tld.bkup-Mon-20YY

mv awstatsMM20YY.domainname.tld.txt awstatsMM20YY.domainname.tld.txt.bak

mv awstatsMM20YY.txt awstatsMM20YY.domainnamed.tld.txt

chown cPUser awstatsMM20YY.domainname.tld.txt
Above: replace MON with the Month (Jan, Feb, Mar, etc...), replace cPUser with the cPanel Username, domainname.tld with the domain name, MM with the month (01,02,03, etc...) and 20YY with the year.

Thank you,

I have the backup files coped over to the server, but I am not sure where the
step 1 comes into place. (for cd /home/CPUser/tmp/awstats)

and the second is just copy the domainname logs. Does that mean, just copy it anywhere?
and should I run awstats.pl on each of the log files from backup?

I apologize, I just don't want to mess up someone's stats :)
 

cPanelLauren

Product Owner
Staff member
Nov 14, 2017
13,296
1,271
313
Houston
Hi @ca2236

You're going to copy (not move) the backup file (for the missing time) at /home/cPuser/logs/domainname.tld.bkup-Month-YYYY.gz to /home/cPUser/tmp/awstats.

The action + filename looks like this on my server:

Code:
[[email protected] awstats]# cp /home/lauren/logs/mydomain.com-Apr-2018.gz /home/lauren/tmp/awstats/
Once that's done you need to decompress the file:

Code:
[[email protected] awstats]# gzip -d /home/lauren/tmp/awstats/mydomain.com-Apr-2018.gz
Once that's complete you'll need to configure awstats to run against those new records:

Code:
[[email protected] awstats]# /usr/local/cpanel/3rdparty/bin/awstats.pl -config=/home/lauren/tmp/awstats/awstats.mydomain.com.conf -LogFile=/home/lauren/tmp/awstats/mydomain.com-Apr-2018
Create/Update database for config "/home/lauren/tmp/awstats/awstats.mydomain.com.conf" by AWStats version 7.6 (build 20161204)
From data in log file "/home/lauren/tmp/awstats/mydomain.com-Apr-2018"...
Phase 1 : First bypass old records, searching new record...
Searching new records from beginning of log file...
Phase 2 : Now process new records (Flush history on disk after 20000 hosts)...
Jumped lines in file: 0
Parsed lines in file: 14043
 Found 0 dropped records,
 Found 0 comments,
 Found 0 blank records,
 Found 0 corrupted records,
 Found 0 old records,
 Found 14043 new qualified records.
That will create a couple new files which is why moving them is necessary hence the final steps:
Code:
mv awstatsMM20YY.domainname.tld.txt awstatsMM20YY.domainname.tld.txt.bak

mv awstatsMM20YY.txt awstatsMM20YY.domainnamed.tld.txt

chown cPUser awstatsMM20YY.domainname.tld.txt
Above: replace MON with the Month (Jan, Feb, Mar, etc...), replace cPUser with the cPanel Username, domainname.tld with the domain name, MM with the month (01,02,03, etc...) and 20YY with the year.
 

ca2236

Well-Known Member
Feb 2, 2018
228
22
18
Nebraska
cPanel Access Level
DataCenter Provider
Hi @cPanelLauren

I appreciate the detailed steps you have given me. One more question.

I copied the backup files from /var/log/apache2/domlogs/<domainname>/

which has two files

domainname.tld and domainname.tld-ssl_log


are these two files incorrect? They don't seem to match what you have given in example.
 

cPanelLauren

Product Owner
Staff member
Nov 14, 2017
13,296
1,271
313
Houston
Hello,


Well the files in /var/log/apache2/domlogs/domainname/domainname.tld and domainname.tld-ssl_log are both files that are automatically added to awstats.

What I explained was how to extract data from the user's archived data in /home/$user/logs/ not what's in /var/log/apache2/domlogs/

Does your user you're doing this for have the missing months archived in /home/$user/logs/ ?
 

ca2236

Well-Known Member
Feb 2, 2018
228
22
18
Nebraska
cPanel Access Level
DataCenter Provider
I just checked, it doesn't appear that he has logs for the missing month. (or at least part of it)

is it possible to rebuild from the directory path I gave you?
 

cPanelLauren

Product Owner
Staff member
Nov 14, 2017
13,296
1,271
313
Houston
If you run /scripts/runweblogs for that user does it automatically add in the user's data? If not, I believe it is possible to get the data as long as you have the weblogs but I would suggest opening a ticket using the link in my signature to ensure that is the case.
 

ca2236

Well-Known Member
Feb 2, 2018
228
22
18
Nebraska
cPanel Access Level
DataCenter Provider
Hi,

how do I use /scripts/runweblogs? It doesn't appear that obvious that I can copy files from a backup server, run it on those files and export them to the awstats folder
 

cPanelLauren

Product Owner
Staff member
Nov 14, 2017
13,296
1,271
313
Houston
Hi @ca2236

The usage of runweblogs is just to run statistics for one user. It won't grab the data from another server if that's where it's stored. If that's the case you may want to copy the logs to the live server in /etc/apache2/domlogs/domain.tld and then use /scripts/runweblogs to update the data


Thank you,
 

ca2236

Well-Known Member
Feb 2, 2018
228
22
18
Nebraska
cPanel Access Level
DataCenter Provider
Thanks,

but I guess I was wondering what the syntax of the steps are.

for example, if I copy the files from the backup server to the cpanel server and put them in a directory, can I run the script or another script to generate files that awstats will read.

If so, can it be any directory? if I copy them to the directory they are supposed to be on and rename it to not overwrite the current file, will that program work?
 
Last edited: