VPS Down What logs file need to be checked

lldeepakll

Well-Known Member

Jul 22, 2012

#1

Hi,

I have a VPS that was suddenly down, and now it is working fine. I want to find out the reason what causes the vps to down. So please tell me what logs files I should check to find out the reason of server down or how to check what process was consuming high cpu resources ?

C

CitizenK

Well-Known Member

Jul 22, 2012

#2

Hello,

Below are some of the common logfiles you will want to check when looking for the cause of errors. If you have questions about any of these logs please let us know, and if possible include the part of the log file in question.

Code:

System Messages: /var/log/messages
MySQL Error Log: /var/lib/mysql/hostname.err  (replace hostname with your servers hostname)
Server Status Log: /var/log/chkservd.log
cPanel Main Error Log: /usr/local/cpanel/logs/error_log
Maillog: /var/log/maillog
Apache: /usr/local/apache/domlogs/

L

lldeepakll

Well-Known Member

Jul 22, 2012

#3

Thanks for your reply. I could find the below logs that seems to me causes the problem, can you please suggest.
In /var/log/chkservd.log

Code:

[2012-07-21 04:11:06 -0500] Disk check .... /dev/sda3 (/) [7%] ... /dev/sdb1 (/disk2) [18%] ... /dev/sda1 (/boot) [82%] ... {status:ok} ... Done
[2012-07-21 04:11:06 -0500] Service check ....syslogd [[check command:+][socket connect:N/A]]...sshd [[check command:+][socket connect:N/A]]...spamd [[check command:+][socket connect:N/A]]...queueprocd [[check command:+][socket connect:N/A]]...named [[check command:+][socket connect:N/A]]...mysql [[check command:+][socket connect:N/A]]...mailman [[check command:+][socket connect:N/A]]...lfd [[check command:+][socket connect:N/A]]...ipaliases [[check command:+][socket connect:N/A]]...imap [[socket_service_auth:1][check command:+][socket connect:+]]...httpd [Timeout while trying to get data from service: Died at /usr/local/cpanel/Cpanel/TailWatch/ChkServd.pm line 607.
[2012-07-21 05:51:45 -0500] Disk check .... /dev/sda3 (/) [7%] ... /dev/sdb1 (/disk2) [18%] ... /dev/sda1 (/boot) [82%] ... {status:ok} ... Done
[2012-07-21 05:51:45 -0500] Service check ....syslogd [[check command:+][socket connect:N/A]]...sshd [[check command:+][socket connect:N/A]]...spamd [too soon after restart to check]...queueprocd [[check command:+][socket connect:N/A]]...named [[check command:+][socket connect:N/A]]...mysql [[check command:+][socket connect:N/A]]...mailman [[check command:-][socket connect:N/A][fail count:1]Restarting mailman....
[2012-07-21 06:32:43 -0500] Disk check .... /dev/sda3 (/) [7%] ... /dev/sdb1 (/disk2) [18%] ... /dev/sda1 (/boot) [82%] ... {status:ok} ... Done
[2012-07-21 06:32:43 -0500] Service check ....syslogd [[check command:+][socket connect:N/A]]...sshd [[check command:+][socket connect:N/A]]...spamd [too soon after restart to check]...queueprocd [[check command:+][socket connect:N/A]]...named [[check command:+][socket connect:N/A]]...mysql [[check command:+][socket connect:N/A]]...mailman [[check command:-][socket connect:N/A][fail count:1]Restarting mailman....
[2012-07-21 06:40:49 -0500] Disk check .... /dev/sda3 (/) [7%] ... /dev/sdb1 (/disk2) [18%] ... /dev/sda1 (/boot) [82%] ... {status:ok} ... Done
[2012-07-21 06:40:49 -0500] Service check ....syslogd [[check command:+][socket connect:N/A]]...sshd [[check command:+][socket connect:N/A]]...spamd [Service Check Interrupted
[2012-07-21 07:02:15 -0500] Disk check .... /dev/sda3 (/) [7%] ... /dev/sdb1 (/disk2) [18%] ... /dev/sda1 (/boot) [82%] ... {status:ok} ... Done
[2012-07-21 07:02:15 -0500] Service check ....syslogd [[check command:+][socket connect:N/A]]...sshd [[check command:+][socket connect:N/A]]...spamd [too soon after restart to check]...queueprocd [[check command:+][socket connect:N/A]]...named [[check command:+][socket connect:N/A]]...mysql [[check command:+][socket connect:N/A]]...mailman [[check command:-][socket connect:N/A][fail count:1]Restarting mailman....
[2012-07-21 07:08:02 -0500] Disk check .... /dev/sda3 (/) [7%] ... /dev/sdb1 (/disk2) [18%] ... /dev/sda1 (/boot) [82%] ... {status:ok} ... Done

In /var/log/messages

Code:

Jul 21 04:06:21 ehostrus pure-ftpd: ([email protected]) [NOTICE] /home/jaipur//www/advt/Hotel-Indiana-Classic-200X200.html uploaded  (421 bytes, 1.48KB/sec)
Jul 21 04:06:21 ehostrus pure-ftpd: ([email protected]) [INFO] Can't change directory to /www/advt/Hotel-Indiana-Classic-200X200.jpg: No such file or directory
Jul 21 04:06:24 ehostrus pure-ftpd: ([email protected]) [NOTICE] /home/jaipur//www/advt/Hotel-Indiana-Classic-200X200.jpg uploaded  (33034 bytes, 27.42KB/sec)
Jul 21 04:06:56 ehostrus pure-ftpd: ([email protected]) [INFO] Logout.
Jul 21 04:11:44 ehostrus kernel: Firewall: *UDP_IN Blocked* IN=eth0 OUT= MAC=00:25:90:74:fc:2a:00:15:63:3f:f9:e4:08:00 SRC=188.138.124.110 DST=184.154.204.123 LEN=60 TOS=0x00 PREC=0x00 TTL=1 ID=32373 PROTO=UDP SPT=42660 DPT=33470 LEN=40 
Jul 21 04:11:44 ehostrus kernel: Firewall: *UDP_IN Blocked* IN=eth0 OUT= MAC=00:25:90:74:fc:2a:00:15:63:3f:f9:e4:08:00 SRC=188.138.124.110 DST=184.154.204.123 LEN=60 TOS=0x00 PREC=0x00 TTL=2 ID=32374 PROTO=UDP SPT=44423 DPT=33471 LEN=40 
Jul 21 04:11:44 ehostrus kernel: Firewall: *UDP_IN Blocked* IN=eth0 OUT= MAC=00:25:90:74:fc:2a:00:15:63:3f:f9:e4:08:00 SRC=188.138.124.110 DST=184.154.204.123 LEN=60 TOS=0x00 PREC=0x00 TTL=1 ID=32375 PROTO=UDP SPT=51997 DPT=33472 LEN=40 
Jul 21 04:11:44 ehostrus kernel: Firewall: *UDP_IN Blocked* IN=eth0 OUT= MAC=00:25:90:74:fc:2a:00:15:63:3f:f9:e4:08:00 SRC=188.138.124.110 DST=184.154.204.123 LEN=60 TOS=0x00 PREC=0x00 TTL=2 ID=32376 PROTO=UDP SPT=43773 DPT=33473 LEN=40 
Jul 21 04:11:44 ehostrus kernel: Firewall: *UDP_IN Blocked* IN=eth0 OUT= MAC=00:25:90:74:fc:2a:00:15:63:3f:f9:e4:08:00 SRC=188.138.124.110 DST=184.154.204.123 LEN=60 TOS=0x00 PREC=0x00 TTL=3 ID=32377 PROTO=UDP SPT=44876 DPT=33474 LEN=40 
Jul 21 04:11:50 ehostrus kernel: Firewall: *UDP_IN Blocked* IN=eth0 OUT= MAC=00:25:90:74:fc:2a:00:15:63:3f:f9:e4:08:00 SRC=188.138.124.110 DST=184.154.204.123 LEN=60 TOS=0x00 PREC=0x00 TTL=5 ID=32386 PROTO=UDP SPT=36698 DPT=33483 LEN=40 
Jul 21 04:11:50 ehostrus kernel: Firewall: *UDP_IN Blocked* IN=eth0 OUT= MAC=00:25:90:74:fc:2a:00:15:63:3f:f9:e4:08:00 SRC=188.138.124.110 DST=184.154.204.123 LEN=60 TOS=0x00 PREC=0x00 TTL=7 ID=32388 PROTO=UDP SPT=59209 DPT=33485 LEN=40 
Jul 21 04:11:55 ehostrus kernel: Firewall: *UDP_IN Blocked* IN=eth0 OUT= MAC=00:25:90:74:fc:2a:00:15:63:3f:f9:e4:08:00 SRC=188.138.124.110 DST=184.154.204.123 LEN=60 TOS=0x00 PREC=0x00 TTL=11 ID=32402 PROTO=UDP SPT=58468 DPT=33499 LEN=40 
Jul 21 04:11:55 ehostrus kernel: Firewall: *UDP_IN Blocked* IN=eth0 OUT= MAC=00:25:90:74:fc:2a:00:15:63:3f:f9:e4:08:00 SRC=188.138.124.110 DST=184.154.204.123 LEN=60 TOS=0x00 PREC=0x00 TTL=11 ID=32404 PROTO=UDP SPT=37256 DPT=33501 LEN=40 
Jul 21 04:11:55 ehostrus kernel: Firewall: *UDP_IN Blocked* IN=eth0 OUT= MAC=00:25:90:74:fc:2a:00:15:63:3f:f9:e4:08:00 SRC=188.138.124.110 DST=184.154.204.123 LEN=60 TOS=0x00 PREC=0x00 TTL=12 ID=32405 PROTO=UDP SPT=44281 DPT=33502 LEN=40 
Jul 21 04:12:00 ehostrus kernel: Firewall: *UDP_IN Blocked* IN=eth0 OUT= MAC=00:25:90:74:fc:2a:00:15:63:3f:f9:e4:08:00 SRC=188.138.124.110 DST=184.154.204.123 LEN=60 TOS=0x00 PREC=0x00 TTL=16 ID=32419 PROTO=UDP SPT=45804 DPT=33516 LEN=40 
Jul 21 04:12:00 ehostrus kernel: Firewall: *UDP_IN Blocked* IN=eth0 OUT= MAC=00:25:90:74:fc:2a:00:15:63:3f:f9:e4:08:00 SRC=188.138.124.110 DST=184.154.204.123 LEN=60 TOS=0x00 PREC=0x00 TTL=16 ID=32418 PROTO=UDP SPT=48229 DPT=33515 LEN=40 
Jul 21 04:13:22 ehostrus pure-ftpd: ([email protected]) [INFO] New connection from 127.0.0.1
Jul 21 04:13:58 ehostrus pure-ftpd: ([email protected]) [INFO] __cpanel__service__auth__ftpd__O8CnRne2jxzfDnnZ9r53jiDg4EoCQw3wK54nFd1SRs2jrww18LvAzIeMsmp6fPHV is now logged in
Jul 21 04:13:59 ehostrus pure-ftpd: (__cpanel__service__auth__ftpd__O8CnRne2jxzfDnnZ9r53jiDg4EoCQw3wK54nFd1SRs2jrww18LvAzIeMsmp6fPHV@127.0.0.1) [INFO] Logout.
Jul 21 04:15:47 ehostrus kernel: httpd invoked oom-killer: gfp_mask=0x201d2, order=0, oomkilladj=0
Jul 21 04:15:48 ehostrus kernel: 
Jul 21 04:15:48 ehostrus kernel: Call Trace:
Jul 21 04:15:48 ehostrus kernel:  [<ffffffff800d4a52>] out_of_memory+0x9f/0x25c
Jul 21 04:15:48 ehostrus kernel:  [<ffffffff8000f15f>] __alloc_pages+0x2a0/0x34d
Jul 21 04:15:48 ehostrus kernel:  [<ffffffff80154b24>] __first_cpu+0xe/0x1d
Jul 21 04:15:48 ehostrus kernel:  [<ffffffff80012c5b>] __do_page_cache_readahead+0x96/0x18e
Jul 21 04:15:48 ehostrus kernel:  [<ffffffff8001364d>] filemap_nopage+0x14c/0x386
Jul 21 04:15:48 ehostrus kernel:  [<ffffffff800082e4>] __handle_mm_fault+0x26f/0xd9b
Jul 21 04:15:48 ehostrus kernel:  [<ffffffff8006555d>] thread_return+0x6a/0x177
Jul 21 04:15:48 ehostrus kernel:  [<ffffffff800693d8>] do_page_fault+0x4c9/0x846
Jul 21 04:15:48 ehostrus kernel:  [<ffffffff8005edff>] del_timer_sync+0xc/0x16
Jul 21 04:16:02 ehostrus kernel:  [<ffffffff8009a6a9>] process_timeout+0x0/0x5
Jul 21 04:16:03 ehostrus pure-ftpd: ([email protected]) [INFO] Logout.
Jul 21 04:16:45 ehostrus kernel:  [<ffffffff8010af98>] sys_epoll_wait+0x3b8/0x3f9
Jul 21 04:18:37 ehostrus kernel:  [<ffffffff80061e39>] error_exit+0x0/0x84
Jul 21 04:19:44 ehostrus kernel: 
Jul 21 04:20:34 ehostrus kernel: Mem-info:
Jul 21 04:20:48 ehostrus kernel: Node 0 DMA per-cpu:
Jul 21 04:21:45 ehostrus kernel: cpu 0 hot: high 0, batch 1 used:0
Jul 21 04:22:43 ehostrus kernel: cpu 0 cold: high 0, batch 1 used:0
Jul 21 04:23:00 ehostrus kernel: cpu 1 hot: high 0, batch 1 used:0
Jul 21 04:23:16 ehostrus kernel: cpu 1 cold: high 0, batch 1 used:0
Jul 21 04:23:34 ehostrus kernel: cpu 2 hot: high 0, batch 1 used:0
Jul 21 04:23:41 ehostrus kernel: cpu 2 cold: high 0, batch 1 used:0
Jul 21 04:24:09 ehostrus kernel: cpu 3 hot: high 0, batch 1 used:0
Jul 21 07:04:00 ehostrus kernel: Firewall: *ICMP_IN Blocked* IN=eth0 OUT= MAC=00:25:90:74:fc:2a:00:15:63:3f:f9:e4:08:00 SRC=74.217.89.116 DST=184.154.204.123 LEN=84 TOS=0x00 PREC=0x00 TTL=48 ID=58521 PROTO=ICMP TYPE=8 CODE=0 ID=13830 SEQ=9291 
Jul 21 07:04:02 ehostrus kernel: Firewall: *ICMP_IN Blocked* IN=eth0 OUT= MAC=00:25:90

C

CitizenK

Well-Known Member

Jul 22, 2012

#4

Hello,

I did not see any issues in the logs you posted, however I recommend that if you want a cPanel Tech to take a deeper look at your system that you open a ticket at https://tickets.cpanel.net

L

lldeepakll

Well-Known Member

Jul 22, 2012

#5

OK. Can you please guide me about this process. (

PHP:

 <defunct>). I have noticed that this process taking too much cpu memory. 
USER     PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME   COMMAND

ehostr    432915  16.0  0.0      0     0 ?        ZN   06:42       0:00         [php] <defunct>

C

CitizenK

Well-Known Member

Jul 22, 2012

#6

if you notice that php processes are running too long and consuming too many resources this is commonly caused my a excessive value for "max_execution_time"

In WHM >> Service Configuration >> PHP Configuration Editor and check to see if the "max_execution_time" is set above 30 and the max_input_time is set above 60. If they are above these values you may want to consider lowering them. ß

L

lldeepakll

Well-Known Member

Jul 22, 2012

#7

I will check the same. Thanks for your support cpCitizenK

C

CitizenK

Well-Known Member

Jul 22, 2012

#8

Thanks! Let us know if that helped.

L

lldeepakll

Well-Known Member

Jul 22, 2012

#9

I have checked and max_execution_time is already set 30 to and max_input_time is set to 60. Do I need to increase these value ? Further I have killed that process.

C

CitizenK

Well-Known Member

Jul 22, 2012

#10

No, I would not recommend increasing these values. Unless you have a specific need, it is best to keep these at the default values I posted above.

cPanelTristan

Quality Assurance Analyst

Staff member

Jul 22, 2012

#11

The issue where you see "httpd invoked oom-killer" means you've run out of memory on the system at that point in time. I discuss that error specifically:

http://forums.cpanel.net/f5/out-memory-283892.html#post1186502

It's most definitely an issue to run out of memory. That log entry in /var/log/messages is definitely important on what happened.

L

lldeepakll

Well-Known Member

Jul 23, 2012

#12

Hi Tristan,

I did not find the keyword "incorrect key" in /var/lib/mysql/*.err as you suggested. But I have found the below logs in /var/log/messages just before the "Jul 21 04:15:47 ehostrus kernel: httpd invoked oom-killer: gfp_mask=0x201d2, order=0, oomkilladj=0
Jul 21 04:15:48 ehostrus kernel:
Jul 21 04:15:48 ehostrus kernel: Call Trace:
Jul 21 04:15:48 ehostrus kernel: [<ffffffff800d4a52>] out_of_memory+0x9f/0x25c"

Can you please assist about below logs.

Jul 21 03:56:46 ehostrus suhosin[862623]: ALERT - configured GET variable value length limit exceeded - dropped variable 'maneref' (attacker '203.88.22.141', file

'/home/japur/public_html/index.php')
Jul 21 03:56:46 ehostrus suhosin[862623]: ALERT - configured GET variable value length limit exceeded - dropped variable 'maneref' (attacker '203.88.22.141', file

'/home/japur/public_html/index.php')
Jul 21 03:56:51 ehostrus suhosin[862628]: ALERT - configured GET variable value length limit exceeded - dropped variable 'maneref' (attacker '203.88.22.141', file

'/home/japur/public_html/index.php')
Jul 21 03:56:51 ehostrus last message repeated 2 times

cPanelTristan

Quality Assurance Analyst

Staff member

Jul 23, 2012

#13

The logs simply show suhosin dropping a connection due to the GET variable being too long for the IP 203.88.22.141 to page /home/japur/public_html/index.php

I'm not certain unless there were a lot of these at the time that it could have caused the issue for memory depletion. You might have a DoS type attack going on if that's the case. CSF, ddos-deflate and mod_qos can all be used to help in the case of a denial of service attack.

Thread starter	Similar threads	Forum	Replies	Date
G	2 vps for optimal performance	Site Performance Analytics	2	Apr 4, 2021
Z	Dedicated vs VPS CPanel Performance	Site Performance Analytics	1	Dec 9, 2020
	VPS Crashes Every 24 Hours - No Logs?	Site Performance Analytics	2	Feb 16, 2018
	Performance Comparison: Dedicated vs VPS	Site Performance Analytics	1	May 23, 2017
A	Cloud / VPS Performance and Licensing	Site Performance Analytics	1	Aug 13, 2012

Search

Search

VPS Down What logs file need to be checked

lldeepakll

Well-Known Member

CitizenK

Well-Known Member

lldeepakll

Well-Known Member

CitizenK

Well-Known Member

lldeepakll

Well-Known Member

CitizenK

Well-Known Member

lldeepakll

Well-Known Member

CitizenK

Well-Known Member

lldeepakll

Well-Known Member

CitizenK

Well-Known Member

cPanelTristan

Quality Assurance Analyst

lldeepakll

Well-Known Member

cPanelTristan

Quality Assurance Analyst