Website down due to httpd

[sohailkmu]

Dear All,

I am pretty new to this forum. I am quite hopeful that I ll receive expert opinion from respected members of the forum.

My website is hosted on a vps with 30 GB space and 1GB RAM with Intel(R) Xeon(R) CPU E5620 @ 2.40GHz.

My site platform is drupal.

At times the load averages reach 23 and the site becomes inaccessible. sometimes it becomes stable with restarting of mysql and httpd. But some times it stucks.

Recently I found that it is due to multiple httpd lines showing in process list with each consuming 5% of the memory. My result from top is below:

top - 09:14:35 up 27 days,  8:04,  1 user,  load average: 3.05, 6.29, 6.93
Tasks:  55 total,   7 running,  48 sleeping,   0 stopped,   0 zombie
Cpu(s): 89.0%us, 10.4%sy,  0.0%ni,  0.6%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
Mem:   1048576k total,   622328k used,   426248k free,        0k buffers
Swap:        0k total,        0k used,        0k free,        0k cached

  PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND
26294 kmuedupk  18   0 54996  38m 6044 R 23.5  3.8   0:00.71 /usr/bin/php /home/kmuedupk/public_html/index.php
26297 kmuedupk  18   0 34292  17m 5512 R  8.0  1.7   0:00.24 /usr/bin/php /home/kmuedupk/public_html/index.php
26239 nobody    15   0 62020  51m 1968 S  2.7  5.0   0:00.17 /usr/local/apache/bin/httpd -k start -DSSL
24211 nobody    15   0 62044  51m 2004 S  2.0  5.0   0:00.85 /usr/local/apache/bin/httpd -k start -DSSL
26262 nobody    15   0 61528  50m 1940 S  2.0  5.0   0:00.07 /usr/local/apache/bin/httpd -k start -DSSL
24138 nobody    15   0 61664  50m 1988 S  1.7  5.0   0:00.31 /usr/local/apache/bin/httpd -k start -DSSL
26136 nobody    15   0 61672  50m 1960 S  1.7  5.0   0:00.22 /usr/local/apache/bin/httpd -k start -DSSL
26240 nobody    15   0 61528  50m 1948 S  1.7  5.0   0:00.07 /usr/local/apache/bin/httpd -k start -DSSL
22462 nobody    15   0 62032  51m 1968 S  1.3  5.0   0:00.90 /usr/local/apache/bin/httpd -k start -DSSL
26013 nobody    15   0 61664  50m 1964 S  1.0  5.0   0:00.20 /usr/local/apache/bin/httpd -k start -DSSL
26292 nobody    15   0 61528  50m 1940 S  1.0  5.0   0:00.03 /usr/local/apache/bin/httpd -k start -DSSL
16289 mysql     15   0  180m  73m 5608 S  0.7  7.2  15:02.82 /usr/sbin/mysqld --basedir=/ --datadir=/var/lib/mysql --user=mysql --log-error=/var/lib/mysql/server.kmu.edu.pk.err
25831 nobody    15   0 61612  50m 2024 S  0.7  5.0   0:00.27 /usr/local/apache/bin/httpd -k start -DSSL
26286 root      15   0  2564 1088  880 R  0.7  0.1   0:00.02 top c
26293 nobody    15   0 61528  50m 1944 S  0.7  5.0   0:00.02 /usr/local/apache/bin/httpd -k start -DSSL
 3245 root      18   0 61260  51m 2796 S  0.3  5.0   0:08.43 /usr/local/apache/bin/httpd -k start -DSSL
26009 nobody    15   0 61528  50m 1964 S  0.3  5.0   0:00.22 /usr/local/apache/bin/httpd -k start -DSSL
26241 nobody    15   0 62012  51m 1960 S  0.3  5.0   0:00.10 /usr/local/apache/bin/httpd -k start -DSSL
26242 nobody    15   0 61532  50m 1952 S  0.3  5.0   0:00.04 /usr/local/apache/bin/httpd -k start -DSSL
26287 nobody    15   0 61528  50m 1928 S  0.3  5.0   0:00.01 /usr/local/apache/bin/httpd -k start -DSSL
26290 nobody    16   0 61532  50m 1932 S  0.3  4.9   0:00.01 /usr/local/apache/bin/httpd -k start -DSSL
26291 nobody    16   0 61532  50m 1828 S  0.3  4.9   0:00.01 /usr/local/apache/bin/httpd -k start -DSSL
26296 nobody    20   0 61528  50m 1932 R  0.3  5.0   0:00.01 /usr/local/apache/bin/httpd -k start -DSSL
    1 root      15   0  2896 1380 1200 S  0.0  0.1   0:36.14 init
 1477 root      18   0 37192 2616 1008 S  0.0  0.2   0:17.16 /sbin/rsyslogd -i /var/run/syslogd.pid -c 5
 1515 root      15   0  8636 1032  540 S  0.0  0.1   0:12.15 /usr/sbin/sshd
 1522 root      25   0  3216  920  728 S  0.0  0.1   0:00.00 xinetd -stayalive -pidfile /var/run/xinetd.pid
 1636 root      18   0  8832  792  348 S  0.0  0.1   0:00.00 /usr/sbin/saslauthd -m /var/run/saslauthd -a pam -n 2
 1637 root      18   0  8832  512   68 S  0.0  0.0   0:00.00 /usr/sbin/saslauthd -m /var/run/saslauthd -a pam -n 2
 1728 root      18   0  2940  524  380 S  0.0  0.0   0:00.82 /usr/sbin/atd
 1852 root      15   0 17920  12m 5096 S  0.0  1.2   0:48.05 cpsrvd (SSL) - waiting for c --llu=1383856787 --listen=3,4,5,6,7,8,9,10
 9828 root      14  -4  2460  568  348 S  0.0  0.1   0:00.00 /sbin/udevd -d
10012 root      15   0  6220 3788 1432 S  0.0  0.4   0:03.27 queueprocd - wait to process a task
11718 mailnull  15   0 11172 2516 1972 S  0.0  0.2   0:12.35 /usr/sbin/exim -bd -q1h
11906 root      15   0 39988  32m 2880 S  0.0  3.1   0:20.78 /usr/local/cpanel/3rdparty/perl/514/bin/spamd -d --allowed-ips=127.0.0.1 --pidfile=/var/run/spamd.pid --max-children
13379 root      15   0  7440 4572  888 S  0.0  0.4   0:07.56 tailwatchd
13883 named     25   0 51116  12m 2588 S  0.0  1.2   0:00.11 /usr/sbin/named -u named
14159 root      15   0 17760 8960  680 S  0.0  0.9   0:04.59 cpdavd - accepting connections on 2077 and 2078
14168 root      33  18  5148 2644 1252 S  0.0  0.3   0:00.78 cpanellogd - sleeping for logs

Here is my output from error_log of apache:

[email protected] [~]# tail -f /usr/local/apache/logs/error_log
[Sun Nov 17 07:28:26 2013] [error] [client 66.249.73.14] (12)Cannot allocate memory: couldn't create child process: /opt/suphp/sbin/suphp for /home/---edupk/public_html/index.php
[Sun Nov 17 07:28:26 2013] [error] [client 66.249.73.14] (12)Cannot allocate memory: couldn't create child process: /opt/suphp/sbin/suphp for /home/---edupk/public_html/index.php
[Sun Nov 17 07:29:57 2013] [error] [client 82.145.220.74] (70007)The timeout specified has expired: ap_content_length_filter: apr_bucket_read() failed
[Sun Nov 17 07:46:34 2013] [error] [client 208.115.111.72] ModSecurity: Rule be42070 [id "390149"][file "/usr/local/apache/conf/modsec_rules/50_asl_rootkits.conf"][line "104"] - Execution error - PCRE limits exceeded (-8): (null). [hostname "---.edu.pk"] [uri "/qec/index.php"] [unique_id "UoguAkj5l5wAAD7kYLAAAAAK"]
[Sun Nov 17 07:57:05 2013] [error] [client 115.167.47.187] ModSecurity: Rule be42070 [id "390149"][file "/usr/local/apache/conf/modsec_rules/50_asl_rootkits.conf"][line "104"] - Execution error - PCRE limits exceeded (-8): (null). [hostname "ipmr.---.edu.pk"] [uri "/index.php"] [unique_id "UogwXkj5l5wAAD-0ekEAAAAL"]
[Sun Nov 17 07:57:10 2013] [error] [client 115.167.47.187] ModSecurity: Rule be42070 [id "390149"][file "/usr/local/apache/conf/modsec_rules/50_asl_rootkits.conf"][line "104"] - Execution error - PCRE limits exceeded (-8): (null). [hostname "ipmr.---.edu.pk"] [uri "/index.php"] [unique_id "Uogwgkj5l5wAAEQPu34AAAAK"]
[Sun Nov 17 08:13:38 2013] [notice] Graceful restart requested, doing restart
[Sun Nov 17 08:13:38 2013] [error] (9)Bad file descriptor: apr_socket_accept: (client socket)
[Sun Nov 17 08:13:40 2013] [notice] SSL FIPS mode disabled
[Sun Nov 17 08:13:41 2013] [notice] Apache/2.2.23 (Unix) mod_ssl/2.2.23 OpenSSL/1.0.0-fips mod_auth_passthrough/2.1 mod_bwlimited/1.4 FrontPage/5.0.2.2635 Apache configured -- resuming normal operations

thanks

 

[sohailkmu]

VPS on high average load due to httpd processes

Dear Members,

Kindly help me in resolving my issue.

I have VPS based web hosting with 30GB disk space and 1.0 GB guaranteed RAM.

My server remains overloaded (average load).

I have multiple linnes of /usr/local/apache/bin/httpd -k start -DSSL consuming 5% of the memory each.
I think this is the reason for high load average. How can I reduce it.

Besides this whenever I make changes to apache configuration through WHM, following errors are displayed.

[Tue Nov 19 12:58:57 2013] [crit] (EAI 9)Address family for hostname not supported: alloc_listener: failed to set up sockaddr for ::

Configuration problem detected on line 172 of file /usr/local/apache/conf/httpd.conf.work.O60GBbOFSwotmcqu:	Listen setup failed

	--- /usr/local/apache/conf/httpd.conf.work.O60GBbOFSwotmcqu ---
	166# # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # #
	167
	168
	169PidFile logs/httpd.pid
	170LockFile logs/accept.lock
	171Listen 0.0.0.0:80
	172 ===> Listen [::]:80 <===
	173User nobody
	174Group nobody
	175ExtendedStatus On
	176ServerAdmin  [email protected] xxx.edu.pk
	177ServerName server. xxx.edu.pk
	178LogLevel warn
	--- /usr/local/apache/conf/httpd.conf.work.O60GBbOFSwotmcqu ---


Rebuilding configuration without any local modifications.

Failed to generate a syntactically correct Apache configuration.
Bad configuration file located at /usr/local/apache/conf/httpd.conf.work.O60GBbOFSwotmcqu
Error:
[Tue Nov 19 12:59:15 2013] [crit] (EAI 9)Address family for hostname not supported: alloc_listener: failed to set up sockaddr for ::

Configuration problem detected on line 172 of file /usr/local/apache/conf/httpd.conf.work.O60GBbOFSwotmcqu:	Listen setup failed

	--- /usr/local/apache/conf/httpd.conf.work.O60GBbOFSwotmcqu ---
	166# # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # #
	167
	168
	169PidFile logs/httpd.pid
	170LockFile logs/accept.lock
	171Listen 0.0.0.0:80
	172 ===> Listen [::]:80 <===
	173User nobody
	174Group nobody
	175ExtendedStatus On
	176ServerAdmin  webmaster @  xxx.edu.pk
	177ServerName server. xxx.edu.pk
	178LogLevel warn
	--- /usr/local/apache/conf/httpd.conf.work.O60GBbOFSwotmcqu ---



Restarting httpd (hard)...

Apache successfully restarted.

Done!

 

[Bidi]

Im not an expert but i gve an advice check if you install sompting strange or plugin on the platform witch is using lot of stuff and makes theath hapen.

 

[cPanelMichael]

Hello

I reviewed a few other instances where the "(EAI 9)Address family for hostname not supported: alloc_listener: failed to set up sockaddr for" error message occurred and each time it was resolved by rebuilding Apache via EasyApache:

# /scripts/easyapache

Or;

"WHM Home » Software » EasyApache (Apache Update)"

As for the load issue, I recommend reviewing the following thread:

Troubleshooting High Load On Linux Systems

Thank you.

 

[sohailkmu]

Thanks alot for your reply.

I already used /scripts/upcp.

I ll give it a try as well.