Please whitelist cPanel in your adblocker so that you’re able to see our version release promotions, thanks!

The Community Forums

Interact with an entire community of cPanel & WHM users!

SOLVED The service “httpd” appears to be down.

Discussion in 'General Discussion' started by LordLiverpool, Nov 10, 2017.

  1. LordLiverpool

    LordLiverpool Well-Known Member

    Joined:
    Dec 27, 2014
    Messages:
    53
    Likes Received:
    9
    Trophy Points:
    8
    cPanel Access Level:
    Root Administrator
    Hello cPanel

    In the past 2 weeks my HTTPD has gone down 3 times.
    Normally it would recover itself.

    Each time I got this email:

    01 httpd appears to be down.JPG

    Looks like the reason it can't recover is:

    "Unable to connect to port 80 on 127.0.0.1: Connection refused: died"

    I looked on the cPanel forum and out on Google for a solution and found similar threads:
    Following the advice contained within them,
    I checked the error log at:

    /usr/local/cpanel/logs/error_log

    And found this at the time it fell over:

    Code:
    [2017-11-09 01:17:12 +0000] info [gather-update-logs] File /usr/local/cpanel/logs/update_analysis/2017-11-09T01:17:07Z.tar.gz has been sent successfully to updatelogdrop.cpanel.net.
    ==> cpsrvd 11.68.0.10 started
    ==> cpsrvd: loading security policy....Done
    ==> cpsrvd: Setting up SSL support ... Done
    ==> cpsrvd: transferred port bindings
    ==> cpsrvd: bound to ports
    ==> cpsrvd 11.68.0.10 started
    ==> cpsrvd: loading security policy....Done
    ==> cpsrvd: Setting up SSL support ... Done
    ==> cpsrvd: transferred port bindings
    ==> cpsrvd: bound to ports
    ==> cpsrvd 11.68.0.10 started
    ==> cpsrvd: loading security policy....Done
    ==> cpsrvd: Setting up SSL support ... Done
    ==> cpsrvd: transferred port bindings
    ==> cpsrvd: bound to ports
    [2017-11-09 01:31:57 +0000] warn [restartsrv_httpd] The 'httpd' service's PID file '/var/run/apache2/httpd.pid' did not appear after 180 seconds.
    
    [2017-11-09 01:31:58 +0000] info [queueprocd] chkservd::Notify Notification => ***********@**mail.*** via EMAIL [eventimportance => High (1)]
    [2017-11-09 01:32:02 +0000] info [cpaddons] Successfully verified signature for cpanel (key types: release).
    [2017-11-09 01:32:04 +0000] info [cpanelsync] Successfully verified signature for cpanel (key types: release).
    [2017-11-09 01:32:04 +0000] info [cpanelsync] Successfully verified signature for cpanel (key types: release).
    [2017-11-09 01:32:05 +0000] info [cpanelsync] Successfully verified signature for cpanel (key types: release).
    [2017-11-09 01:32:05 +0000] info [cpanelsync] Successfully verified signature for cpanel (key types: release).
    [2017-11-09 01:32:12 +0000] info [cpaddons] Successfully verified signature for cpanel (key types: release).
    [2017-11-09 01:36:58 +0000] warn [restartsrv_httpd] The 'httpd' service's PID file '/var/run/apache2/httpd.pid' did not appear after 180 seconds.
    
    I have CSF firewall installed, but I haven't made any changes to it myself for months.

    I can post the iptables rules here if you like? But I noticed they were redacted from other posts.

    I searched the iptable rules for the string "127.0.0.1" but got zero matches. Is that good?

    Search.JPG

    I'm not an expert in server administration so if you want access to my server I can open a support ticket.

    Any help is greatly appreciated.

    Best Regards

    Mike
     
    Stop hovering to collapse... Click to collapse... Hover to expand... Click to expand...
  2. cPanelMichael

    cPanelMichael Technical Support Community Manager
    Staff Member

    Joined:
    Apr 11, 2011
    Messages:
    44,803
    Likes Received:
    1,898
    Trophy Points:
    363
    cPanel Access Level:
    Root Administrator
    Twitter:
    Hello,

    Could you open a support ticket so we can take a closer look?

    Thank you.
     
    Stop hovering to collapse... Click to collapse... Hover to expand... Click to expand...
    LordLiverpool likes this.
  3. LordLiverpool

    LordLiverpool Well-Known Member

    Joined:
    Dec 27, 2014
    Messages:
    53
    Likes Received:
    9
    Trophy Points:
    8
    cPanel Access Level:
    Root Administrator
    Michael

    I really appreciated you helping me.

    I've opened a ticket: #9019765 and provided a more detailed explanation, as best as I understand.

    Thanks very much.

    Mike
     
    Stop hovering to collapse... Click to collapse... Hover to expand... Click to expand...
  4. LordLiverpool

    LordLiverpool Well-Known Member

    Joined:
    Dec 27, 2014
    Messages:
    53
    Likes Received:
    9
    Trophy Points:
    8
    cPanel Access Level:
    Root Administrator
    OK just to give an update on this thread, so hopefully it helps someone else in the future.

    Apologies in advance to any Level 18 Fire Breathing +5 SysAdmins; this is noob stuff :)

    The cPanel team examined my server and determined that Apache had ran out of Semaphores.

    Each time HTTPD crashed it didn't release the semaphores and eventually my server ran out (I had 128). The cPanel team kindly cleared down the "locked" semaphores as follows:

    Code:
    [19:48:55 mail root@9019765 ~/cptechsjt]cPs# for i in `ipcs | grep admin | awk {'print $2'}`; do ipcrm -s $i; done
    [19:49:06 mail root@9019765 ~/cptechsjt]cPs# /scripts/restartsrv_httpd === 
    And the support analyst advised to read this article:

    Apache: No space left on device: Couldn't create accept lock - major.io

    After reading it, I googled for corroboratory advice and found this article:

    Semaphore limits and many Apache instances on Linux | End Point

    So I decide to increase the number of semaphores from 128 to 256 as follows:


    How many semaphores do I have?

    Code:
    # cat /proc/sys/kernel/sem
    250 32000 32 128
    
    Answer = 128

    Add this line to /etc/sysctl.conf:

    Code:
    kernel.sem = 500 64000 64 256
    
    To make the change take immediate effect, issue this command:

    Code:
    # sysctl -p
    
    Alternatively perform a graceful reboot of your server, (If you can.)

    So how many semaphores do I have now?

    Code:
    # cat /proc/sys/kernel/sem
    500     64000   64      256
    
    You can also check the number of "running" semaphores with this command:

    Code:
    # ipcs -s
    
    I also noticed that one website (wordpress) was creating a disproportionate number of semaphores compared to the other websites (also wordpress) on my server, so it must be a rogue plugin. I will investigate further.

    Thank you to great team at cPanel for helping me out, its genuinely appreciated.

    Best Regards

    Mike
     
    Stop hovering to collapse... Click to collapse... Hover to expand... Click to expand...
    cPanelMichael likes this.
  5. oprezy

    oprezy Registered

    Joined:
    Mar 2, 2015
    Messages:
    3
    Likes Received:
    0
    Trophy Points:
    1
    cPanel Access Level:
    Website Owner
    Am presently facing similar issues with my web host
     
  6. cPanelMichael

    cPanelMichael Technical Support Community Manager
    Staff Member

    Joined:
    Apr 11, 2011
    Messages:
    44,803
    Likes Received:
    1,898
    Trophy Points:
    363
    cPanel Access Level:
    Root Administrator
    Twitter:
    Hello @oprezy,

    Have you reported the issue to your web hosting provider so they can take a closer look?

    Thank you.
     
    Stop hovering to collapse... Click to collapse... Hover to expand... Click to expand...
Loading...

Share This Page

  1. This site uses cookies to help personalise content, tailor your experience and to keep you logged in if you register.
    By continuing to use this site, you are consenting to our use of cookies.
    Dismiss Notice