Please whitelist cPanel in your adblocker so that you’re able to see our version release promotions, thanks!

The Community Forums

Interact with an entire community of cPanel & WHM users!

Apache crashes when the server load reaches 6%.

Discussion in 'Workarounds and Optimization' started by Clixer, May 22, 2018.

Tags:
  1. Clixer

    Clixer Active Member

    Joined:
    Feb 24, 2018
    Messages:
    30
    Likes Received:
    1
    Trophy Points:
    8
    Location:
    Australia
    cPanel Access Level:
    Root Administrator
    Hello,

    I run a website with a large number of posts (about 500,000) and whenever Google bots come to index the posts, the server load will exceed 6% and apache will crash. Surely, apache will self- start and a few minutes later will crash again.

    I run a VPS with 5 processors and 5GB of RAM along with PHP-FPM 7.1 and MySQL. Memcached server and client are also installed. I am not aware if there any other PHP cache module is installed.

    Below is the output of the top command:

    Code:
    top - 12:07:16 up 2 days, 15:12,  1 user,  load average: 6.63, 6.39, 5.91
    Tasks: 199 total,   2 running, 196 sleeping,   0 stopped,   1 zombie
    %Cpu(s): 48.5 us, 47.2 sy,  0.0 ni,  0.6 id,  0.0 wa,  0.0 hi,  0.3 si,  3.3 st
    KiB Mem :  4931796 total,   120748 free,  2632020 used,  2179028 buff/cache
    KiB Swap:  4980732 total,  4715004 free,   265728 used.  1705944 avail Mem
    
       PID USER      PR  NI    VIRT    RES    SHR S  %CPU %MEM     TIME+ COMMAND
      1181 mysql     20   0 2544528 242456   4980 S 480.1  4.9   4487:20 mysqld
    100796 adb98765  20   0  546476  44700  16540 S  10.9  0.9   0:02.74 php-fpm
      1356 memcach+  20   0  566320  65232    944 S   4.0  1.3  48:01.76 memcached
     53625 root      20   0  183584  27560   2156 S   0.3  0.6   2:17.95 lfd - sleeping
         1 root      20   0  193908   5216   3000 S   0.0  0.1   4:27.34 systemd
         2 root      20   0       0      0      0 S   0.0  0.0   0:00.32 kthreadd
         3 root      20   0       0      0      0 S   0.0  0.0   0:00.55 ksoftirqd/0
         5 root       0 -20       0      0      0 S   0.0  0.0   0:00.00 kworker/0:0H
         7 root      rt   0       0      0      0 S   0.0  0.0   0:00.47 migration/0
         8 root      20   0       0      0      0 S   0.0  0.0   0:00.00 rcu_bh
         9 root      20   0       0      0      0 R   0.0  0.0   2:33.64 rcu_sched
        10 root       0 -20       0      0      0 S   0.0  0.0   0:00.00 lru-add-drain
        11 root      rt   0       0      0      0 S   0.0  0.0   0:01.85 watchdog/0
        12 root      rt   0       0      0      0 S   0.0  0.0   0:01.72 watchdog/1
        13 root      rt   0       0      0      0 S   0.0  0.0   0:00.54 migration/1
        14 root      20   0       0      0      0 S   0.0  0.0   0:00.59 ksoftirqd/1
        16 root       0 -20       0      0      0 S   0.0  0.0   0:00.00 kworker/1:0H
        17 root      rt   0       0      0      0 S   0.0  0.0   0:01.67 watchdog/2
        18 root      rt   0       0      0      0 S   0.0  0.0   0:00.42 migration/2
        19 root      20   0       0      0      0 S   0.0  0.0   0:00.54 ksoftirqd/2
        21 root       0 -20       0      0      0 S   0.0  0.0   0:00.00 kworker/2:0H
        22 root      rt   0       0      0      0 S   0.0  0.0   0:01.70 watchdog/3
        23 root      rt   0       0      0      0 S   0.0  0.0   0:00.54 migration/3
        24 root      20   0       0      0      0 S   0.0  0.0   0:00.54 ksoftirqd/3
        26 root       0 -20       0      0      0 S   0.0  0.0   0:00.00 kworker/3:0H
        27 root      rt   0       0      0      0 S   0.0  0.0   0:01.72 watchdog/4
        28 root      rt   0       0      0      0 S   0.0  0.0   0:00.44 migration/4
        29 root      20   0       0      0      0 S   0.0  0.0   0:01.03 ksoftirqd/4
    Below is the apache status log:

    Code:
    Server Version: Apache/2.4.33 (cPanel) OpenSSL/1.0.2o mod_bwlimited/1.4
    Server MPM: event
    Server Built: Apr 23 2018 21:10:29
    
    Current Time: Wednesday, 23-May-2018 12:25:25 AEST
    Restart Time: Wednesday, 23-May-2018 05:58:31 AEST
    Parent Server Config. Generation: 9
    Parent Server MPM Generation: 8
    Server uptime: 6 hours 26 minutes 53 seconds
    Server load: 5.92 6.16 5.99
    Total accesses: 8622 - Total Traffic: 164.9 MB
    CPU Usage: u171.78 s16.27 cu0 cs0 - .81% CPU load
    .371 requests/sec - 7.3 kB/second - 19.6 kB/request
    63 requests currently being processed, 87 idle workers
    
    Slot    PID    Stopping    Connections     Threads    Async connections
    total    accepting    busy    idle    writing    keep-alive    closing
    0    90890    yes (old gen)    22    no    0    0    0    0    0
    1    90891    yes (old gen)    23    no    0    0    0    0    0
    2    90892    yes (old gen)    19    no    0    0    0    0    0
    3    90893    yes (old gen)    23    no    0    0    0    0    0
    4    90894    yes (old gen)    27    no    0    0    0    0    0
    5    91260    yes (old gen)    25    no    0    0    0    0    0
    6    92447    yes (old gen)    25    no    0    0    0    0    0
    7    92448    yes (old gen)    25    no    0    0    0    0    0
    8    92449    yes (old gen)    25    no    0    0    0    0    0
    9    92450    yes (old gen)    24    no    0    0    0    0    0
    10    92451    yes (old gen)    24    no    0    0    0    0    0
    11    92605    yes (old gen)    24    no    0    0    0    0    0
    12    96656    yes (old gen)    26    no    0    0    0    0    0
    13    96657    yes (old gen)    25    no    0    0    0    0    0
    14    96658    yes (old gen)    24    no    0    0    0    0    0
    15    96659    yes (old gen)    24    no    0    0    0    0    0
    16    96660    yes (old gen)    25    no    0    0    0    0    0
    17    96820    yes (old gen)    25    no    0    0    0    0    0
    18    84498    yes (old gen)    24    no    0    0    0    0    0
    19    101429    no    11    yes    10    15    0    1    0
    20    84500    yes (old gen)    24    no    0    0    0    0    0
    21    84501    yes (old gen)    24    no    0    0    0    0    0
    22    84502    yes (old gen)    25    no    0    0    0    0    0
    23    84653    yes (old gen)    25    no    0    0    0    0    0
    24    101430    no    15    yes    14    11    0    0    0
    25    101431    no    15    yes    14    11    0    0    0
    26    101432    no    10    yes    10    15    0    0    0
    27    101433    no    12    yes    11    14    0    0    1
    28    101584    no    5    yes    4    21    0    0    0
    Sum    29    23    625         63    87    0    1    1
    
    GGGGGGGG.GGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGG
    GGG.GGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGG
    GGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGG
    GGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGG
    GGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGG
    GGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGG
    GGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGG
    GW___W_W____WW_W_W_WW_W___GGGGGGGGWW___WWW_WWWW__WWW_W___W_W_WW_
    _W__WW_WW__WWWW_W__W_WW_____WW_WW_WW__W_W____W__W_WW_WW_W_____W_
    W_WW__WW_____________W____W____
    
    Scoreboard Key:
    "_" Waiting for Connection, "S" Starting up, "R" Reading Request,
    "W" Sending Reply, "K" Keepalive (read), "D" DNS Lookup,
    "C" Closing connection, "L" Logging, "G" Gracefully finishing,
    "I" Idle cleanup of worker, "." Open slot with no current process
    I would like to receive some feedback from the community in order to optimise apache, mysql and php-fpm.

    Many thanks.
     
  2. Clixer

    Clixer Active Member

    Joined:
    Feb 24, 2018
    Messages:
    30
    Likes Received:
    1
    Trophy Points:
    8
    Location:
    Australia
    cPanel Access Level:
    Root Administrator
    Hello folks,

    Just learnt that the apache crashing issue may be related to "broken pipe, error writing to error". Please see the log below:

    Code:
    [Wed May 23 12:15:06.795489 2018] [log_config:warn] [pid 84499:tid 139816469452544] (32)Broken pipe: [client 103.1.205.107:37260] AH00646: Error writing to |/usr/local/cpanel/bin/splitlogs --main
    if this is the cause then how do I fix this issue?

    Thanks for help.
     
  3. Clixer

    Clixer Active Member

    Joined:
    Feb 24, 2018
    Messages:
    30
    Likes Received:
    1
    Trophy Points:
    8
    Location:
    Australia
    cPanel Access Level:
    Root Administrator
    Hello folks. This is an update on the issue that apache is crashing every 10 to 20 minutes when the server load reaches about 6%. Have been searching the net to resolve the error about "broken pipe" issue and have seen some recommendations to add the directives "EnableMMAP off" and "EnableSendfile off" in the top section of httpd.conf file, which I did but the issue remains.

    Any suggestions?

    Thanks.
     
  4. cPanelLauren

    cPanelLauren Forums Analyst II
    Staff Member

    Joined:
    Nov 14, 2017
    Messages:
    3,065
    Likes Received:
    214
    Trophy Points:
    173
    Location:
    Houston
    cPanel Access Level:
    DataCenter Provider
    Hi @Clixer

    That error indicates that there's something going on when writing to the splitlogs. since /etc/apache2/logs is symlinked to /var/log/apache2/logs can you tell me if the symlink is present:

    Code:
    stat /etc/apache2/logs
    You might also want to review the documentation here: The splitlogs Binary - Version 68 Documentation - cPanel Documentation

    Can you also check the apache error logs for the following for me:
    Code:
    grep Max /etc/apache2/log/error_log
     
    Stop hovering to collapse... Click to collapse... Hover to expand... Click to expand...
  5. Clixer

    Clixer Active Member

    Joined:
    Feb 24, 2018
    Messages:
    30
    Likes Received:
    1
    Trophy Points:
    8
    Location:
    Australia
    cPanel Access Level:
    Root Administrator
    Hi @cPanelLauren. The outputs are below:

    Code:
    [root@sputnik ~]# stat /etc/apache2/logs
    File: ‘/etc/apache2/logs’ -> ‘../../var/log/apache2’
    Size: 21              Blocks: 0          IO Block: 4096   symbolic link
    Device: fd00h/64768d    Inode: 4124129     Links: 1
    Access: (0777/lrwxrwxrwx)  Uid: (    0/    root)   Gid: (    0/    root)
    Context: system_u:object_r:httpd_config_t:s0
    Access: 2018-05-24 05:57:58.879356210 +1000
    Modify: 2018-04-25 05:57:56.668379769 +1000
    Change: 2018-04-25 05:57:56.668379769 +1000
    Birth: -
    
    And for the error log:

    Code:
    [root@sputnik ~]# grep Max /etc/apache2/log/error_log
    grep: /etc/apache2/log/error_log: No such file or directory
    
    I am not sure why there is no error log. I am not sure what I am doing so I am relying upon your help.

    There is no directory called log in the /etc/apache2/ folder:

    Code:
    [root@sputnik ~]# cd /etc/apache2/
    [root@sputnik apache2]# ls -l
    total 8
    drwxr-xr-x. 2 root root   36 Apr 24 07:12 bin
    drwxr-xr-x. 2 root root  103 May 24 06:03 conf
    drwxr-xr-x. 7 root root  269 May 23 05:58 conf.d
    drwxr-xr-x. 2 root root 4096 May 23 05:58 conf.modules.d
    lrwxrwxrwx. 1 root root   21 Apr 25 05:57 logs -> ../../var/log/apache2
    lrwxrwxrwx. 1 root root   31 Apr 25 05:57 modules -> ../../usr/lib64/apache2/modules
    lrwxrwxrwx. 1 root root   21 Apr 25 05:57 run -> ../../var/run/apache2
    
    Here is the output for grep Max /etc/apache2/logs/error_log:

    Code:
    [Tue May 22 21:15:45.645180 2018] [pagespeed:error] [pid 118175:tid 140090198083328] [mod_pagespeed 1.11.33.4-0 @118175] /var/mod_pagespeed/cache/v3/example.com/https,3A/,2Fwww.example.com/wp-content/plugins/w3-total-cache/Cdn_Plugin_WidgetMaxCdn_View.js,3Fver=1.0,.tempYhGbbd:0: opening temp file: No such file or directory
    [Tue May 22 21:15:45.645492 2018] [pagespeed:error] [pid 118175:tid 140090223261440] [mod_pagespeed 1.11.33.4-0 @118175] Could not create directories for file /var/mod_pagespeed/cache/rname/ji_iSyw0W7OG6bbUTqWEw0u/https,3A/,2Fwww.example.com/wp-content/plugins/w3-total-cache/Cdn_Plugin_WidgetMaxCdn_View.js,3Fver=1.0,40,40_,.temp
    [Tue May 22 21:15:45.645519 2018] [pagespeed:error] [pid 118175:tid 140090223261440] [mod_pagespeed 1.11.33.4-0 @118175] /var/mod_pagespeed/cache/rname/ji_iSyw0W7OG6bbUTqWEw0u/https,3A/,2Fwww.example.com/wp-content/plugins/w3-total-cache/Cdn_Plugin_WidgetMaxCdn_View.js,3Fver=1.0,40,40_,.temphVdbfb:0: opening temp file: No such file or directory
    [Tue May 22 21:15:45.710382 2018] [pagespeed:error] [pid 118175:tid 140090198083328] [mod_pagespeed 1.11.33.4-0 @118175] Could not create directories for file /var/mod_pagespeed/cache/v3/example.com/https,3A/,2Fwww.example.com/wp-content/plugins/w3-total-cache/Cdn_Plugin_WidgetMaxCdn_View.js,3Fver=1.0,.temp
    [Tue May 22 21:15:45.710413 2018] [pagespeed:error] [pid 118175:tid 140090198083328] [mod_pagespeed 1.11.33.4-0 @118175] /var/mod_pagespeed/cache/v3/example.com/https,3A/,2Fwww.example.com/wp-content/plugins/w3-total-cache/Cdn_Plugin_WidgetMaxCdn_View.js,3Fver=1.0,.tempA54InC:0: opening temp file: No such file or directory
    [Tue May 22 21:15:45.710731 2018] [pagespeed:error] [pid 118175:tid 140090223261440] [mod_pagespeed 1.11.33.4-0 @118175] Could not create directories for file /var/mod_pagespeed/cache/rname/ce_iSyw0W7OG6bbUTqWEw0u/https,3A/,2Fwww.example.com/wp-content/plugins/w3-total-cache/Cdn_Plugin_WidgetMaxCdn_View.js,3Fver=1.0,40,40_,.temp
    [Tue May 22 21:15:45.710762 2018] [pagespeed:error] [pid 118175:tid 140090223261440] [mod_pagespeed 1.11.33.4-0 @118175] /var/mod_pagespeed/cache/rname/ce_iSyw0W7OG6bbUTqWEw0u/https,3A/,2Fwww.example.com/wp-content/plugins/w3-total-cache/Cdn_Plugin_WidgetMaxCdn_View.js,3Fver=1.0,40,40_,.tempdQOsDA:0: opening temp file: No such file or directory
    [Wed May 23 00:03:04.749731 2018] [mpm_event:error] [pid 1621:tid 140090795137152] AH00484: server reached MaxRequestWorkers setting, consider raising the MaxRequestWorkers setting
    [Wed May 23 06:26:55.349947 2018] [mpm_event:error] [pid 75732:tid 139816991602816] AH00484: server reached MaxRequestWorkers setting, consider raising the MaxRequestWorkers setting
    [Wed May 23 13:03:48.254622 2018] [pagespeed:error] [pid 104012:tid 139816519808768] [mod_pagespeed 1.11.33.4-0 @104012] /var/mod_pagespeed/cache/prop_page/https,3A/,2Fwww.example.com/_iSyw0W7OG6,40Desktop,40beacon_cohort,.tempFMaxch:0: opening temp file: No such file or directory
    [Wed May 23 20:01:22.366287 2018] [mpm_event:error] [pid 1603:tid 140137861802112] AH00484: server reached MaxRequestWorkers setting, consider raising the MaxRequestWorkers setting
    [Wed May 23 21:36:19.277172 2018] [pagespeed:error] [pid 10604:tid 140123828049664] [mod_pagespeed 1.11.33.4-0 @10604] /var/mod_pagespeed/cache/prop_page/http,3A/,2Fwww.example.com/jobs/part-time/manningham/2018/05/04/occupational-therapist-grade-2-adelaide-sa/_iSyw0W7OG6,40Desktop,40beacon_cohort,.templMaxyR:0: opening temp file: No such file or directory
    [Wed May 23 22:12:07.132263 2018] [mpm_event:error] [pid 10595:tid 140124362528896] AH00484: server reached MaxRequestWorkers setting, consider raising the MaxRequestWorkers setting
    [Wed May 23 22:31:53.013149 2018] [mpm_event:error] [pid 14399:tid 140002601973888] AH00484: server reached MaxRequestWorkers setting, consider raising the MaxRequestWorkers setting
    
     
    #5 Clixer, May 23, 2018
    Last edited by a moderator: May 24, 2018
  6. Clixer

    Clixer Active Member

    Joined:
    Feb 24, 2018
    Messages:
    30
    Likes Received:
    1
    Trophy Points:
    8
    Location:
    Australia
    cPanel Access Level:
    Root Administrator
    Hi @cPanelLauren,

    I increased the value of Max Request Workers from 150 to 250, this helped in removing the error from apache log. The remaining error is about pagespeed not able to create directory. I am still looking into this. The server load is still too high, ranges from 3% to 8%.

    Below is the output of the apache log:

    Code:
    [root@sputnik ~]# grep Max /etc/apache2/logs/error_log
    [pagespeed:error] [pid 78352:tid 139670566356736] [mod_pagespeed 1.11.33.4-0 @78352] Could not create directories for file /var/mod_pagespeed/cache/v3/example.com/https,3A/,2Fwww.example.com/wp-content/plugins/w3-total-cache/Cdn_Plugin_WidgetMaxCdn_View.css,3Fver=4.9.6,.temp
    [Thu May 24 15:01:55.543972 2018] [pagespeed:error] [pid 78352:tid 139670566356736] [mod_pagespeed 1.11.33.4-0 @78352] /var/mod_pagespeed/cache/v3/example.com/https,3A/,2Fwww.example.com/wp-content/plugins/w3-total-cache/Cdn_Plugin_WidgetMaxCdn_View.css,3Fver=4.9.6,.temp7ZBhA3:0: opening temp file: No such file or directory
    [Thu May 24 15:01:55.549991 2018] [pagespeed:error] [pid 78352:tid 139670583142144] [mod_pagespeed 1.11.33.4-0 @78352] Could not create directories for file /var/mod_pagespeed/cache/rname/cf_iSyw0W7OG6bbUTqWEw0u/https,3A/,2Fwww.example.com/wp-content/plugins/w3-total-cache/Cdn_Plugin_WidgetMaxCdn_View.css,3Fver=4.9.6,40A.,40.A_,.temp
    [Thu May 24 15:01:55.550029 2018] [pagespeed:error] [pid 78352:tid 139670583142144] [mod_pagespeed 1.11.33.4-0 @78352] /var/mod_pagespeed/cache/rname/cf_iSyw0W7OG6bbUTqWEw0u/https,3A/,2Fwww.example.com/wp-content/plugins/w3-total-cache/Cdn_Plugin_WidgetMaxCdn_View.css,3Fver=4.9.6,40A.,40.A_,.tempa3gVio:0: opening temp file: No such file or directory
    [Thu May 24 15:01:55.614948 2018] [pagespeed:error] [pid 78352:tid 139670566356736] [mod_pagespeed 1.11.33.4-0 @78352] Could not create directories for file /var/mod_pagespeed/cache/v3/example.com/https,3A/,2Fwww.example.com/wp-content/plugins/w3-total-cache/Cdn_Plugin_WidgetMaxCdn_View.js,3Fver=1.0,.temp
    [Thu May 24 15:01:55.614979 2018] [pagespeed:error] [pid 78352:tid 139670566356736] [mod_pagespeed 1.11.33.4-0 @78352] /var/mod_pagespeed/cache/v3/example.com/https,3A/,2Fwww.example.com/wp-content/plugins/w3-total-cache/Cdn_Plugin_WidgetMaxCdn_View.js,3Fver=1.0,.temp9uxmWp:0: opening temp file: No such file or directory
    [Thu May 24 15:01:55.694630 2018] [pagespeed:error] [pid 78352:tid 139670566356736] [mod_pagespeed 1.11.33.4-0 @78352] Could not create directories for file /var/mod_pagespeed/cache/v3/example.com/https,3A/,2Fwww.example.com/wp-content/plugins/w3-total-cache/Cdn_Plugin_WidgetMaxCdn_View.css,3Fver=4.9.6,.temp
    [Thu May 24 15:01:55.694667 2018] [pagespeed:error] [pid 78352:tid 139670566356736] [mod_pagespeed 1.11.33.4-0 @78352] /var/mod_pagespeed/cache/v3/example.com/https,3A/,2Fwww.example.com/wp-content/plugins/w3-total-cache/Cdn_Plugin_WidgetMaxCdn_View.css,3Fver=4.9.6,.tempEjSAOQ:0: opening temp file: No such file or directory
    [Thu May 24 15:01:55.699071 2018] [pagespeed:error] [pid 78352:tid 139670583142144] [mod_pagespeed 1.11.33.4-0 @78352] Could not create directories for file /var/mod_pagespeed/cache/rname/ci_iSyw0W7OG6bbUTqWEw0u/https,3A/,2Fwww.example.com/wp-content/plugins/w3-total-cache/Cdn_Plugin_WidgetMaxCdn_View.css,3Fver=4.9.6,40,40_,.temp
    [Thu May 24 15:01:55.699109 2018] [pagespeed:error] [pid 78352:tid 139670583142144] [mod_pagespeed 1.11.33.4-0 @78352] /var/mod_pagespeed/cache/rname/ci_iSyw0W7OG6bbUTqWEw0u/https,3A/,2Fwww.example.com/wp-content/plugins/w3-total-cache/Cdn_Plugin_WidgetMaxCdn_View.css,3Fver=4.9.6,40,40_,.tempDgx4Xb:0: opening temp file: No such file or directory
    [Thu May 24 15:01:55.743664 2018] [pagespeed:error] [pid 78352:tid 139670566356736] [mod_pagespeed 1.11.33.4-0 @78352] Could not create directories for file /var/mod_pagespeed/cache/v3/example.com/https,3A/,2Fwww.example.com/wp-content/plugins/w3-total-cache/Cdn_Plugin_WidgetMaxCdn_View.js,3Fver=1.0,.temp
    [Thu May 24 15:01:55.743699 2018] [pagespeed:error] [pid 78352:tid 139670566356736] [mod_pagespeed 1.11.33.4-0 @78352] /var/mod_pagespeed/cache/v3/example.com/https,3A/,2Fwww.example.com/wp-content/plugins/w3-total-cache/Cdn_Plugin_WidgetMaxCdn_View.js,3Fver=1.0,.tempBrCJXY:0: opening temp file: No such file or directory
    [Thu May 24 15:01:55.746040 2018] [pagespeed:error] [pid 78352:tid 139670583142144] [mod_pagespeed 1.11.33.4-0 @78352] Could not create directories for file /var/mod_pagespeed/cache/rname/jm_iSyw0W7OG6bbUTqWEw0u/https,3A/,2Fwww.example.com/wp-content/plugins/w3-total-cache/Cdn_Plugin_WidgetMaxCdn_View.js,3Fver=1.0,40,40_,.temp
    [Thu May 24 15:01:55.746074 2018] [pagespeed:error] [pid 78352:tid 139670583142144] [mod_pagespeed 1.11.33.4-0 @78352] /var/mod_pagespeed/cache/rname/jm_iSyw0W7OG6bbUTqWEw0u/https,3A/,2Fwww.example.com/wp-content/plugins/w3-total-cache/Cdn_Plugin_WidgetMaxCdn_View.js,3Fver=1.0,40,40_,.temp2ZyFfk:0: opening temp file: No such file or directory
    [Thu May 24 15:01:55.793846 2018] [pagespeed:error] [pid 78352:tid 139670566356736] [mod_pagespeed 1.11.33.4-0 @78352] Could not create directories for file /var/mod_pagespeed/cache/v3/example.com/https,3A/,2Fwww.example.com/wp-content/plugins/w3-total-cache/Cdn_Plugin_WidgetMaxCdn_View.css,3Fver=4.9.6,.temp
    [Thu May 24 15:01:55.793900 2018] [pagespeed:error] [pid 78352:tid 139670566356736] [mod_pagespeed 1.11.33.4-0 @78352] /var/mod_pagespeed/cache/v3/example.com/https,3A/,2Fwww.example.com/wp-content/plugins/w3-total-cache/Cdn_Plugin_WidgetMaxCdn_View.css,3Fver=4.9.6,.tempPdxXi7:0: opening temp file: No such file or directory
    [Thu May 24 15:01:55.799873 2018] [pagespeed:error] [pid 78352:tid 139670583142144] [mod_pagespeed 1.11.33.4-0 @78352] Could not create directories for file /var/mod_pagespeed/cache/rname/ce_iSyw0W7OG6bbUTqWEw0u/https,3A/,2Fwww.example.com/wp-content/plugins/w3-total-cache/Cdn_Plugin_WidgetMaxCdn_View.css,3Fver=4.9.6,40,40_,.temp
    [Thu May 24 15:01:55.799905 2018] [pagespeed:error] [pid 78352:tid 139670583142144] [mod_pagespeed 1.11.33.4-0 @78352] /var/mod_pagespeed/cache/rname/ce_iSyw0W7OG6bbUTqWEw0u/https,3A/,2Fwww.example.com/wp-content/plugins/w3-total-cache/Cdn_Plugin_WidgetMaxCdn_View.css,3Fver=4.9.6,40,40_,.tempRiLAKs:0: opening temp file: No such file or directory
    [Thu May 24 15:01:55.802091 2018] [pagespeed:error] [pid 79332:tid 139670871766784] [mod_pagespeed 1.11.33.4-0 @79332] Could not create directories for file /var/mod_pagespeed/cache/v3/example.com/https,3A/,2Fwww.example.com/wp-content/plugins/w3-total-cache/Cdn_Plugin_WidgetMaxCdn_View.css,3Fver=4.9.6,.temp
    [Thu May 24 15:01:55.802133 2018] [pagespeed:error] [pid 79332:tid 139670871766784] [mod_pagespeed 1.11.33.4-0 @79332] /var/mod_pagespeed/cache/v3/example.com/https,3A/,2Fwww.examle.com/wp-content/plugins/w3-total-cache/Cdn_Plugin_WidgetMaxCdn_View.css,3Fver=4.9.6,.tempClx6Es:0: opening temp file: No such file or directory
    [Thu May 24 15:01:55.861964 2018] [pagespeed:error] [pid 79491:tid 139670767781632] [mod_pagespeed 1.11.33.4-0 @79491] Could not create directories for file /var/mod_pagespeed/cache/v3/example.com/https,3A/,2Fwww.example.com/wp-content/plugins/w3-total-cache/Cdn_Plugin_WidgetMaxCdn_View.js,3Fver=1.0,.temp
    [Thu May 24 15:01:55.861992 2018] [pagespeed:error] [pid 79491:tid 139670767781632] [mod_pagespeed 1.11.33.4-0 @79491] /var/mod_pagespeed/cache/v3/example.com/https,3A/,2Fwww.example.com/wp-content/plugins/w3-total-cache/Cdn_Plugin_WidgetMaxCdn_View.js,3Fver=1.0,.temptLTX5r:0: opening temp file: No such file or directory
    [Thu May 24 15:01:55.871681 2018] [pagespeed:error] [pid 78352:tid 139670566356736] [mod_pagespeed 1.11.33.4-0 @78352] Could not create directories for file /var/mod_pagespeed/cache/v3/example.com/https,3A/,2Fwww.example.com/wp-content/plugins/w3-total-cache/Cdn_Plugin_WidgetMaxCdn_View.js,3Fver=1.0,.temp
    [Thu May 24 15:01:55.871710 2018] [pagespeed:error] [pid 78352:tid 139670566356736] [mod_pagespeed 1.11.33.4-0 @78352] /var/mod_pagespeed/cache/v3/example.com/https,3A/,2Fwww.example.com/wp-content/plugins/w3-total-cache/Cdn_Plugin_WidgetMaxCdn_View.js,3Fver=1.0,.temp4KJlwk:0: opening temp file: No such file or directory
    [Thu May 24 15:01:55.873086 2018] [pagespeed:error] [pid 78352:tid 139670583142144] [mod_pagespeed 1.11.33.4-0 @78352] Could not create directories for file /var/mod_pagespeed/cache/rname/ji_iSyw0W7OG6bbUTqWEw0u/https,3A/,2Fwww.example.com/wp-content/plugins/w3-total-cache/Cdn_Plugin_WidgetMaxCdn_View.js,3Fver=1.0,40,40_,.temp
    [Thu May 24 15:01:55.873113 2018] [pagespeed:error] [pid 78352:tid 139670583142144] [mod_pagespeed 1.11.33.4-0 @78352] /var/mod_pagespeed/cache/rname/ji_iSyw0W7OG6bbUTqWEw0u/https,3A/,2Fwww.example.com/wp-content/plugins/w3-total-cache/Cdn_Plugin_WidgetMaxCdn_View.js,3Fver=1.0,40,40_,.temp4nIVP1:0: opening temp file: No such file or directory
    [Thu May 24 15:01:55.913908 2018] [pagespeed:error] [pid 78352:tid 139670566356736] [mod_pagespeed 1.11.33.4-0 @78352] Could not create directories for file /var/mod_pagespeed/cache/v3/example.com/https,3A/,2Fwww.example.com/wp-content/plugins/w3-total-cache/Cdn_Plugin_WidgetMaxCdn_View.js,3Fver=1.0,.temp
    [Thu May 24 15:01:55.913935 2018] [pagespeed:error] [pid 78352:tid 139670566356736] [mod_pagespeed 1.11.33.4-0 @78352] /var/mod_pagespeed/cache/v3/example.com/https,3A/,2Fwww.example.com/wp-content/plugins/w3-total-cache/Cdn_Plugin_WidgetMaxCdn_View.js,3Fver=1.0,.temprpyLLS:0: opening temp file: No such file or directory
    [Thu May 24 15:01:55.914245 2018] [pagespeed:error] [pid 78352:tid 139670583142144] [mod_pagespeed 1.11.33.4-0 @78352] Could not create directories for file /var/mod_pagespeed/cache/rname/ce_iSyw0W7OG6bbUTqWEw0u/https,3A/,2Fwww.example.com/wp-content/plugins/w3-total-cache/Cdn_Plugin_WidgetMaxCdn_View.js,3Fver=1.0,40,40_,.temp
    [Thu May 24 15:01:55.914296 2018] [pagespeed:error] [pid 78352:tid 139670583142144] [mod_pagespeed 1.11.33.4-0 @78352] /var/mod_pagespeed/cache/rname/ce_iSyw0W7OG6bbUTqWEw0u/https,3A/,2Fwww.example.com/wp-content/plugins/w3-total-cache/Cdn_Plugin_WidgetMaxCdn_View.js,3Fver=1.0,40,40_,.temp5ZTYxe:0: opening temp file: No such file or directory
    
    Any ideas how I could solve the pagespeed issue? Thanks for help.
     
    #6 Clixer, May 24, 2018
    Last edited by a moderator: May 24, 2018
  7. Clixer

    Clixer Active Member

    Joined:
    Feb 24, 2018
    Messages:
    30
    Likes Received:
    1
    Trophy Points:
    8
    Location:
    Australia
    cPanel Access Level:
    Root Administrator
    Hi @cPanelLauren,

    The pagespeed issue has now disappeared. I have provided the folder ownership to httpd at /var/mod_pagespeed/cache/ so httpd could write to this folder.

    The Max Request Worker issue is back, currently set to 250. I will keep working on it but appreciate some advise from you. Thanks.
     
  8. Clixer

    Clixer Active Member

    Joined:
    Feb 24, 2018
    Messages:
    30
    Likes Received:
    1
    Trophy Points:
    8
    Location:
    Australia
    cPanel Access Level:
    Root Administrator
    Hi @cPanelLauren,

    I have been trying to optimise apache through "Home » Service Configuration » Apache Configuration » Global Configuration" but later learnt that this is for MPM Prefork only. How do I optimise apache for mpm-event?

    Thanks for help.
     
  9. cPanelLauren

    cPanelLauren Forums Analyst II
    Staff Member

    Joined:
    Nov 14, 2017
    Messages:
    3,065
    Likes Received:
    214
    Trophy Points:
    173
    Location:
    Houston
    cPanel Access Level:
    DataCenter Provider
    Hi @Clixer

    Actually, that's incorrect and I'm opening a case with our documentation team on this. It looks like the note has been added to too many directives. Modifying these is definitely ok with prefork, event and worker.


    Thanks!
     
    Stop hovering to collapse... Click to collapse... Hover to expand... Click to expand...
  10. Clixer

    Clixer Active Member

    Joined:
    Feb 24, 2018
    Messages:
    30
    Likes Received:
    1
    Trophy Points:
    8
    Location:
    Australia
    cPanel Access Level:
    Root Administrator
    Thanks @cPanelLauren.

    The server is generating heaps of errors about "scoreboard is full, not at MaxRequestWorkers. Increase ServerLimit", these errors are ongoing and the apahce is crashing. Output is below:

    Code:
    [Fri May 25 09:29:01.913814 2018] [mpm_event:error] [pid 109589:tid 140179259779200] AH03490: scoreboard is full, not at MaxRequestWorkers.Increase ServerLimit.
    [Fri May 25 09:29:02.920947 2018] [mpm_event:error] [pid 109589:tid 140179259779200] AH03490: scoreboard is full, not at MaxRequestWorkers.Increase ServerLimit.
    [Fri May 25 09:29:03.929326 2018] [mpm_event:error] [pid 109589:tid 140179259779200] AH03490: scoreboard is full, not at MaxRequestWorkers.Increase ServerLimit.
    [Fri May 25 09:29:04.935802 2018] [mpm_event:error] [pid 109589:tid 140179259779200] AH03490: scoreboard is full, not at MaxRequestWorkers.Increase ServerLimit.
    [Fri May 25 09:29:05.940789 2018] [mpm_event:error] [pid 109589:tid 140179259779200] AH03490: scoreboard is full, not at MaxRequestWorkers.Increase ServerLimit.
    [Fri May 25 09:29:06.948809 2018] [mpm_event:error] [pid 109589:tid 140179259779200] AH03490: scoreboard is full, not at MaxRequestWorkers.Increase ServerLimit.
    [Fri May 25 09:29:07.954094 2018] [mpm_event:error] [pid 109589:tid 140179259779200] AH03490: scoreboard is full, not at MaxRequestWorkers.Increase ServerLimit.
    [Fri May 25 09:29:08.960791 2018] [mpm_event:error] [pid 109589:tid 140179259779200] AH03490: scoreboard is full, not at MaxRequestWorkers.Increase ServerLimit.
    [Fri May 25 09:29:09.967799 2018] [mpm_event:error] [pid 109589:tid 140179259779200] AH03490: scoreboard is full, not at MaxRequestWorkers.Increase ServerLimit.
    [Fri May 25 09:29:10.970795 2018] [mpm_event:error] [pid 109589:tid 140179259779200] AH03490: scoreboard is full, not at MaxRequestWorkers.Increase ServerLimit.
    [Fri May 25 09:29:11.976205 2018] [mpm_event:error] [pid 109589:tid 140179259779200] AH03490: scoreboard is full, not at MaxRequestWorkers.Increase ServerLimit.
    [Fri May 25 09:29:12.978801 2018] [mpm_event:error] [pid 109589:tid 140179259779200] AH03490: scoreboard is full, not at MaxRequestWorkers.Increase ServerLimit.
    [Fri May 25 09:29:13.998792 2018] [mpm_event:error] [pid 109589:tid 140179259779200] AH03490: scoreboard is full, not at MaxRequestWorkers.Increase ServerLimit.
    [Fri May 25 09:29:14.999933 2018] [mpm_event:error] [pid 109589:tid 140179259779200] AH03490: scoreboard is full, not at MaxRequestWorkers.Increase ServerLimit.
    [Fri May 25 09:29:16.001609 2018] [mpm_event:error] [pid 109589:tid 140179259779200] AH03490: scoreboard is full, not at MaxRequestWorkers.Increase ServerLimit.
    [Fri May 25 09:29:17.002797 2018] [mpm_event:error] [pid 109589:tid 140179259779200] AH03490: scoreboard is full, not at MaxRequestWorkers.Increase ServerLimit.
    The following is the current httpd configuration:

    Code:
    StartServers 3
    <IfModule prefork.c>
    MinSpareServers 3
    MaxSpareServers 16
    </IfModule>
    ServerLimit 16
    MaxRequestWorkers 400
    MaxConnectionsPerChild 10000
    KeepAlive On
    KeepAliveTimeout 5
    MaxKeepAliveRequests 100
    Timeout 300
    
    what would I need to do to increase the server limit? Is there a formula I need to apply in calculating the new severlimit? And Can I make the changes directly to the httpd.conf file.
     
  11. Clixer

    Clixer Active Member

    Joined:
    Feb 24, 2018
    Messages:
    30
    Likes Received:
    1
    Trophy Points:
    8
    Location:
    Australia
    cPanel Access Level:
    Root Administrator
    Hi @cPanelLauren,

    In order to completely get rid of the MaxRequestWorkers hitting the maximum allowed, I had to increase its value as illustrated below:

    Code:
    StartServers 3
    <IfModule prefork.c>
    MinSpareServers 3
    MaxSpareServers 64
    </IfModule>
    ServerLimit 64
    MaxRequestWorkers 1600
    MaxConnectionsPerChild 10000
    KeepAlive On
    KeepAliveTimeout 5
    MaxKeepAliveRequests 200
    Timeout 300
    
    I could not do this change directly on WHM so I did it manually and modified the httpd.conf file. I then started httpd with the command "systemctl restart httpd". The apache has been running fine since then and hasnt crashed yet. What I have noted is that in WHM, the old apache configuration is still present. Is this a bug in WHM or I should not have manually modified the httpd.conf file.

    The second question is, is the configuration illustrated makes sense? Whilst apache is no longer crashing, the sever load ranges between 6% to 7%. Most of the resource are taken up by sqld. It is illustrated in the output below.

    Code:
    top - 16:15:20 up 1 day, 20:34,  1 user,  load average: 7.55, 6.82, 6.30
    Tasks: 208 total,   4 running, 204 sleeping,   0 stopped,   0 zombie
    %Cpu(s): 61.3 us, 34.8 sy,  0.0 ni,  0.3 id,  0.1 wa,  0.0 hi,  0.3 si,  3.4 st
    KiB Mem :  4931780 total,   110448 free,  2565548 used,  2255784 buff/cache
    KiB Swap:  4980732 total,  4690940 free,   289792 used.  1809384 avail Mem
    
       PID USER      PR  NI    VIRT    RES    SHR S  %CPU %MEM     TIME+ COMMAND
     14932 mysql     20   0 2478408 299788   5212 S 362.4  6.1   9372:09 mysqld
     52854 dotcap0+  20   0  595704  84020   6036 D  43.2  1.7   0:03.01 php-fpm
     42049 nobody    20   0 2508020  84828  16892 S  42.2  1.7   0:04.74 httpd
     52864 dotcap0+  20   0  517164  78244   4284 S  29.0  1.6   0:04.27 php-fpm
     52284 adb98765  20   0  483740  56544  16008 R   6.6  1.1   0:20.23 php-fpm
     50943 nobody    20   0 2509044  72852  13300 S   4.0  1.5   0:01.58 httpd
     51793 nobody    20   0 2508016  51536   9516 S   3.6  1.0   0:00.72 httpd
      1357 memcach+  20   0  567368  62484    848 S   2.6  1.3 110:58.19 memcached
     13133 root      20   0  182904  30192   2016 S   1.0  0.6   1:38.91 lfd - sleeping
     51205 nobody    20   0 2509044  81148  13708 S   1.0  1.6   0:04.06 httpd
     48204 nobody    20   0 2509044  62520  12632 S   0.7  1.3   0:02.50 httpd
         9 root      20   0       0      0      0 R   0.3  0.0   2:21.66 rcu_sched
       698 root      20   0   21644    948    784 S   0.3  0.0   0:39.24 irqbalance
     48855 nobody    20   0 2508020  50708  11716 S   0.3  1.0   0:01.48 httpd
     52468 nobody    20   0 2041068  39160   7328 S   0.3  0.8   0:01.19 httpd
     52794 root      20   0  160064   2308   1548 R   0.3  0.0   0:00.52 top
         1 root      20   0  193924   5148   2904 S   0.0  0.1   4:00.62 systemd
         2 root      20   0       0      0      0 S   0.0  0.0   0:00.28 kthreadd
         3 root      20   0       0      0  
    Any recommendations as to how I should optimise mysql. The output of my.cnf is below:

    Code:
    [mysqld]
    default-storage-engine=innodb
    performance-schema=0
    innodb_file_per_table=1
    innodb_buffer_pool_instances=1
    innodb_buffer_pool_size=134217728
    innodb_log_file_size=512M
    max_connections=300
    max_allowed_packet=268435456
    open_files_limit=10000
    query_cache_type=1
    query_cache_limit=256K
    query_cache_min_res_unit=2k
    query_cache_size=80M
    tmp_table_size=64M
    max_heap_table_size=64M
    slow-query-log=1
    slow-query-log-file=/var/lib/mysql/mysql-slow.log
    long_query_time=1
    local-infile=0
    # MySQL idle Connections
    wait_timeout=60
    # Skip reverse DNS lookup of clients
    skip-name-resolve
    
    Thanks for help.
     
  12. cPanelLauren

    cPanelLauren Forums Analyst II
    Staff Member

    Joined:
    Nov 14, 2017
    Messages:
    3,065
    Likes Received:
    214
    Trophy Points:
    173
    Location:
    Houston
    cPanel Access Level:
    DataCenter Provider
    Why couldn't you make the changes through WHM? Edits made directly to the Apache configuration will be overwritten, to modify the MaxRequestWorkers option you'll increase MaxClients which is available through the Global Configuration options in WHM.

    I would refer to Apache's Documentation on ServerLimit before making modifications mpm_common - Apache HTTP Server Version 2.4

    For the MySQL optimization you might want to look at major/MySQLTuner-perl

    The following thread might be a good starting point as well:
    mysqlmymonlite.sh server stats gathering tool for cPanel Server
     
    Stop hovering to collapse... Click to collapse... Hover to expand... Click to expand...
Loading...

Share This Page

  1. This site uses cookies to help personalise content, tailor your experience and to keep you logged in if you register.
    By continuing to use this site, you are consenting to our use of cookies.
    Dismiss Notice