Please whitelist cPanel in your adblocker so that you’re able to see our version release promotions, thanks!

The Community Forums

Interact with an entire community of cPanel & WHM users!

500 errors during basic tweaks in WHM

Discussion in 'General Discussion' started by LukeFive Dev, Oct 16, 2018.

  1. LukeFive Dev

    LukeFive Dev Member

    Joined:
    Sep 14, 2018
    Messages:
    18
    Likes Received:
    2
    Trophy Points:
    3
    Location:
    United States
    cPanel Access Level:
    Root Administrator
    Trying to tweak basic configuration on our VPS, keep getting 500 errors, and now the following...
    Code:
    (XID j8669k) The system failed to fork a new process because of an error: Cannot allocate memory at /usr/local/cpanel/Cpanel/ForkAsync.pm line 62.
        Cpanel::ForkAsync::do_in_child(CODE(0x4ed1658)) called at /usr/local/cpanel/Cpanel/SafeRun/Object.pm line 363
        Cpanel::SafeRun::Object::new("Cpanel::SafeRun::Object", "program", "/usr/bin/repoquery", "args", ARRAY(0x51c55a8), "before_exec", CODE(0x49e0180)) called at /usr/local/cpanel/Cpanel/SafeRun/Object.pm line 497
        Cpanel::SafeRun::Object::new_or_die("Cpanel::SafeRun::Object", "program", "/usr/bin/repoquery", "args", ARRAY(0x51c55a8), "before_exec", CODE(0x49e0180)) called at /usr/local/cpanel/Cpanel/SysPkgs/RepoQuery.pm line 93
        Cpanel::SysPkgs::RepoQuery::get_all_packages("http://httpupdate.cpanel.net/cpanelsync/repos/CentOS/6/cpanel"..., "name", "summary", "description", "version", "release", "url") called at /usr/local/cpanel/Whostmgr/Addons/RPM/Cache.pm line 162
        Whostmgr::Addons::RPM::Cache::_yum_modules() called at /usr/local/cpanel/Whostmgr/Addons/RPM/Cache.pm line 100
        Whostmgr::Addons::RPM::Cache::_LOAD_FRESH("Whostmgr::Addons::RPM::Cache") called at /usr/local/cpanel/Cpanel/CacheFile.pm line 225
        Cpanel::CacheFile::load("Whostmgr::Addons::RPM::Cache") called at /usr/local/cpanel/Whostmgr/Addons/RPM/Cache.pm line 88
        Whostmgr::Addons::RPM::Cache::load("Whostmgr::Addons::RPM::Cache") called at /usr/local/cpanel/Whostmgr/Addons/RPM.pm line 62
        Whostmgr::Addons::RPM::get_modules() called at whostmgr/bin/whostmgr2.pl line 2362
        main::manage_plugins("manage_plugins") called at /usr/local/cpanel/Whostmgr/Dispatch.pm line 230
        Whostmgr::Dispatch::_do_call("manage_plugins", HASH(0x449d180), HASH(0x44a1120)) called at /usr/local/cpanel/Whostmgr/Dispatch.pm line 144
        Whostmgr::Dispatch::dispatch("manage_plugins", 1, ARRAY(0x44a0d18), HASH(0x44a1120)) called at whostmgr/bin/whostmgr2.pl line 687
    
    
     
  2. SS-Maddy

    SS-Maddy Well-Known Member

    Joined:
    Mar 28, 2009
    Messages:
    117
    Likes Received:
    10
    Trophy Points:
    68
    cPanel Access Level:
    Root Administrator
    Hello @LukeFive Dev

    How much RAM does your VPS have ? It seems that you are running out of memory. What is the cPanel memory limit configured in "WHM >> Tweak Settings" . Try upgrading the memory of your VPS by 1GB or so and then see whether the issue persists.
     
    Stop hovering to collapse... Click to collapse... Hover to expand... Click to expand...
  3. LukeFive Dev

    LukeFive Dev Member

    Joined:
    Sep 14, 2018
    Messages:
    18
    Likes Received:
    2
    Trophy Points:
    3
    Location:
    United States
    cPanel Access Level:
    Root Administrator
    Hello @SS-Maddy :

    I made these two changes and now waiting to see what happens ...

    "Max cPanel memory" from 768MB to 1,768MB
    "Conserve memory" now OFF (before it was ON)

    I'm a relative newbie to back-end work. Please let me know if there is a good tutorial on best practices for WHM/cPanel.
     
  4. LukeFive Dev

    LukeFive Dev Member

    Joined:
    Sep 14, 2018
    Messages:
    18
    Likes Received:
    2
    Trophy Points:
    3
    Location:
    United States
    cPanel Access Level:
    Root Administrator
    While I am in the WHM, I notice something unsettling about the status of my services ...

    SERVICE STATUS
    clamd Down
    dnsadmin Down
    spamd Down
     
  5. LukeFive Dev

    LukeFive Dev Member

    Joined:
    Sep 14, 2018
    Messages:
    18
    Likes Received:
    2
    Trophy Points:
    3
    Location:
    United States
    cPanel Access Level:
    Root Administrator
    @SS-Maddy I am not sure how often this thread may be checked, so an update.
    I am now rebuilding the RPM database. Shortly before that I retrieved the error log. Here are the last 100 lines (scrubbed) ...
    Code:
    open3: fork failed: Cannot allocate memory at /usr/local/cpanel/Cpanel/Services.pm line 173.
    [2018-10-17 08:29:06 -0700] warn [whostmgrd] The subprocess (whostmgr (whostmgr)) exited with an error: The subprocess reported error number 2 when it ended. at /usr/local/cpanel/Cpanel/Server/Handlers/SubProcess.pm line 251.
            Cpanel::Server::Handlers::SubProcess::_report_subprocess_errors(Cpanel::Server::Handlers::SubProcess=HASH(0x2a8d8c8)) called at /usr/local/cpanel/Cpanel/Server/Handlers/SubProcess.pm line 109
            Cpanel::Server::Handlers::SubProcess::handler(Cpanel::Server::Handlers::SubProcess=HASH(0x2a8d8c8), "subprocess_name", "whostmgr (whostmgr)", "subprocess_read_handle", IO::Handle=GLOB(0x2a0c840), "subprocess_write_handle", IO::Handle=GLOB(0x2a8d568), "subprocess_pid_to_reap", ...) called at cpsrvd.pl line 6819
            cpanel::cpsrvd::cpHandler("app", "whostmgr", "json", 0, "document", "./scripts/resmysql") called at cpsrvd.pl line 5979
            cpanel::cpsrvd::dodoc_whostmgrd() called at cpsrvd.pl line 1683
            cpanel::cpsrvd::dodoc(HASH(0x1347a00)) called at cpsrvd.pl line 1456
            cpanel::cpsrvd::handle_one_connection(7) called at cpsrvd.pl line 894
            cpanel::cpsrvd::script() called at cpsrvd.pl line 325
    [2018-10-17 08:29:07 -0700] warn [cpsrvd] Failed to fork(): for main connection loop: Cannot allocate memory at cpsrvd.pl line 898.
            cpanel::cpsrvd::script() called at cpsrvd.pl line 325
    [2018-10-17 08:29:07 -0700] warn [cpsrvd] Failed to fork(): for main connection loop: Cannot allocate memory at cpsrvd.pl line 898.
            cpanel::cpsrvd::script() called at cpsrvd.pl line 325
    [2018-10-17 08:29:08 -0700] warn [cpsrvd] Failed to fork(): for main connection loop: Cannot allocate memory at cpsrvd.pl line 898.
            cpanel::cpsrvd::script() called at cpsrvd.pl line 325
    [2018-10-17 08:29:08 -0700] warn [cpsrvd] Failed to fork(): for main connection loop: Cannot allocate memory at cpsrvd.pl line 898.
            cpanel::cpsrvd::script() called at cpsrvd.pl line 325
    [2018-10-17 08:29:08 -0700] warn [cpsrvd] Failed to fork(): for main connection loop: Cannot allocate memory at cpsrvd.pl line 898.
            cpanel::cpsrvd::script() called at cpsrvd.pl line 325
    [2018-10-17 08:29:08 -0700] warn [cpsrvd] Failed to fork(): for main connection loop: Cannot allocate memory at cpsrvd.pl line 898.
            cpanel::cpsrvd::script() called at cpsrvd.pl line 325
    [2018-10-17 08:29:08 -0700] warn [cpsrvd] Failed to fork(): for main connection loop: Cannot allocate memory at cpsrvd.pl line 898.
            cpanel::cpsrvd::script() called at cpsrvd.pl line 325
    Out of memory!
    Out of memory!
    [2018-10-17 08:29:09 -0700] warn [cpsrvd] Failed to fork(): for main connection loop: Cannot allocate memory at cpsrvd.pl line 898.
            cpanel::cpsrvd::script() called at cpsrvd.pl line 325
    [2018-10-17 08:32:49 -0700] warn [tailwatchd] [21788] TIME 1539790369 UID 0 clobbered stale lock file “/var/run/chkservd.checking.lock” (NAME“tailwatchd - chkservd”, UID 0) written by PID 19903 at 1539790068 at /usr/local/cpanel/Cpanel/SafeFile.pm line 913.
            Cpanel::SafeFile::_lock_wait("/var/run/chkservd.checking") called at /usr/local/cpanel/Cpanel/SafeFile.pm line 355
            Cpanel::SafeFile::_safelock("/var/run/chkservd.checking") called at /usr/local/cpanel/Cpanel/SafeFile.pm line 293
            Cpanel::SafeFile::safelock("/var/run/chkservd.checking") called at /usr/local/cpanel/Cpanel/TailWatch/ChkServd.pm line 252
            Cpanel::TailWatch::ChkServd::_run_checks(Cpanel::TailWatch::ChkServd=HASH(0x1bee698), Cpanel::TailWatch=HASH(0x7c3c68), 1539790369) called at /usr/local/cpanel/Cpanel/TailWatch/ChkServd.pm line 208
            Cpanel::TailWatch::ChkServd::run(Cpanel::TailWatch::ChkServd=HASH(0x1bee698), Cpanel::TailWatch=HASH(0x7c3c68), 1539790369) called at/usr/local/cpanel/Cpanel/TailWatch.pm line 748
            Cpanel::TailWatch::tail_logs(Cpanel::TailWatch=HASH(0x7c3c68)) called at /usr/local/cpanel/Cpanel/TailWatch.pm line 648
            eval {...} called at /usr/local/cpanel/Cpanel/TailWatch.pm line 616
            Cpanel::TailWatch::run(Cpanel::TailWatch=HASH(0x7c3c68)) called at libexec/tailwatch/tailwatchd.pl line 58
            main::run("--start") called at libexec/tailwatch/tailwatchd.pl line 35
    [2018-10-17 08:32:49 -0700] info [tailwatchd] chkservd::Hang Notification => info@s000-000-000-000.secureserve***** via EMAIL [eventimportance => High (1)]
    ==> cpsrvd 11.74.0.9 started
    ==> cpsrvd: loading security policy....Done
    ==> cpsrvd: Setting up SSL support ... Done
    ==> cpsrvd: transferred port bindings
    ==> cpsrvd: bound to ports
    [2018-10-17 15:33:12 +0000] warn [restartsrv_spamd] The 'spamd' service's PID file '/var/run/spamd.pid' did not appear after 10 seconds.
    [2018-10-17 08:33:16 -0700] info [queueprocd] chkservd::Notify Notification => info@s000-000-000-000.secureserve***** via EMAIL [eventimportance => High (1)]
    ==> cpsrvd 11.74.0.9 started
    ==> cpsrvd: loading security policy....Done
    ==> cpsrvd: Setting up SSL support ... Done
    ==> cpsrvd: transferred port bindings
    ==> cpsrvd: bound to ports
    [2018-10-17 08:40:02 -0700] info [queueprocd] chkservd::Notify Notification => info@s000-000-000-000.secureserve***** via EMAIL [eventimportance => High (1)]
    [2018-10-17 15:41:00 +0000] warn [restartsrv_clamd] The system encountered an error while starting the “clamd” service with the command “/usr/local/cpanel/3rdparty/bin/clamd”: The subprocess reported error number 1 when it ended. at /usr/local/cpanel/Cpanel/ServiceManager/Base.pm line 1403.
            Cpanel::ServiceManager::Base::warn(Cpanel::ServiceManager::Services::Clamd=HASH(0x10dd6f8), "The system encountered an error while starting the \x{e2}\x{80}\x{9c}clamd\x{e2}\x{80}"...) called at /usr/local/cpanel/Cpanel/ServiceManager/Manager/Initd.pm line 103
            Cpanel::ServiceManager::Manager::Initd::start(Cpanel::ServiceManager::Manager::Initd=HASH(0x113dbd0), Cpanel::ServiceManager::Services::Clamd=HASH(0x10dd6f8)) called at /usr/local/cpanel/Cpanel/ServiceManager/Base.pm line 848
            Cpanel::ServiceManager::Base::start(Cpanel::ServiceManager::Services::Clamd=HASH(0x10dd6f8)) called at /usr/local/cpanel/Cpanel/ServiceManager/Services/Clamd.pm line 60
            Cpanel::ServiceManager::Services::Clamd::restart(Cpanel::ServiceManager::Services::Clamd=HASH(0x10dd6f8), "graceful", undef) called at /usr/local/cpanel/Cpanel/ServiceManager/Base.pm line 540
            Cpanel::ServiceManager::Base::run_from_argv(Cpanel::ServiceManager::Services::Clamd=HASH(0x10dd6f8), "--verbose", "--restart", "--attempt", 99) called at bin/restartsrv_base.pl line 96
            scripts::restartsrv_base::__ANON__() called at /usr/local/cpanel/3rdparty/perl/526/lib64/perl5/cpanel_lib/Try/Tiny.pm line 99
            eval {...} called at /usr/local/cpanel/3rdparty/perl/526/lib64/perl5/cpanel_lib/Try/Tiny.pm line 90
            Try::Tiny::try(CODE(0x10e1cb8), Try::Tiny::Catch=REF(0xb44f68)) called at bin/restartsrv_base.pl line 130
            scripts::restartsrv_base::run("--restart", "--attempt", 99) called at bin/restartsrv_base.pl line 33
    [2018-10-17 08:41:02 -0700] info [queueprocd] chkservd::Notify Notification => info@s000-000-000-000.secureserve***** via EMAIL [eventimportance => High (1)]
    [2018-10-17 08:41:02 -0700] info [queueprocd] chkservd::Notify Notification => info@s000-000-000-000.secureserve***** via EMAIL [eventimportance => High (1)]
    [2018-10-17 08:41:54 -0700] info [xml-api] Syncing version information from httpupdate.cpanel.net/cpanelsync/TIERS.json
    [2018-10-17 08:41:56 -0700] info [xml-api] Successfully verified signature for cpanel (key types: release).
    [2018-10-17 08:41:58 -0700] info [whostmgrd] Internal Server Error: "GET /cpsess6115035217/scripts/reshttpd?confirm=1 HTTP/1.1" 500 No response from subprocess (whostmgr (whostmgr)):
    [2018-10-17 08:42:05 -0700] warn [cpsrvd] Failed to fork(): for main connection loop: Cannot allocate memory at cpsrvd.pl line 898.
            cpanel::cpsrvd::script() called at cpsrvd.pl line 325
    [2018-10-17 08:42:34 -0700] info [whostmgrd] Internal Server Error: "GET /cpsess6115035217/scripts/servup HTTP/1.1" 500 No response from subprocess (whostmgr (whostmgr)):
    [2018-10-17 15:43:14 +0000] warn [restartsrv_spamd] The 'spamd' service's PID file '/var/run/spamd.pid' did not appear after 10 seconds.
    [2018-10-17 08:43:18 -0700] info [queueprocd] chkservd::Notify Notification => info@s000-000-000-000.secureserve***** via EMAIL [eventimportance => High (1)]
    [2018-10-17 08:47:52 -0700] warn [tailwatchd] [26365] TIME 1539791272 UID 0 clobbered stale lock file “/var/run/chkservd.checking.lock” (NAME“tailwatchd - chkservd”, UID 0) written by PID 25971 at 1539790971 at /usr/local/cpanel/Cpanel/SafeFile.pm line 913.
            Cpanel::SafeFile::_lock_wait("/var/run/chkservd.checking") called at /usr/local/cpanel/Cpanel/SafeFile.pm line 355
            Cpanel::SafeFile::_safelock("/var/run/chkservd.checking") called at /usr/local/cpanel/Cpanel/SafeFile.pm line 293
            Cpanel::SafeFile::safelock("/var/run/chkservd.checking") called at /usr/local/cpanel/Cpanel/TailWatch/ChkServd.pm line 252
            Cpanel::TailWatch::ChkServd::_run_checks(Cpanel::TailWatch::ChkServd=HASH(0x1bee698), Cpanel::TailWatch=HASH(0x7c3c68), 1539791272) called at /usr/local/cpanel/Cpanel/TailWatch/ChkServd.pm line 208
            Cpanel::TailWatch::ChkServd::run(Cpanel::TailWatch::ChkServd=HASH(0x1bee698), Cpanel::TailWatch=HASH(0x7c3c68), 1539791272) called at/usr/local/cpanel/Cpanel/TailWatch.pm line 748
            Cpanel::TailWatch::tail_logs(Cpanel::TailWatch=HASH(0x7c3c68)) called at /usr/local/cpanel/Cpanel/TailWatch.pm line 648
            eval {...} called at /usr/local/cpanel/Cpanel/TailWatch.pm line 616
            Cpanel::TailWatch::run(Cpanel::TailWatch=HASH(0x7c3c68)) called at libexec/tailwatch/tailwatchd.pl line 58
            main::run("--start") called at libexec/tailwatch/tailwatchd.pl line 35
    [2018-10-17 08:47:52 -0700] info [tailwatchd] chkservd::Hang Notification => info@s000-000-000-000.secureserve***** via EMAIL [eventimportance => High (1)]
    [2018-10-17 08:48:05 -0700] info [queueprocd] chkservd::Notify Notification => info@s000-000-000-000.secureserve***** via EMAIL [eventimportance => High (1)]
    [2018-10-17 08:54:51 -0700] info [queueprocd] chkservd::Notify Notification => info@s000-000-000-000.secureserve***** via EMAIL [eventimportance => High (1)]
    [2018-10-17 15:55:50 +0000] warn [restartsrv_clamd] The system encountered an error while starting the “clamd” service with the command “/usr/local/cpanel/3rdparty/bin/clamd”: The subprocess reported error number 1 when it ended. at /usr/local/cpanel/Cpanel/ServiceManager/Base.pm line 1403.
            Cpanel::ServiceManager::Base::warn(Cpanel::ServiceManager::Services::Clamd=HASH(0x29506f8), "The system encountered an error while starting the \x{e2}\x{80}\x{9c}clamd\x{e2}\x{80}"...) called at /usr/local/cpanel/Cpanel/ServiceManager/Manager/Initd.pm line 103
            Cpanel::ServiceManager::Manager::Initd::start(Cpanel::ServiceManager::Manager::Initd=HASH(0x29b0ba0), Cpanel::ServiceManager::Services::Clamd=HASH(0x29506f8)) called at /usr/local/cpanel/Cpanel/ServiceManager/Base.pm line 848
            Cpanel::ServiceManager::Base::start(Cpanel::ServiceManager::Services::Clamd=HASH(0x29506f8)) called at /usr/local/cpanel/Cpanel/ServiceManager/Services/Clamd.pm line 60
            Cpanel::ServiceManager::Services::Clamd::restart(Cpanel::ServiceManager::Services::Clamd=HASH(0x29506f8), "graceful", undef) called at /usr/local/cpanel/Cpanel/ServiceManager/Base.pm line 540
            Cpanel::ServiceManager::Base::run_from_argv(Cpanel::ServiceManager::Services::Clamd=HASH(0x29506f8), "--verbose", "--restart", "--attempt", 100) called at bin/restartsrv_base.pl line 96
            scripts::restartsrv_base::__ANON__() called at /usr/local/cpanel/3rdparty/perl/526/lib64/perl5/cpanel_lib/Try/Tiny.pm line 99
            eval {...} called at /usr/local/cpanel/3rdparty/perl/526/lib64/perl5/cpanel_lib/Try/Tiny.pm line 90
            Try::Tiny::try(CODE(0x2954c88), Try::Tiny::Catch=REF(0xb44f68)) called at bin/restartsrv_base.pl line 130
            scripts::restartsrv_base::run("--restart", "--attempt", 100) called at bin/restartsrv_base.pl line 33
    [2018-10-17 08:55:54 -0700] info [queueprocd] chkservd::Notify Notification => info@s000-000-000-000.secureserve***** via EMAIL [eventimportance => High (1)]
    [2018-10-17 08:58:02 -0700] info [queueprocd] chkservd::Notify Notification => info@s000-000-000-000.secureserve***** via EMAIL [eventimportance => High (1)]
    
     
    #5 LukeFive Dev, Oct 17, 2018
    Last edited by a moderator: Oct 17, 2018
  6. cPanelLauren

    cPanelLauren Forums Analyst II Staff Member

    Joined:
    Nov 14, 2017
    Messages:
    5,707
    Likes Received:
    436
    Trophy Points:
    233
    Location:
    Houston
    cPanel Access Level:
    DataCenter Provider
    Hi @LukeFive Dev


    I think at this point it might be important to know how much memory the server has allocated to it. You can see this at WHM>>Server Status>>Server Information.

    Thanks!
     
    Stop hovering to collapse... Click to collapse... Hover to expand... Click to expand...
    SS-Maddy likes this.
  7. LukeFive Dev

    LukeFive Dev Member

    Joined:
    Sep 14, 2018
    Messages:
    18
    Likes Received:
    2
    Trophy Points:
    3
    Location:
    United States
    cPanel Access Level:
    Root Administrator
    @cPanelLauren here is both Current Memory Usage and Current Disk Usage ...
    total used free shared buffers cached
    Mem: 2097152 1664052 433100 267816 0 601528
    -/+ buffers/cache: 1062524 1034628
    Swap: 0 0 0
    Total: 2097152 1664052 433100


    Filesystem Size Used Avail Use% Mounted on
    /dev/ploop54140p1 59G 20G 37G 36% /
    none 1.0G 4.0K 1.0G 1% /dev
    none 1.0G 0 1.0G 0% /dev/shm
     
  8. cPanelLauren

    cPanelLauren Forums Analyst II Staff Member

    Joined:
    Nov 14, 2017
    Messages:
    5,707
    Likes Received:
    436
    Trophy Points:
    233
    Location:
    Houston
    cPanel Access Level:
    DataCenter Provider
    Hi @LukeFive Dev


    That first one is what I wanted but it shows you have plenty of memory available. When you're experiencing the issue I'm curious what it reports.

    Could you run the following (just to get a bit of historical data for the day)

    Code:
    sar -r 
    Also, do you have shell fork bomb protection enabled on the server? You can find this at WHM>>Security Center>>Shell Fork Bomb Protection. If you do, could you please disable it and let me know if the issue persists.

    Thanks!
     
    Stop hovering to collapse... Click to collapse... Hover to expand... Click to expand...
  9. LukeFive Dev

    LukeFive Dev Member

    Joined:
    Sep 14, 2018
    Messages:
    18
    Likes Received:
    2
    Trophy Points:
    3
    Location:
    United States
    cPanel Access Level:
    Root Administrator
    @cPanelLauren Great difficulty trying to login today. Had to power off/on. Shell Fork Bomb Protection disabled. Here is the sar :

    Code:
     sar -r
    Linux 2.6.32-042stab120.16 (s000-000-000-000.secureserver.net)  10/19/2018      _x86_64_        (120 CPU)
    12:00:01 AM kbmemfree kbmemused  %memused kbbuffers  kbcached  kbcommit   %commit
    12:10:02 AM         0   2097152    100.00         0   1118460         0      0.00
    12:20:01 AM         0   2097152    100.00         0   1114816         0      0.00
    12:40:01 AM         0   2097152    100.00         0   1112640         0      0.00
    12:50:01 AM         0   2097152    100.00         0   1112996         0      0.00
    01:10:01 AM         0   2097152    100.00         0   1111620         0      0.00
    01:20:01 AM         0   2097152    100.00         0   1108932         0      0.00
    01:40:01 AM         0   2097152    100.00         0   1096824         0      0.00
    01:50:02 AM         0   2097152    100.00         0   1096820         0      0.00
    02:00:01 AM         0   2097152    100.00         0   1096988         0      0.00
    02:10:01 AM         0   2097152    100.00         0   1099464         0      0.00
    02:20:01 AM         0   2097152    100.00         0   1099492         0      0.00
    02:30:01 AM         0   2097152    100.00         0   1099736         0      0.00
    02:40:01 AM         0   2097152    100.00         0   1100428         0      0.00
    02:50:01 AM         0   2097152    100.00         0   1083336         0      0.00
    03:00:01 AM         0   2097152    100.00         0   1065172         0      0.00
    03:10:01 AM         0   2097152    100.00         0   1065300         0      0.00
    03:20:01 AM         0   2097152    100.00         0   1065268         0      0.00
    03:30:02 AM         0   2097152    100.00         0   1064708         0      0.00
    03:40:01 AM         0   2097152    100.00         0   1060732         0      0.00
    03:50:01 AM         0   2097152    100.00         0   1065940         0      0.00
    04:00:01 AM         0   2097152    100.00         0   1065948         0      0.00
    04:10:01 AM         0   2097152    100.00         0   1066484         0      0.00
    04:10:01 AM kbmemfree kbmemused  %memused kbbuffers  kbcached  kbcommit   %commit
    04:20:01 AM         0   2097152    100.00         0    734800         0      0.00
    04:30:01 AM         0   2097152    100.00         0    757216         0      0.00
    04:40:01 AM         0   2097152    100.00         0    759624         0      0.00
    04:50:01 AM         0   2097152    100.00         0    757556         0      0.00
    05:00:01 AM         0   2097152    100.00         0    758316         0      0.00
    05:10:01 AM         0   2097152    100.00         0    757796         0      0.00
    05:20:01 AM         0   2097152    100.00         0    760668         0      0.00
    05:30:01 AM         0   2097152    100.00         0    765628         0      0.00
    05:40:01 AM         0   2097152    100.00         0    774808         0      0.00
    05:50:01 AM         0   2097152    100.00         0    774380         0      0.00
    06:00:01 AM         0   2097152    100.00         0    773540         0      0.00
    06:10:01 AM         0   2097152    100.00         0    768840         0      0.00
    06:20:01 AM         0   2097152    100.00         0    754728         0      0.00
    06:30:01 AM         0   2097152    100.00         0    754980         0      0.00
    06:40:01 AM         0   2097152    100.00         0    758388         0      0.00
    06:50:01 AM         0   2097152    100.00         0    758456         0      0.00
    07:00:01 AM         0   2097152    100.00         0    758764         0      0.00
    07:10:02 AM         0   2097152    100.00         0    758868         0      0.00
    07:20:01 AM         0   2097152    100.00         0    753324         0      0.00
    07:30:01 AM    147012   1950140     92.99         0    777664         0      0.00
    07:40:01 AM    147860   1949292     92.95         0    777892         0      0.00
    07:50:01 AM    151076   1946076     92.80         0    778448         0      0.00
    07:50:01 AM kbmemfree kbmemused  %memused kbbuffers  kbcached  kbcommit   %commit
    08:00:01 AM         0   2097152    100.00         0   1191780         0      0.00
    08:10:01 AM         0   2097152    100.00         0   1185560         0      0.00
    08:20:01 AM         0   2097152    100.00         0   1178756         0      0.00
    08:30:01 AM         0   2097152    100.00         0   1180992         0      0.00
    08:40:01 AM         0   2097152    100.00         0   1176740         0      0.00
    08:50:01 AM         0   2097152    100.00         0   1178204         0      0.00
    09:00:01 AM         0   2097152    100.00         0   1182032         0      0.00
    Average:         8744   2088408     99.58         0    961193         0      0.00
    09:08:44 AM       LINUX RESTART
     
    #9 LukeFive Dev, Oct 19, 2018
    Last edited: Oct 19, 2018
  10. LukeFive Dev

    LukeFive Dev Member

    Joined:
    Sep 14, 2018
    Messages:
    18
    Likes Received:
    2
    Trophy Points:
    3
    Location:
    United States
    cPanel Access Level:
    Root Administrator
    Trying to be proactive here is result of top ...

    Code:
    top - 09:34:48 up 26 min,  0 users,  load average: 0.13, 0.10, 0.08
    Tasks:  54 total,   1 running,  52 sleeping,   0 stopped,   1 zombie
    Cpu(s):  0.2%us,  0.2%sy,  0.5%ni, 99.1%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
    Mem:   2097152k total,  1717032k used,   380120k free,        0k buffers
    Swap:        0k total,        0k used,        0k free,   658316k cached
      PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND
     4261 admin     20   0  546m  30m  18m S  4.7  1.5   0:00.14 php-fpm: pool DOMAIN_COM
     1333 mysql     20   0 1420m  50m 8404 S  0.7  2.5   0:03.24 /usr/sbin/mysqld --basedir=/usr --datadir=/var/lib/mysql --plugin-dir=/usr/lib64/
     2182 nobody    20   0 2574m  18m 8452 S  0.3  0.9   0:00.50 /usr/sbin/httpd
     2278 cpanelco  20   0 12872 4800 4348 S  0.3  0.2   0:02.37 /usr/local/cpanel/3rdparty/sbin/p0f -i any -u cpanelconnecttrack -d -s /var/cpane
     4181 root      20   0 15000 1284 1008 R  0.3  0.1   0:00.06 top c
        1 root      20   0 19240 1540 1272 S  0.0  0.1   0:00.04 init
        2 root      20   0     0    0    0 S  0.0  0.0   0:00.00 [kthreadd/176124]
        3 root      20   0     0    0    0 S  0.0  0.0   0:00.00 [khelper/176124]
      134 root      16  -4 10648  596  324 S  0.0  0.0   0:00.00 /sbin/udevd -d
      702 root      20   0  179m 1476 1064 S  0.0  0.1   0:00.03 /sbin/rsyslogd -i /var/run/syslogd.pid -c 5
      724 named     20   0 1616m  32m 2600 S  0.0  1.6   0:00.07 /usr/sbin/named -u named
      764 dbus      20   0 21444  888  636 S  0.0  0.0   0:00.00 dbus-daemon --system
     1029 cpanelso  20   0 5133m 245m  20m S  0.0 12.0   0:14.78 /usr/lib/jvm/jre-1.8.0/bin/java -server -Xms512m -Xmx512m -XX:NewRatio=3 -XX:Surv
     1164 root      20   0 66296 1236  492 S  0.0  0.1   0:00.00 /usr/sbin/sshd
     1175 root      20   0 21720  952  716 S  0.0  0.0   0:00.00 xinetd -stayalive -pidfile /var/run/xinetd.pid
     1185 root      20   0  105m 1644 1332 S  0.0  0.1   0:00.02 /bin/sh /usr/bin/mysqld_safe --datadir=/var/lib/mysql --pid-file=/var/lib/mysql/s
     1371 root      20   0 19984 1172  832 S  0.0  0.1   0:00.01 /usr/sbin/dovecot
     
    #10 LukeFive Dev, Oct 19, 2018
    Last edited: Oct 19, 2018
  11. cPanelLauren

    cPanelLauren Forums Analyst II Staff Member

    Joined:
    Nov 14, 2017
    Messages:
    5,707
    Likes Received:
    436
    Trophy Points:
    233
    Location:
    Houston
    cPanel Access Level:
    DataCenter Provider
    HI @LukeFive Dev

    That's some really interesting usage output there. While your commit% is at 0 your usage is 100% across the board. That's not normal behavior and would definitely explain why you're having trouble accessing the server. None the less 2097152k translates to 2GB of memory which is really the bare minimum.

    If your provider offers support I would suggest contacting them to address the high memory usage, I don't know if you're on a VM or a hardware server but if you're on a VM it may be worth investigating the IO usage as well. You might also check out the tutorial here: Tutorial - Troubleshooting high server loads on Linux servers (while your load average is low your memory usage is quite high at least according to sar and for the amount of memory you have on the server)

    Thanks!
     
    Stop hovering to collapse... Click to collapse... Hover to expand... Click to expand...
  12. LukeFive Dev

    LukeFive Dev Member

    Joined:
    Sep 14, 2018
    Messages:
    18
    Likes Received:
    2
    Trophy Points:
    3
    Location:
    United States
    cPanel Access Level:
    Root Administrator
    Well it's a 2GB VPS on GoDaddy, and the account basically has no support. I will try the tutorial, but after that try opening a ticket. The two larger sites we have there are noticing the error messages and wonder why our usually speedy service, this time is not.
    Thank you Lauren and others for analysis of this item.
     
  13. cPanelLauren

    cPanelLauren Forums Analyst II Staff Member

    Joined:
    Nov 14, 2017
    Messages:
    5,707
    Likes Received:
    436
    Trophy Points:
    233
    Location:
    Houston
    cPanel Access Level:
    DataCenter Provider
    Hi @LukeFive Dev

    Can you run this:

    Code:
    sar -p 
    It'll give you an idea of the IO wait. Seemingly phantom IO wait and high memory usage (sometimes along with high load) is not uncommon on VPS's on overloaded host nodes.
     
    Stop hovering to collapse... Click to collapse... Hover to expand... Click to expand...
  14. LukeFive Dev

    LukeFive Dev Member

    Joined:
    Sep 14, 2018
    Messages:
    18
    Likes Received:
    2
    Trophy Points:
    3
    Location:
    United States
    cPanel Access Level:
    Root Administrator
    @cPanelLauren - the restarts are me trying to get it stable enough to log in...
    10/19/2018 _x86_64_ (120 CPU)
    Code:
    12:00:01 AM     CPU     %user     %nice   %system   %iowait    %steal     %idle
    12:10:02 AM     all      0.03      0.00      0.02      0.00      0.00     99.96
    12:20:01 AM     all      0.03      0.00      0.01      0.00      0.00     99.95
    12:40:01 AM     all      0.03      0.00      0.01      0.01      0.00     99.94
    12:50:01 AM     all      0.03      0.00      0.02      0.00      0.00     99.95
    01:10:01 AM     all      0.03      0.00      0.02      0.01      0.00     99.94
    01:20:01 AM     all      0.02      0.00      0.01      0.00      0.00     99.97
    01:40:01 AM     all      0.02      0.00      0.02      0.00      0.00     99.96
    01:50:02 AM     all      0.02      0.00      0.01      0.00      0.00     99.97
    02:00:01 AM     all      0.02      0.00      0.01      0.00      0.00     99.97
    02:10:01 AM     all      0.03      0.00      0.02      0.00      0.00     99.95
    02:20:01 AM     all      0.04      0.00      0.01      0.01      0.00     99.94
    02:30:01 AM     all      0.03      0.00      0.01      0.00      0.00     99.95
    02:40:01 AM     all      0.04      0.00      0.02      0.00      0.00     99.94
    02:50:01 AM     all      0.02      0.00      0.02      0.03      0.00     99.93
    03:00:01 AM     all      0.03      0.00      0.01      0.00      0.00     99.95
    03:10:01 AM     all      0.03      0.00      0.01      0.00      0.00     99.95
    03:20:01 AM     all      0.02      0.00      0.01      0.00      0.00     99.96
    03:30:02 AM     all      0.02      0.00      0.01      0.00      0.00     99.96
    03:40:01 AM     all      0.05      0.00      0.02      0.01      0.00     99.92
    03:50:01 AM     all      0.04      0.00      0.02      0.00      0.00     99.94
    04:00:01 AM     all      0.02      0.00      0.01      0.00      0.00     99.97
    04:10:01 AM     all      0.04      0.01      0.02      0.00      0.00     99.94
    04:10:01 AM     CPU     %user     %nice   %system   %iowait    %steal     %idle
    04:20:01 AM     all      0.03      0.00      0.02      0.00      0.00     99.95
    04:30:01 AM     all      0.08      0.00      0.04      0.01      0.00     99.88
    04:40:01 AM     all      0.02      0.00      0.01      0.00      0.00     99.96
    04:50:01 AM     all      0.02      0.00      0.01      0.01      0.00     99.96
    05:00:01 AM     all      0.02      0.00      0.01      0.00      0.00     99.96
    05:10:01 AM     all      0.03      0.00      0.01      0.00      0.00     99.96
    05:20:01 AM     all      0.03      0.00      0.02      0.00      0.00     99.95
    05:30:01 AM     all      0.03      0.00      0.01      0.00      0.00     99.95
    05:40:01 AM     all      0.05      0.00      0.02      0.06      0.00     99.86
    05:50:01 AM     all      0.02      0.00      0.01      0.00      0.00     99.96
    06:00:01 AM     all      0.02      0.00      0.02      0.01      0.00     99.95
    06:10:01 AM     all      0.03      0.00      0.02      0.00      0.00     99.95
    06:20:01 AM     all      0.04      0.00      0.02      0.00      0.00     99.93
    06:30:01 AM     all      0.03      0.00      0.01      0.00      0.00     99.96
    06:40:01 AM     all      0.04      0.00      0.02      0.00      0.00     99.93
    06:50:01 AM     all      0.02      0.00      0.01      0.01      0.00     99.96
    07:00:01 AM     all      0.04      0.00      0.01      0.00      0.00     99.95
    07:10:02 AM     all      0.03      0.00      0.02      0.02      0.00     99.93
    07:20:01 AM     all      0.02      0.00      0.01      0.00      0.00     99.97
    07:30:01 AM     all      0.02      0.03      0.01      0.00      0.00     99.94
    07:40:01 AM     all      0.02      0.00      0.01      0.00      0.00     99.96
    07:50:01 AM     all      0.03      0.00      0.02      0.00      0.00     99.95
    07:50:01 AM     CPU     %user     %nice   %system   %iowait    %steal     %idle
    08:00:01 AM     all      0.84      0.00      0.47      0.85      0.00     97.84
    08:10:01 AM     all      0.04      0.00      0.02      0.01      0.00     99.93
    08:20:01 AM     all      0.03      0.01      0.02      0.01      0.00     99.94
    08:30:01 AM     all      0.02      0.00      0.01      0.00      0.00     99.96
    08:40:01 AM     all      0.05      0.00      0.02      0.02      0.00     99.91
    08:50:01 AM     all      0.04      0.00      0.02      0.00      0.00     99.93
    09:00:01 AM     all      0.04      0.00      0.02      0.01      0.00     99.93
    Average:        all      0.04      0.00      0.02      0.02      0.00     99.91
    09:08:44 AM       LINUX RESTART
    09:10:01 AM     CPU     %user     %nice   %system   %iowait    %steal     %idle
    09:20:01 AM     all      0.09      0.00      0.04      0.03      0.00     99.85
    09:30:01 AM     all      0.06      0.00      0.02      0.17      0.00     99.74
    09:40:01 AM     all      0.16      0.00      0.04      0.02      0.00     99.78
    09:50:01 AM     all      0.12      0.00      0.03      0.17      0.00     99.68
    10:00:01 AM     all      0.08      0.00      0.03      0.00      0.00     99.89
    10:10:02 AM     all      0.16      0.00      0.04      0.07      0.00     99.73
    10:20:01 AM     all      0.16      0.00      0.04      0.04      0.00     99.76
    10:30:01 AM     all      0.12      0.00      0.03      0.03      0.00     99.82
    10:40:01 AM     all      0.11      0.00      0.03      0.01      0.00     99.85
    10:50:01 AM     all      0.13      0.01      0.04      0.00      0.00     99.83
    11:00:01 AM     all      0.09      0.00      0.03      0.06      0.00     99.82
    11:10:01 AM     all      0.09      0.00      0.03      0.24      0.00     99.64
    11:20:01 AM     all      0.09      0.00      0.03      0.00      0.00     99.87
    11:30:01 AM     all      0.11      0.00      0.04      0.03      0.00     99.82
    11:40:01 AM     all      0.05      0.00      0.02      0.00      0.00     99.93
    11:50:01 AM     all      0.07      0.00      0.03      0.01      0.00     99.89
    12:00:01 PM     all      0.03      0.00      0.01      0.00      0.00     99.96
    12:10:01 PM     all      0.03      0.00      0.02      0.00      0.00     99.95
    12:20:01 PM     all      0.20      0.00      0.04      0.14      0.00     99.62
    12:30:01 PM     all      0.04      0.00      0.01      0.00      0.00     99.94
    12:40:01 PM     all      0.10      0.00      0.03      0.03      0.00     99.85
    12:50:01 PM     all      0.11      0.00      0.03      0.00      0.00     99.86
    12:50:01 PM     CPU     %user     %nice   %system   %iowait    %steal     %idle
    01:00:01 PM     all      0.07      0.00      0.02      0.00      0.00     99.90
    01:10:01 PM     all      0.07      0.00      0.02      0.00      0.00     99.91
    01:20:01 PM     all      0.05      0.01      0.02      0.00      0.00     99.92
    01:30:01 PM     all      0.05      0.00      0.02      0.02      0.00     99.90
    01:40:01 PM     all      0.12      0.00      0.04      0.07      0.00     99.78
    01:50:01 PM     all      0.61      0.00      0.08      0.05      0.00     99.26
    02:00:01 PM     all      0.07      0.00      0.03      0.00      0.00     99.90
    02:20:01 PM     all      0.08      0.00      0.03      0.04      0.00     99.85
    Average:        all      0.11      0.00      0.03      0.04      0.00     99.82
    02:21:44 PM       LINUX RESTART
    02:30:01 PM     CPU     %user     %nice   %system   %iowait    %steal     %idle
    02:40:01 PM     all      0.16      0.00      0.04      0.04      0.00     99.75
    02:50:01 PM     all      0.29      0.00      0.07      0.14      0.00     99.49
    03:00:01 PM     all      0.12      0.00      0.04      0.11      0.00     99.73
    03:10:01 PM     all      0.10      0.00      0.03      0.25      0.00     99.62
    03:20:01 PM     all      0.06      0.00      0.02      0.02      0.00     99.90
    03:30:01 PM     all      0.08      0.00      0.03      0.02      0.00     99.87
    03:40:01 PM     all      0.05      0.00      0.02      0.00      0.00     99.93
    03:50:01 PM     all      0.08      0.00      0.02      0.01      0.00     99.89
    Average:        all      0.12      0.00      0.03      0.07      0.00     99.77
    03:53:12 PM       LINUX RESTART
    04:00:01 PM     CPU     %user     %nice   %system   %iowait    %steal     %idle
    04:10:01 PM     all      0.04      0.00      0.02      0.01      0.00     99.92
    04:20:01 PM     all      0.02      0.01      0.01      0.00      0.00     99.96
    04:30:01 PM     all      0.07      0.00      0.03      0.04      0.00     99.86
    04:40:01 PM     all      0.09      0.00      0.03      0.04      0.00     99.83
    04:50:01 PM     all      0.13      0.00      0.04      0.02      0.00     99.81
    Average:        all      0.07      0.00      0.03      0.02      0.00     99.88
    05:00:05 PM       LINUX RESTART
     
    #14 LukeFive Dev, Oct 19, 2018
    Last edited by a moderator: Oct 19, 2018
  15. cPanelLauren

    cPanelLauren Forums Analyst II Staff Member

    Joined:
    Nov 14, 2017
    Messages:
    5,707
    Likes Received:
    436
    Trophy Points:
    233
    Location:
    Houston
    cPanel Access Level:
    DataCenter Provider
    HI @LukeFive Dev


    The IOwait is really quite low and points to something else being the source of the issue you're experiencing. It's possible that you go through the tutorial I provided on troubleshooting high load and find that your traffic warrants more memory/resources.
     
    Stop hovering to collapse... Click to collapse... Hover to expand... Click to expand...
  16. LukeFive Dev

    LukeFive Dev Member

    Joined:
    Sep 14, 2018
    Messages:
    18
    Likes Received:
    2
    Trophy Points:
    3
    Location:
    United States
    cPanel Access Level:
    Root Administrator
    @cPanelLauren Yes I am pursuing the tutorial and also opening an official ticket. Thanks for your support!
     
    cPanelLauren likes this.
Loading...

Share This Page

  1. This site uses cookies to help personalise content, tailor your experience and to keep you logged in if you register.
    By continuing to use this site, you are consenting to our use of cookies.
    Dismiss Notice