cpanel-ccs failing

GoWilkes · Apr 20, 2021

I woke up to 300 emails today, all showing that cpanel-ccs failed :-/

I first disabled CSF firewall, thinking it could be a failing IP connection, but that had no impact.

I'm using CentOS 7.9 virtuozzo, v94.0.4.

Here's the error I'm getting:

Code:

# /usr/local/cpanel/scripts/restartsrv_cpanel_ccs
Waiting for âcpanel_ccsâcpanel_ccsâ

Cpanel::Exception::Services::StartError
Service Status
        undefined status from Cpanel::ServiceManager::Services::Cpanel_ccs

Service Error
        (XID ducyxc) The âcpanel_ccsâ

Startup Log
        Apr 20 13:14:48 server.example.com ccs_init[1468]: Using /opt/cpanel-ccs/.develop/virtualenv/bin/python as Python
        Apr 20 13:14:48 server.example.com ccs_init[1468]: Starting server...
        Apr 20 13:14:48 server.example.com ccs_init[1468]: /opt/cpanel-ccs/bin/caldavd  -f /opt/cpanel-ccs/conf/caldavd-dev.plist -P caldav -t Combined
        Apr 20 13:14:51 server.example.com systemd[1]: Started Apple Calendar Server.
        Apr 20 13:14:52 server.example.com ccs_init[1508]: kill = true
        Apr 20 13:14:52 server.example.com ccs_init[1508]: restart = false
        Apr 20 13:14:56 server.example.com ccs_init[1508]: pidfile = /opt/cpanel-ccs/./data/Logs/state/caldavd.pid
        Apr 20 13:14:56 server.example.com systemd[1]: cpanel-ccs.service: control process exited, code=exited status=1
        Apr 20 13:14:56 server.example.com systemd[1]: Unit cpanel-ccs.service entered failed state.
        Apr 20 13:14:56 server.example.com systemd[1]: cpanel-ccs.service failed.

cpanel_ccs has failed. Contact your system administrator if the service does not automagically recover.

I've read in previous threads that cpdavd was often the problem, but I was able to restart cpdavd with no errors so I think it's OK this time:

Code:

# /scripts/restartsrv_cpdavd --restart
Waiting for âcpdavdâcpdavdâ

Service Status
        cpdavd (cpdavd - accepting connections on: 2077, 2078 (dormant)) is running as root with PID 1000 (systemd+/proc check method).

Startup Log
        Dec 07 01:01:59 server.example.com systemd[1]: Starting cPanel dav services...
        Dec 07 01:02:02 server.example.com systemd[1]: Can't open PID file /var/run/cpdavd.pid (yet?) after start: No such file or directory
        Dec 07 01:02:02 server.example.com systemd[1]: Started cPanel dav services.

cpdavd restarted successfully.

Any suggestions?

cPRex · Apr 20, 2021

Hey there! Do you see anything helpful in the cPanel error log at /usr/local/cpanel/logs/error_log? If not, do you see any high resource usage on the machine?

GoWilkes · Apr 20, 2021

cPRex said:
Hey there! Do you see anything helpful in the cPanel error log at /usr/local/cpanel/logs/error_log?

Not that I recognize... it looks like it started at 3:56am, and after that I just have a ton of these:

Code:

[2021-04-20 14:01:18 -0400]: “/usr/local/cpanel/scripts/restartsrv_cpanel_ccs --restart --hard --attempt 121” called by (27514 - tailwatchd - chkservd - cpanel-ccs check)
==> cpsrvd 11.94.0.4 started
==> cpsrvd: loading security policy....Done
==> cpsrvd: Setting up SSL support ... Done
==> cpsrvd: transferred port bindings: 10,11,12,3,4,5,6,7,8,9
==> cpsrvd: bound to ports
[2021-04-20 14:01:20 -0400] info [cpsrvd] version 11.94.0.4 online
[2021-04-20 14:02:28 -0400] info [cpsrvd] reloading config based on -HUP signal
[2021-04-20 14:03:25 -0400] info [queueprocd] chkservd::Notify Notification => go***@gmai***** via EMAIL [eventimportance => High (1)]
[2021-04-20 14:03:25 -0400] info [queueprocd] chkservd::Notify Notification => 1336****@messaging.sprintpc***** via PAGER [eventimportance => High (1)]
==> cpsrvd 11.94.0.4 started
==> cpsrvd: loading security policy....Done
==> cpsrvd: Setting up SSL support ... Done
==> cpsrvd: transferred port bindings: 10,11,12,3,4,5,6,7,8,9
==> cpsrvd: bound to ports
[2021-04-20 14:04:34 -0400] info [cpsrvd] version 11.94.0.4 online
[2021-04-20 14:06:30 -0400]: “/usr/local/cpanel/scripts/restartsrv_cpanel_ccs --restart --hard --attempt 122” called by (30521 - tailwatchd - chkservd - cpanel-ccs check)
[2021-04-20 14:08:37 -0400] info [queueprocd] chkservd::Notify Notification => go***@gmai***** via EMAIL [eventimportance => High (1)]
[2021-04-20 14:08:37 -0400] info [queueprocd] chkservd::Notify Notification => 1336****@messaging.sprintpc***** via PAGER [eventimportance => High (1)]
==> cpsrvd 11.94.0.4 started
==> cpsrvd: loading security policy....Done
==> cpsrvd: Setting up SSL support ... Done
==> cpsrvd: transferred port bindings: 10,11,12,3,4,5,6,7,8,9
==> cpsrvd: bound to ports

I turned off the monitoring of CCS in WHM > Service Manager at around 2:10pm, and then I got this handful of data:

Code:

[2021-04-20 14:11:33 -0400]: “/usr/local/cpanel/scripts/restartsrv_cpanel_ccs --restart --hard --attempt 123” called by (583 - tailwatchd - chkservd - cpanel-ccs check)
[2021-04-20 14:12:27 -0400] info [whostmgrd] Successfully verified signature for cpanel (key types: release).
cpsrvd e6683788d777: Cpanel::Exception::NetSSLeay/(XID bxxj2j) Net::SSLeay::ssl_write_all(SCALAR(0x20f5488)) produced an operating system error (EPIPE, Broken pipe) and 1 [asis,OpenSSL] [numerate,_6,error,errors]: ARRAY(0x22b4378)
 at /usr/local/cpanel/Cpanel/Server/Connection/SSL.pm line 64.
    Cpanel::Server::Connection::SSL::write_buffer(Cpanel::Server::Connection::SSL=HASH(0x20eead0), SCALAR(0x20f5488)) called at /usr/local/cpanel/Cpanel/Server.pm line 1682
    Cpanel::Server::write_buffer(Cpanel::Server=HASH(0x1f1c568), SCALAR(0x20f5488)) called at /usr/local/cpanel/Cpanel/Server/Handlers/SSE.pm line 141
    Cpanel::Server::Handlers::SSE::__ANON__(SCALAR(0x20f5488)) called at /usr/local/cpanel/Cpanel/Server/Responders/Stream/Gzip.pm line 101
    Cpanel::Server::Responders::Stream::Gzip::write(Cpanel::Server::Responders::Stream::Gzip=HASH(0x22033c8), 1) called at /usr/local/cpanel/Cpanel/Server/SSE.pm line 212
    Cpanel::Server::SSE::__responder_write(Cpanel::Server::SSE::whostmgr::Tasks=HASH(0x213e6b8), ":\x{d}\x{a}") called at /usr/local/cpanel/Cpanel/Server/SSE.pm line 160
    Cpanel::Server::SSE::_send_sse_heartbeat(Cpanel::Server::SSE::whostmgr::Tasks=HASH(0x213e6b8)) called at /usr/local/cpanel/Cpanel/Server/SSE/whostmgr/Tasks.pm line 160
    Cpanel::Server::SSE::whostmgr::Tasks::run(Cpanel::Server::SSE::whostmgr::Tasks=HASH(0x213e6b8)) called at /usr/local/cpanel/Cpanel/Server/Handlers/SSE.pm line 179
    Cpanel::Server::Handlers::SSE::handler(Cpanel::Server::Handlers::SSE=HASH(0x2110800), "Tasks", ARRAY(0x1411f90)) called at cpsrvd.pl line 8616
    cpanel::cpsrvd::sse_handler("Tasks") called at cpsrvd.pl line 6678
    cpanel::cpsrvd::dodoc_whostmgrd() called at cpsrvd.pl line 2058
    cpanel::cpsrvd::dodoc(HASH(0x14461f0)) called at cpsrvd.pl line 1795
    cpanel::cpsrvd::handle_one_connection(6) called at cpsrvd.pl line 1116
    cpanel::cpsrvd::script() called at cpsrvd.pl line 432

[2021-04-20 14:13:32 -0400] warn [whostmgrd] Duplicate logaccess (GET /cpsess9293551131/sse/Tasks HTTP/1.1):  at /usr/local/cpanel/Cpanel/Server/Logger.pm line 80.
    Cpanel::Server::Logger::logaccess(Cpanel::Server::Logger=HASH(0x1f1cdd8)) called at /usr/local/cpanel/Cpanel/Server.pm line 753
    Cpanel::Server::body_internal_error(Cpanel::Server=HASH(0x1f1c568), 500, "Error ID e6683788d777") called at /usr/local/cpanel/Cpanel/Server.pm line 676
    Cpanel::Server::internal_error_with_http_status(Cpanel::Server=HASH(0x1f1c568), 500, "Error ID e6683788d777") called at /usr/local/cpanel/Cpanel/Server.pm line 659
    Cpanel::Server::internal_error(Cpanel::Server=HASH(0x1f1c568), "Error ID e6683788d777") called at cpsrvd.pl line 969
    cpanel::cpsrvd::__ANON__(Cpanel::Exception::NetSSLeay=HASH(0x22b8d40)) called at /usr/local/cpanel/Cpanel/Server/Connection/SSL.pm line 64
    Cpanel::Server::Connection::SSL::write_buffer(Cpanel::Server::Connection::SSL=HASH(0x20eead0), SCALAR(0x20f5488)) called at /usr/local/cpanel/Cpanel/Server.pm line 1682
    Cpanel::Server::write_buffer(Cpanel::Server=HASH(0x1f1c568), SCALAR(0x20f5488)) called at /usr/local/cpanel/Cpanel/Server/Handlers/SSE.pm line 141
    Cpanel::Server::Handlers::SSE::__ANON__(SCALAR(0x20f5488)) called at /usr/local/cpanel/Cpanel/Server/Responders/Stream/Gzip.pm line 101
    Cpanel::Server::Responders::Stream::Gzip::write(Cpanel::Server::Responders::Stream::Gzip=HASH(0x22033c8), 1) called at /usr/local/cpanel/Cpanel/Server/SSE.pm line 212
    Cpanel::Server::SSE::__responder_write(Cpanel::Server::SSE::whostmgr::Tasks=HASH(0x213e6b8), ":\x{d}\x{a}") called at /usr/local/cpanel/Cpanel/Server/SSE.pm line 160
    Cpanel::Server::SSE::_send_sse_heartbeat(Cpanel::Server::SSE::whostmgr::Tasks=HASH(0x213e6b8)) called at /usr/local/cpanel/Cpanel/Server/SSE/whostmgr/Tasks.pm line 160
    Cpanel::Server::SSE::whostmgr::Tasks::run(Cpanel::Server::SSE::whostmgr::Tasks=HASH(0x213e6b8)) called at /usr/local/cpanel/Cpanel/Server/Handlers/SSE.pm line 179
    Cpanel::Server::Handlers::SSE::handler(Cpanel::Server::Handlers::SSE=HASH(0x2110800), "Tasks", ARRAY(0x1411f90)) called at cpsrvd.pl line 8616
    cpanel::cpsrvd::sse_handler("Tasks") called at cpsrvd.pl line 6678
    cpanel::cpsrvd::dodoc_whostmgrd() called at cpsrvd.pl line 2058
    cpanel::cpsrvd::dodoc(HASH(0x14461f0)) called at cpsrvd.pl line 1795
    cpanel::cpsrvd::handle_one_connection(6) called at cpsrvd.pl line 1116
    cpanel::cpsrvd::script() called at cpsrvd.pl line 432
 at /usr/local/cpanel/Cpanel/Server/Logger.pm line 80.
    Cpanel::Server::Logger::logaccess(Cpanel::Server::Logger=HASH(0x1f1cdd8)) called at /usr/local/cpanel/Cpanel/Server.pm line 753
    Cpanel::Server::body_internal_error(Cpanel::Server=HASH(0x1f1c568), 500, "Error ID e6683788d777") called at /usr/local/cpanel/Cpanel/Server.pm line 676
    Cpanel::Server::internal_error_with_http_status(Cpanel::Server=HASH(0x1f1c568), 500, "Error ID e6683788d777") called at /usr/local/cpanel/Cpanel/Server.pm line 659
    Cpanel::Server::internal_error(Cpanel::Server=HASH(0x1f1c568), "Error ID e6683788d777") called at cpsrvd.pl line 969
    cpanel::cpsrvd::__ANON__(Cpanel::Exception::NetSSLeay=HASH(0x22b8d40)) called at /usr/local/cpanel/Cpanel/Server/Connection/SSL.pm line 64
    Cpanel::Server::Connection::SSL::write_buffer(Cpanel::Server::Connection::SSL=HASH(0x20eead0), SCALAR(0x20f5488)) called at /usr/local/cpanel/Cpanel/Server.pm line 1682
    Cpanel::Server::write_buffer(Cpanel::Server=HASH(0x1f1c568), SCALAR(0x20f5488)) called at /usr/local/cpanel/Cpanel/Server/Handlers/SSE.pm line 141
    Cpanel::Server::Handlers::SSE::__ANON__(SCALAR(0x20f5488)) called at /usr/local/cpanel/Cpanel/Server/Responders/Stream/Gzip.pm line 101
    Cpanel::Server::Responders::Stream::Gzip::write(Cpanel::Server::Responders::Stream::Gzip=HASH(0x22033c8), 1) called at /usr/local/cpanel/Cpanel/Server/SSE.pm line 212
    Cpanel::Server::SSE::__responder_write(Cpanel::Server::SSE::whostmgr::Tasks=HASH(0x213e6b8), ":\x{d}\x{a}") called at /usr/local/cpanel/Cpanel/Server/SSE.pm line 160
    Cpanel::Server::SSE::_send_sse_heartbeat(Cpanel::Server::SSE::whostmgr::Tasks=HASH(0x213e6b8)) called at /usr/local/cpanel/Cpanel/Server/SSE/whostmgr/Tasks.pm line 160
    Cpanel::Server::SSE::whostmgr::Tasks::run(Cpanel::Server::SSE::whostmgr::Tasks=HASH(0x213e6b8)) called at /usr/local/cpanel/Cpanel/Server/Handlers/SSE.pm line 179
    Cpanel::Server::Handlers::SSE::handler(Cpanel::Server::Handlers::SSE=HASH(0x2110800), "Tasks", ARRAY(0x1411f90)) called at cpsrvd.pl line 8616
    cpanel::cpsrvd::sse_handler("Tasks") called at cpsrvd.pl line 6678
    cpanel::cpsrvd::dodoc_whostmgrd() called at cpsrvd.pl line 2058
    cpanel::cpsrvd::dodoc(HASH(0x14461f0)) called at cpsrvd.pl line 1795
    cpanel::cpsrvd::handle_one_connection(6) called at cpsrvd.pl line 1116
    cpanel::cpsrvd::script() called at cpsrvd.pl line 432
Cpanel::Exception::NetSSLeay/(XID cavusy) Net::SSLeay::ssl_write_all(<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01//EN" "http://www.w3.org/TR/html4/strict.dtd">\x{0a}<html>\x{0a}<head>\x{0a}<title>500 Internal Server Error</title>\x{0a}<style type="text/css">\x{0a}.errortype {\x{0a}    font-family:verdana,arial,helvetica,clean,sans-serif;\x{0a}    font-size:x-large;\x{0a}    font-weight:bold;\x{0a}}\x{0a}.errormessage {\x{0a}    font-family:verdana,arial,helvetica,clean,sans-serif;\x{0a}    font-size:large;\x{0a}    whitespace:pre;\x{0a}    margin-left:20px;\x{0a}}\x{0a}p.errormessage { color:#ff0000; }\x{0a}.serverinfo {\x{0a}    font-family:verdana,arial,helvetica,clean,sans-serif;\x{0a}    font-size:medium;\x{0a}}\x{0a}</style>\x{0a}</head>\x{0a}<body>\x{0a}<h1 class="errortype">Internal Server Error</h1>\x{0a}<p class="errormessage">500<pre>Error ID e6683788d777</pre></p>\x{0a}<hr>\x{0a}<address class="serverinfo">cpsrvd Server at server.example.com</address>\x{0a}</body>\x{0a}</html>\x{0a}) produced an operating system error (EPIPE, Broken pipe) and 1 [asis,OpenSSL] [numerate,_6,error,errors]: ARRAY(0x22b5150)
 at /usr/local/cpanel/Cpanel/Server/Connection/SSL.pm line 64.
    Cpanel::Server::Connection::SSL::write_buffer(Cpanel::Server::Connection::SSL=HASH(0x20eead0), "<!DOCTYPE HTML PUBLIC \"-//W3C//DTD HTML 4.01//EN\" \"http://www"...) called at /usr/local/cpanel/Cpanel/Server.pm line 1682
    Cpanel::Server::write_buffer(Cpanel::Server=HASH(0x1f1c568), "<!DOCTYPE HTML PUBLIC \"-//W3C//DTD HTML 4.01//EN\" \"http://www"...) called at /usr/local/cpanel/Cpanel/Server.pm line 789
    Cpanel::Server::body_internal_error(Cpanel::Server=HASH(0x1f1c568), 500, "Error ID e6683788d777") called at /usr/local/cpanel/Cpanel/Server.pm line 676
    Cpanel::Server::internal_error_with_http_status(Cpanel::Server=HASH(0x1f1c568), 500, "Error ID e6683788d777") called at /usr/local/cpanel/Cpanel/Server.pm line 659
    Cpanel::Server::internal_error(Cpanel::Server=HASH(0x1f1c568), "Error ID e6683788d777") called at cpsrvd.pl line 969
    cpanel::cpsrvd::__ANON__(Cpanel::Exception::NetSSLeay=HASH(0x22b8d40)) called at /usr/local/cpanel/Cpanel/Server/Connection/SSL.pm line 64
    Cpanel::Server::Connection::SSL::write_buffer(Cpanel::Server::Connection::SSL=HASH(0x20eead0), SCALAR(0x20f5488)) called at /usr/local/cpanel/Cpanel/Server.pm line 1682
    Cpanel::Server::write_buffer(Cpanel::Server=HASH(0x1f1c568), SCALAR(0x20f5488)) called at /usr/local/cpanel/Cpanel/Server/Handlers/SSE.pm line 141
    Cpanel::Server::Handlers::SSE::__ANON__(SCALAR(0x20f5488)) called at /usr/local/cpanel/Cpanel/Server/Responders/Stream/Gzip.pm line 101
    Cpanel::Server::Responders::Stream::Gzip::write(Cpanel::Server::Responders::Stream::Gzip=HASH(0x22033c8), 1) called at /usr/local/cpanel/Cpanel/Server/SSE.pm line 212
    Cpanel::Server::SSE::__responder_write(Cpanel::Server::SSE::whostmgr::Tasks=HASH(0x213e6b8), ":\x{d}\x{a}") called at /usr/local/cpanel/Cpanel/Server/SSE.pm line 160
    Cpanel::Server::SSE::_send_sse_heartbeat(Cpanel::Server::SSE::whostmgr::Tasks=HASH(0x213e6b8)) called at /usr/local/cpanel/Cpanel/Server/SSE/whostmgr/Tasks.pm line 160
    Cpanel::Server::SSE::whostmgr::Tasks::run(Cpanel::Server::SSE::whostmgr::Tasks=HASH(0x213e6b8)) called at /usr/local/cpanel/Cpanel/Server/Handlers/SSE.pm line 179
    Cpanel::Server::Handlers::SSE::handler(Cpanel::Server::Handlers::SSE=HASH(0x2110800), "Tasks", ARRAY(0x1411f90)) called at cpsrvd.pl line 8616
    cpanel::cpsrvd::sse_handler("Tasks") called at cpsrvd.pl line 6678
    cpanel::cpsrvd::dodoc_whostmgrd() called at cpsrvd.pl line 2058
    cpanel::cpsrvd::dodoc(HASH(0x14461f0)) called at cpsrvd.pl line 1795
    cpanel::cpsrvd::handle_one_connection(6) called at cpsrvd.pl line 1116
    cpanel::cpsrvd::script() called at cpsrvd.pl line 432
[2021-04-20 14:13:40 -0400] info [queueprocd] chkservd::Notify Notification => go****@gmai***** via EMAIL [eventimportance => High (1)]
[2021-04-20 14:13:40 -0400] info [queueprocd] chkservd::Notify Notification => 1336****@messaging.sprintpc***** via PAGER [eventimportance => High (1)]
[2021-04-20 14:15:15 -0400]: “/usr/local/cpanel/scripts/restartsrv_apache_php_fpm --status” called by (3001 - whostmgr - dosrvmng)
[2021-04-20 14:15:16 -0400]: “/usr/local/cpanel/scripts/restartsrv_cpanel_ccs --status” called by (3001 - whostmgr - dosrvmng)
Setting mailman archive permissions.
Setting mailman lists permissions.
Purging mailman queue.
[2021-04-20 14:15:17 -0400]: “/usr/local/cpanel/scripts/restartsrv_mailman --status” called by (3001 - whostmgr - dosrvmng)
[2021-04-20 14:15:19 -0400] info [whostmgr] Whostmgr::Services::_restart_services: cpanel-ccs
[2021-04-20 14:15:19 -0400]: “/usr/local/cpanel/scripts/restartsrv_unknown --html --wait --verbose cpanel-ccs” called by (3169 - whostmgr - dosrvmng)
[2021-04-20 14:15:19 -0400] info [cpsrvd] reloading config based on -HUP signal
[2021-04-20 14:15:19 -0400]: “/usr/local/cpanel/scripts/restartsrv_exim ” called by (3195 - queueprocd - process - restartsrv)
[2021-04-20 14:15:24 -0400] info [restartsrv_mailman] The service 'mailman' is disabled and the system will not attempt to restart it.
[2021-04-20 14:15:25 -0400]: “/usr/local/cpanel/scripts/restartsrv_dovecot ” called by (3303 - queueprocd - process - restartsrv)
[2021-04-20 14:15:30 -0400]: “/usr/local/cpanel/scripts/restartsrv_httpd --start” called by (3137 - queueprocd - process - apache_restart - safeaprestart - forced restart)
[2021-04-20 14:15:30 -0400]: “/usr/local/cpanel/scripts/restartsrv_apache_php_fpm --graceful” called by (3402 - /usr/local/cpanel/scripts/restartsrv_httpd --start)
[2021-04-20 14:15:33 -0400]: “/usr/local/cpanel/scripts/restartsrv_apache_php_fpm --graceful” called by (3402 - /usr/local/cpanel/scripts/restartsrv_httpd --start)
[2021-04-20 14:15:34 -0400]: “/usr/local/cpanel/scripts/restartsrv_apache_php_fpm --graceful” called by (3476 - /usr/local/cpanel/scripts/restartsrv_httpd --no-verbose)
[2021-04-20 14:15:34 -0400]: “/usr/local/cpanel/scripts/restartsrv_cpsrvd ” called by (3480 - queueprocd - process - restartsrv)
[2021-04-20 14:15:34 -0400] info [cpsrvd] Restarting cpsrvd daemon process 4503 via /usr/local/cpanel/cpsrvd
==> cpsrvd 11.94.0.4 started
==> cpsrvd: loading security policy....Done
==> cpsrvd: Setting up SSL support ... Done
==> cpsrvd: transferred port bindings: 10,11,12,3,4,5,6,7,8,9
==> cpsrvd: bound to ports
[2021-04-20 14:15:34 -0400] info [cpsrvd] version 11.94.0.4 online
[2021-04-20 14:15:35 -0400]: “/usr/local/cpanel/scripts/restartsrv_apache_php_fpm --graceful” called by (3476 - /usr/local/cpanel/scripts/restartsrv_httpd --no-verbose)
[2021-04-20 14:16:29 -0400]: “/usr/local/cpanel/scripts/restartsrv_httpd --start” called by (4643 - queueprocd - process - apache_restart - safeaprestart - forced restart)
[2021-04-20 14:16:29 -0400]: “/usr/local/cpanel/scripts/restartsrv_apache_php_fpm --graceful” called by (4796 - /usr/local/cpanel/scripts/restartsrv_httpd --start)
[2021-04-20 14:16:31 -0400]: “/usr/local/cpanel/scripts/restartsrv_apache_php_fpm --graceful” called by (4796 - /usr/local/cpanel/scripts/restartsrv_httpd --start)
[2021-04-20 14:16:32 -0400]: “/usr/local/cpanel/scripts/restartsrv_apache_php_fpm --graceful” called by (4842 - /usr/local/cpanel/scripts/restartsrv_httpd --no-verbose)
[2021-04-20 14:16:33 -0400] info [tailwatchd] Updated chkservd exim driver from /usr/local/cpanel/src/chkservd/chkserv.d.
[2021-04-20 14:16:33 -0400] info [tailwatchd] Updated chkservd spamd driver from /usr/local/cpanel/src/chkservd/chkserv.d.
[2021-04-20 14:16:33 -0400] info [tailwatchd] Updated chkservd apache_php_fpm driver from /usr/local/cpanel/src/chkservd/chkserv.d.
[2021-04-20 14:16:33 -0400] info [tailwatchd] Updated chkservd pop driver from /usr/local/cpanel/src/chkservd/chkserv.d.
[2021-04-20 14:16:33 -0400] info [tailwatchd] Updated chkservd cphulkd driver from /usr/local/cpanel/src/chkservd/chkserv.d.
[2021-04-20 14:16:33 -0400] info [tailwatchd] Updated chkservd p0f driver from /usr/local/cpanel/src/chkservd/chkserv.d.
[2021-04-20 14:16:33 -0400] info [tailwatchd] Updated chkservd sshd driver from /usr/local/cpanel/src/chkservd/chkserv.d.
[2021-04-20 14:16:33 -0400] info [tailwatchd] Updated chkservd named driver from /usr/local/cpanel/src/chkservd/chkserv.d.
[2021-04-20 14:16:33 -0400] info [tailwatchd] Updated chkservd cpanel_php_fpm driver from /usr/local/cpanel/src/chkservd/chkserv.d.
[2021-04-20 14:16:33 -0400] info [tailwatchd] Updated chkservd cpdavd driver from /usr/local/cpanel/src/chkservd/chkserv.d.
[2021-04-20 14:16:33 -0400] info [tailwatchd] Updated chkservd lmtp driver from /usr/local/cpanel/src/chkservd/chkserv.d.
[2021-04-20 14:16:33 -0400] info [tailwatchd] Updated chkservd nscd driver from /usr/local/cpanel/src/chkservd/chkserv.d.
[2021-04-20 14:16:33 -0400] info [tailwatchd] Updated chkservd crond driver from /usr/local/cpanel/src/chkservd/chkserv.d.
[2021-04-20 14:16:33 -0400] info [tailwatchd] Updated chkservd dnsadmin driver from /usr/local/cpanel/src/chkservd/chkserv.d.
[2021-04-20 14:16:33 -0400] info [tailwatchd] Updated chkservd rsyslogd driver from /usr/local/cpanel/src/chkservd/chkserv.d.
[2021-04-20 14:16:33 -0400] info [tailwatchd] Updated chkservd postgresql driver from /usr/local/cpanel/src/chkservd/chkserv.d.
[2021-04-20 14:16:33 -0400] info [tailwatchd] Updated chkservd ipaliases driver from /usr/local/cpanel/src/chkservd/chkserv.d.
[2021-04-20 14:16:33 -0400] info [tailwatchd] Updated chkservd imap driver from /usr/local/cpanel/src/chkservd/chkserv.d.
[2021-04-20 14:16:33 -0400] info [tailwatchd] Updated chkservd ftpd driver from /usr/local/cpanel/src/chkservd/chkserv.d.
[2021-04-20 14:16:33 -0400] info [tailwatchd] Updated chkservd httpd driver from /usr/local/cpanel/src/chkservd/chkserv.d.
[2021-04-20 14:16:33 -0400] info [tailwatchd] Updated chkservd mailman driver from /usr/local/cpanel/src/chkservd/chkserv.d.
[2021-04-20 14:16:33 -0400] info [tailwatchd] Updated chkservd cpgreylistd driver from /usr/local/cpanel/src/chkservd/chkserv.d.
[2021-04-20 14:16:33 -0400] info [tailwatchd] Updated chkservd mysql driver from /usr/local/cpanel/src/chkservd/chkserv.d.
[2021-04-20 14:16:33 -0400] info [tailwatchd] Updated chkservd clamd driver from /usr/local/cpanel/src/chkservd/chkserv.d.
[2021-04-20 14:16:33 -0400]: “/usr/local/cpanel/scripts/restartsrv_apache_php_fpm --graceful” called by (4842 - /usr/local/cpanel/scripts/restartsrv_httpd --no-verbose)
==> cpsrvd 11.94.0.4 started
==> cpsrvd: loading security policy....Done
==> cpsrvd: Setting up SSL support ... Done
==> cpsrvd: transferred port bindings: 10,11,12,3,4,5,6,7,8,9
==> cpsrvd: bound to ports
[2021-04-20 14:16:34 -0400] info [cpsrvd] version 11.94.0.4 online
[2021-04-20 14:19:14 -0400] info [cpsrvd] reloading config based on -HUP signal
[2021-04-20 14:19:26 -0400] info [whostmgr] Whostmgr::Services::_restart_services: lfd
[2021-04-20 14:19:26 -0400]: “/usr/local/cpanel/scripts/restartsrv_unknown --html --wait --verbose lfd” called by (3169 - whostmgr - dosrvmng)
[2021-04-20 14:19:26 -0400] info [restartsrv_unknown] systemd failed to start the service “lfd” (The “/usr/bin/systemctl restart lfd.service --no-ask-password” command (process 8467) reported error number 1 when it ended.): Job for lfd.service failed because the control process exited with error code. See "systemctl status lfd.service" and "journalctl -xe" for details.

[2021-04-20 14:19:32 -0400]: “/usr/local/cpanel/scripts/restartsrv_tailwatchd ” called by (8535 - queueprocd - process - restartsrv)
[2021-04-20 14:19:33 -0400] info [whostmgrd] Successfully verified signature for cpanel (key types: release).
==> cpsrvd 11.94.0.4 started
==> cpsrvd: loading security policy....Done
==> cpsrvd: Setting up SSL support ... Done
==> cpsrvd: transferred port bindings: 10,11,12,3,4,5,6,7,8,9
==> cpsrvd: bound to ports
[2021-04-20 14:29:54 -0400] info [cpsrvd] version 11.94.0.4 online
==> cpsrvd 11.94.0.4 started
==> cpsrvd: loading security policy....Done
==> cpsrvd: Setting up SSL support ... Done
==> cpsrvd: transferred port bindings: 10,11,12,3,4,5,6,7,8,9
==> cpsrvd: bound to ports
[2021-04-20 14:34:57 -0400] info [cpsrvd] version 11.94.0.4 online
[2021-04-20 14:36:00 -0400] info [cpsrvd] reloading config based on -HUP signal
==> cpsrvd 11.94.0.4 started
==> cpsrvd: loading security policy....Done
==> cpsrvd: Setting up SSL support ... Done
==> cpsrvd: transferred port bindings: 10,11,12,3,4,5,6,7,8,9
==> cpsrvd: bound to ports
[2021-04-20 14:40:05 -0400] info [cpsrvd] version 11.94.0.4 online

cPRex said:
If not, do you see any high resource usage on the machine?

Not really, no. The server load was 1.32 at the time of attempt # 121 (according to the email I received), which is a normal load for me at that time of day. Right now it's at 0.72. Most of the CPU (15.22%) and memory (19.10%) is going to MySQL.

cPRex · Apr 20, 2021

The "server produced an operating system error" message there is certainly concerning. Is there anything odd in /var/log/messages on that machine?

GoWilkes · Apr 21, 2021

Not that I can tell... I see that error message was at 14:12 and 14:13, but /var/log/messages near that time shows:

Code:

Apr 20 14:01:25 epsilon systemd: cpanel-ccs.service failed.
Apr 20 14:05:01 epsilon systemd: Started Session 1833978 of user root.
Apr 20 14:05:01 epsilon systemd: Started Session 1833977 of user root.
Apr 20 14:05:01 epsilon systemd: Started Session 1833979 of user munin.
Apr 20 14:06:29 epsilon pure-ftpd: ([email protected]) [INFO] New connection from 127.0.0.1
Apr 20 14:06:29 epsilon pure-ftpd: ([email protected]) [INFO] __cpanel__service__auth__ftpd__OCo_DDBHKTGX2BEc is now logged in
Apr 20 14:06:29 epsilon pure-ftpd: ([email protected]) [INFO] Logout.
Apr 20 14:06:30 epsilon systemd: Cannot add dependency job for unit systemd-vconsole-setup.service, ignoring: Unit is masked.
Apr 20 14:06:30 epsilon systemd: Starting Apple Calendar Server...
Apr 20 14:06:30 epsilon ccs_init: Using /opt/cpanel-ccs/.develop/virtualenv/bin/python as Python
Apr 20 14:06:30 epsilon ccs_init: Starting server...
Apr 20 14:06:30 epsilon ccs_init: /opt/cpanel-ccs/bin/caldavd  -f /opt/cpanel-ccs/conf/caldavd-dev.plist -P caldav -t Combined
Apr 20 14:06:33 epsilon systemd: Started Apple Calendar Server.
Apr 20 14:06:34 epsilon ccs_init: kill = true
Apr 20 14:06:34 epsilon ccs_init: restart = false
Apr 20 14:06:36 epsilon ccs_init: pidfile = /opt/cpanel-ccs/./data/Logs/state/caldavd.pid
Apr 20 14:06:36 epsilon ccs_init: Unreadable PID file: /opt/cpanel-ccs/./data/Logs/state/caldavd.pid
Apr 20 14:06:36 epsilon ccs_init: Killing processs by name user and name..
Apr 20 14:06:36 epsilon ccs_init: Done killing processes
Apr 20 14:06:36 epsilon systemd: cpanel-ccs.service: control process exited, code=exited status=1
Apr 20 14:06:36 epsilon systemd: Unit cpanel-ccs.service entered failed state.
Apr 20 14:06:36 epsilon systemd: cpanel-ccs.service failed.

# Note from me, this next looks like a repeat of the above
Apr 20 14:09:01 epsilon systemd: Started Session 1834058 of user root.
Apr 20 14:10:01 epsilon systemd: Started Session 1834133 of user root.
Apr 20 14:10:02 epsilon systemd: Started Session 1834132 of user root.
Apr 20 14:10:02 epsilon systemd: Started Session 1834134 of user munin.
Apr 20 14:11:01 epsilon systemd: Started Session 1834150 of user root.
Apr 20 14:11:32 epsilon pure-ftpd: ([email protected]) [INFO] New connection from 127.0.0.1
Apr 20 14:11:32 epsilon pure-ftpd: ([email protected]) [INFO] __cpanel__service__auth__ftpd__OCo_DDBHKTGX2BEc is now logged in
Apr 20 14:11:32 epsilon pure-ftpd: ([email protected]) [INFO] Logout.
Apr 20 14:11:33 epsilon systemd: Cannot add dependency job for unit systemd-vconsole-setup.service, ignoring: Unit is masked.
Apr 20 14:11:33 epsilon systemd: Starting Apple Calendar Server...
Apr 20 14:11:33 epsilon ccs_init: Using /opt/cpanel-ccs/.develop/virtualenv/bin/python as Python
Apr 20 14:11:33 epsilon ccs_init: Starting server...
Apr 20 14:11:33 epsilon ccs_init: /opt/cpanel-ccs/bin/caldavd  -f /opt/cpanel-ccs/conf/caldavd-dev.plist -P caldav -t Combined
Apr 20 14:11:36 epsilon systemd: Started Apple Calendar Server.
Apr 20 14:11:37 epsilon ccs_init: kill = true
Apr 20 14:11:37 epsilon ccs_init: restart = false
Apr 20 14:11:40 epsilon ccs_init: pidfile = /opt/cpanel-ccs/./data/Logs/state/caldavd.pid
Apr 20 14:11:40 epsilon ccs_init: Unreadable PID file: /opt/cpanel-ccs/./data/Logs/state/caldavd.pid
Apr 20 14:11:40 epsilon ccs_init: Killing processs by name user and name..
Apr 20 14:11:40 epsilon ccs_init: Done killing processes
Apr 20 14:11:40 epsilon systemd: cpanel-ccs.service: control process exited, code=exited status=1
Apr 20 14:11:40 epsilon systemd: Unit cpanel-ccs.service entered failed state.
Apr 20 14:11:40 epsilon systemd: cpanel-ccs.service failed.

# Note from me, I think that this is all from where I restarted Service Manager
# I had disabled CSF, which explains the CSF error at 14:19:26
Apr 20 14:15:01 epsilon systemd: Started Session 1834226 of user root.
Apr 20 14:15:01 epsilon systemd: Started Session 1834227 of user munin.
Apr 20 14:15:01 epsilon systemd: Started Session 1834228 of user root.
Apr 20 14:15:17 epsilon systemd: Reloading.
Apr 20 14:15:17 epsilon systemd: Configuration file /etc/systemd/system/cpanel-ccs.service is marked world-inaccessible. This has no effect as configuration data is accessible via APIs without restrictions. Proceeding anyway.
Apr 20 14:15:17 epsilon systemd: Dependency Before=cpanel-ccs.service dropped from unit cpanel-ccs.service
Apr 20 14:15:17 epsilon systemd: [/usr/lib/systemd/system/vzfifo.service:20] Support for option SysVStartPriority= has been removed and it is ignored
Apr 20 14:15:18 epsilon systemd: Got notification message from PID 3127, but reception only permitted for main PID 18965
Apr 20 14:15:19 epsilon systemd: Stopping Apache web server managed by cPanel EasyApache...
Apr 20 14:15:19 epsilon systemd: Configuration file /etc/systemd/system/cpanel-ccs.service is marked world-inaccessible. This has no effect as configuration data is accessible via APIs without restrictions. Proceeding anyway.
Apr 20 14:15:19 epsilon systemd: Cannot add dependency job for unit systemd-vconsole-setup.service, ignoring: Unit is masked.
Apr 20 14:15:19 epsilon systemd: Starting Apple Calendar Server...
Apr 20 14:15:19 epsilon ccs_init: Using /opt/cpanel-ccs/.develop/virtualenv/bin/python as Python
Apr 20 14:15:19 epsilon ccs_init: Starting server...
Apr 20 14:15:19 epsilon ccs_init: /opt/cpanel-ccs/bin/caldavd  -f /opt/cpanel-ccs/conf/caldavd-dev.plist -P caldav -t Combined
Apr 20 14:15:19 epsilon systemd: Stopping Exim is a Mail Transport Agent, which is the program that moves mail from one machine to another....
Apr 20 14:15:22 epsilon systemd: Started Apple Calendar Server.
Apr 20 14:15:23 epsilon systemd: Stopped Exim is a Mail Transport Agent, which is the program that moves mail from one machine to another..
Apr 20 14:15:23 epsilon systemd: Cannot add dependency job for unit systemd-vconsole-setup.service, ignoring: Unit is masked.
Apr 20 14:15:23 epsilon systemd: Starting mailman services...
Apr 20 14:15:23 epsilon systemd: Starting Exim is a Mail Transport Agent, which is the program that moves mail from one machine to another....
Apr 20 14:15:23 epsilon ccs_init: kill = true
Apr 20 14:15:23 epsilon ccs_init: restart = false
Apr 20 14:15:23 epsilon systemd: Started Exim is a Mail Transport Agent, which is the program that moves mail from one machine to another..
Apr 20 14:15:24 epsilon restartsrv_mailman: (XID mwtrf7) The “mailman” service is disabled.
Apr 20 14:15:24 epsilon systemd: mailman.service: control process exited, code=exited status=255
Apr 20 14:15:24 epsilon systemd: Failed to start mailman services.
Apr 20 14:15:24 epsilon systemd: Unit mailman.service entered failed state.
Apr 20 14:15:24 epsilon systemd: mailman.service failed.
Apr 20 14:15:25 epsilon systemd: Stopping Dovecot Imap Server...
Apr 20 14:15:25 epsilon dovecot_cpshutdown: Opened “/var/run/dovecot/master.pid” …
Apr 20 14:15:25 epsilon dovecot_cpshutdown: Master Dovecot process = 1193
Apr 20 14:15:25 epsilon dovecot_cpshutdown: Executing “/usr/sbin/dovecot stop” …
Apr 20 14:15:26 epsilon ccs_init: pidfile = /opt/cpanel-ccs/./data/Logs/state/caldavd.pid
Apr 20 14:15:26 epsilon ccs_init: Unreadable PID file: /opt/cpanel-ccs/./data/Logs/state/caldavd.pid
Apr 20 14:15:26 epsilon systemd: cpanel-ccs.service: control process exited, code=exited status=1
Apr 20 14:15:26 epsilon ccs_init: Killing processs by name user and name..
Apr 20 14:15:26 epsilon ccs_init: Done killing processes
Apr 20 14:15:26 epsilon systemd: Unit cpanel-ccs.service entered failed state.
Apr 20 14:15:26 epsilon systemd: cpanel-ccs.service failed.
Apr 20 14:15:27 epsilon dovecot_cpshutdown: Waiting 30 seconds for process 3315 to end …
Apr 20 14:15:27 epsilon dovecot_cpshutdown: Done! Waiting 30 seconds for process 1193 to end …
Apr 20 14:15:27 epsilon dovecot_cpshutdown: Dovecot is now shut down.
Apr 20 14:15:27 epsilon dovecot_cpshutdown: Any remaining Dovecot processes will now be terminated.
Apr 20 14:15:27 epsilon dovecot_cpshutdown: Waiting for dovecot,dovecot-auth,dovecot/pop3-login,dovecot/imap-login,dovecot/anvil,dovecot/log,dovecot/config,dovecot/auth,dovecot-wrap to shutdown ... not running.
Apr 20 14:15:27 epsilon systemd: Stopped Dovecot Imap Server.
Apr 20 14:15:27 epsilon systemd: Cannot add dependency job for unit systemd-vconsole-setup.service, ignoring: Unit is masked.
Apr 20 14:15:27 epsilon systemd: Started Dovecot Imap Server.
Apr 20 14:15:29 epsilon systemd: Stopped Apache web server managed by cPanel EasyApache.
Apr 20 14:15:34 epsilon systemd: Cannot add dependency job for unit systemd-vconsole-setup.service, ignoring: Unit is masked.
Apr 20 14:15:34 epsilon systemd: Starting Apache web server managed by cPanel EasyApache...
Apr 20 14:15:37 epsilon systemd: Can't open PID file /run/apache2/httpd.pid (yet?) after start: No such file or directory
Apr 20 14:15:38 epsilon systemd: Started Apache web server managed by cPanel EasyApache.
Apr 20 14:16:19 epsilon systemd: Stopping Apache web server managed by cPanel EasyApache...
Apr 20 14:16:29 epsilon systemd: Stopped Apache web server managed by cPanel EasyApache.
Apr 20 14:16:32 epsilon systemd: Cannot add dependency job for unit systemd-vconsole-setup.service, ignoring: Unit is masked.
Apr 20 14:16:32 epsilon systemd: Starting Apache web server managed by cPanel EasyApache...
Apr 20 14:16:34 epsilon pure-ftpd: ([email protected]) [INFO] New connection from 127.0.0.1
Apr 20 14:16:34 epsilon pure-ftpd: ([email protected]) [INFO] __cpanel__service__auth__ftpd__OCo_DDBHKTGX2BEc is now logged in
Apr 20 14:16:34 epsilon pure-ftpd: ([email protected]) [INFO] Logout.
Apr 20 14:16:35 epsilon systemd: Can't open PID file /run/apache2/httpd.pid (yet?) after start: No such file or directory
Apr 20 14:16:36 epsilon systemd: Started Apache web server managed by cPanel EasyApache.
Apr 20 14:17:22 epsilon systemd: Cannot add dependency job for unit systemd-vconsole-setup.service, ignoring: Unit is masked.
Apr 20 14:17:22 epsilon systemd: Starting Apple Calendar Server...
Apr 20 14:17:22 epsilon ccs_init: Using /opt/cpanel-ccs/.develop/virtualenv/bin/python as Python
Apr 20 14:17:22 epsilon ccs_init: Starting server...
Apr 20 14:17:22 epsilon ccs_init: /opt/cpanel-ccs/bin/caldavd  -f /opt/cpanel-ccs/conf/caldavd-dev.plist -P caldav -t Combined
Apr 20 14:17:26 epsilon systemd: Started Apple Calendar Server.
Apr 20 14:17:27 epsilon ccs_init: kill = true
Apr 20 14:17:27 epsilon ccs_init: restart = false
Apr 20 14:17:30 epsilon ccs_init: pidfile = /opt/cpanel-ccs/./data/Logs/state/caldavd.pid
Apr 20 14:17:30 epsilon ccs_init: Unreadable PID file: /opt/cpanel-ccs/./data/Logs/state/caldavd.pid
Apr 20 14:17:30 epsilon ccs_init: Killing processs by name user and name..
Apr 20 14:17:30 epsilon ccs_init: Done killing processes
Apr 20 14:17:30 epsilon systemd: cpanel-ccs.service: control process exited, code=exited status=1
Apr 20 14:17:30 epsilon systemd: Unit cpanel-ccs.service entered failed state.
Apr 20 14:17:30 epsilon systemd: cpanel-ccs.service failed.
Apr 20 14:19:26 epsilon systemd: Cannot add dependency job for unit systemd-vconsole-setup.service, ignoring: Unit is masked.
Apr 20 14:19:26 epsilon systemd: Starting ConfigServer Firewall & Security - lfd...
Apr 20 14:19:26 epsilon lfd: csf and lfd have been disabled
Apr 20 14:19:26 epsilon systemd: lfd.service: control process exited, code=exited status=1
Apr 20 14:19:26 epsilon systemd: Failed to start ConfigServer Firewall & Security - lfd.
Apr 20 14:19:26 epsilon systemd: Unit lfd.service entered failed state.
Apr 20 14:19:26 epsilon systemd: lfd.service failed.
Apr 20 14:19:34 epsilon pure-ftpd: ([email protected]) [INFO] New connection from 127.0.0.1
Apr 20 14:19:34 epsilon pure-ftpd: ([email protected]) [INFO] __cpanel__service__auth__ftpd__OCo_DDBHKTGX2BEc is now logged in
Apr 20 14:19:34 epsilon pure-ftpd: ([email protected]) [INFO] Logout.

cPRex · Apr 21, 2021

If you'd like to submit a ticket to our team, that might be best since we're not finding much in the logs.

DataCenterGuy · Aug 25, 2021

cPRex said:
If you'd like to submit a ticket to our team, that might be best since we're not finding much in the logs.

Was a resolution found for this issue? Were seeing the same problem with the production version v98.0.5 "cpanel-ccs" on CentOS 7.9. The issue seems to center around "ccs_init: Unreadable PID file: /opt/cpanel-ccs/./data/Logs/state/caldavd.pid" The file does not exist.

cPRex · Aug 25, 2021

@GoWilkes - did you ever find out more about this one?

GoWilkes · Aug 25, 2021

Nope, I just ended up uninstalling CCS :-(

cPRex · Aug 25, 2021

@DataCenterGuy - feel free to submit a ticket to our team and we'll check it out!

garconcn · Sep 7, 2021

I had the same issue and found that my CSF killed the processes for user "cpanel-ccs". I just added "user:cpanel-ccs" to /etc/csf/csf.pignore to fix the issue.

Thread starter	Similar threads	Forum	Replies	Date
D	Cpanel / WHM Left Column Font Changed	Server Management	5	Feb 1, 2023
B	How to limit CPU usage per cPanel account?	Server Management	13	Jan 30, 2023
Q	In Progress CPANEL-39262 - Errors in EasyApache provisioning	Server Management	8	Jan 20, 2023
W	WHM & cPanel Slow On HTTPS	Server Management	1	Jan 19, 2023
C	Cant' accsess my cpanel	Server Management	5	Aug 18, 2008

Search

Search

cpanel-ccs failing

GoWilkes

Well-Known Member

cPRex

Jurassic Moderator

GoWilkes

Well-Known Member

cPRex

Jurassic Moderator

GoWilkes

Well-Known Member

cPRex

Jurassic Moderator

DataCenterGuy

Registered

cPRex

Jurassic Moderator

GoWilkes

Well-Known Member

cPRex

Jurassic Moderator

garconcn

Well-Known Member