tailwatchd The service “tailwatchd” appears to be down

A

ally1

Member
Nov 13, 2014
12
4
3
cPanel Access Level
Root Administrator
Hello WHM & cPanel support and community,

I encountered the tailwatchd problem. I got a lot of emails with the following messages:

Code: 
---------- Forwarded message ----------
From: cPanel Monitoring <[email protected]***my_server***>
Date: Wed, Nov 29, 2017 at 6:37 AM
Subject: [***my_server***] FAILED : tailwatchd
(10.0.131.33)
To: [email protected]_here


The service “tailwatchd” appears to be down.

Server ***my_server***
Primary IP Address 10.0.131.33
Service Name tailwatchd
Service Status failed
Notification The service “tailwatchd” appears to be down.
Service Check Raw Output Cpanel::Exception::Timeout/(XID d5nfna) The system
failed to lock the file “/var/run/restartsrv_tailwatchd” after 121 seconds.
at /usr/local/cpanel/Cpanel/Exception/CORE.pm line 336.
Cpanel::Exception::create("Timeout", "The system failed to lock the file
\x{e2}\x{80}\x{9c}[_1]\x{e2}\x{80}\x{9d} after [quant,_2"...,
ARRAY(0x2928f60)) called at /usr/local/cpanel/Cpanel/Exception.pm line 46
Cpanel::Exception::__ANON__(__CPANEL_HIDDEN__, __CPANEL_HIDDEN__...,
ARRAY(0x2928f60)) called at /usr/local/cpanel/Cpanel/SafeFile.pm line 710
Cpanel::SafeFile::_timeout_exception("/var/run/restartsrv_tailwatchd", 121)
called at /usr/local/cpanel/Cpanel/SafeFile.pm line 742
Cpanel::SafeFile::_die_if_file_is_flocked_cuz_already_
waited_a_while("/var/run/restartsrv_tailwatchd", 121) called at
/usr/local/cpanel/Cpanel/SafeFile.pm line 811
Cpanel::SafeFile::_lock_wait("/var/run/restartsrv_tailwatchd") called at
/usr/local/cpanel/Cpanel/SafeFile.pm line 339
Cpanel::SafeFile::_safelock("/var/run/restartsrv_tailwatchd") called at
/usr/local/cpanel/Cpanel/SafeFile.pm line 538
Cpanel::SafeFile::_safe_open(undef, ">>", "/var/run/restartsrv_tailwatchd",
CODE(0xc0ea38), "safeopen") called at /usr/local/cpanel/Cpanel/SafeFile.pm
line 107
Cpanel::SafeFile::safeopen(undef, ">>", "/var/run/restartsrv_tailwatchd")
called at /usr/local/cpanel/Cpanel/RestartSrv/Lock.pm line 25
Cpanel::RestartSrv::Lock::new("Cpanel::RestartSrv::Lock", "tailwatchd")
called at /usr/local/cpanel/Cpanel/ServiceManager/Base.pm line 183
Cpanel::ServiceManager::Base::lock(Cpanel::ServiceManager::
Services::Tailwatchd=HASH(0x29036e8)) called at
/usr/local/cpanel/Cpanel/ServiceManager/Base.pm
line 459
Cpanel::ServiceManager::Base::BUILD(Cpanel::ServiceManager::
Services::Tailwatchd=HASH(0x29036e8), HASH(0xc4bd58)) called at
/usr/local/cpanel/3rdparty/perl/524/lib64/perl5/cpanel_
lib/x86_64-linux-64int/Cpanel/Class/Object.pm line 49
Cpanel::Class::Object::BUILDALL(Cpanel::ServiceManager::Services::Tailwatchd=HASH(0x29036e8),
HASH(0xc4bd58)) called at /usr/local/cpanel/3rdparty/
perl/524/lib64/perl5/cpanel_lib/x86_64-linux-64int/Cpanel/Class/Object.pm
line 41
Cpanel::Class::Object::new("Cpanel::ServiceManager::Services::Tailwatchd",
"service", "tailwatchd") called at /usr/local/cpanel/Cpanel/ServiceManager.pm
line 99
Cpanel::ServiceManager::new("Cpanel::ServiceManager", "service",
"tailwatchd") called at bin/restartsrv_base.pl line 62
main::__ANON__() called at /usr/local/cpanel/3rdparty/
perl/524/lib64/perl5/cpanel_lib/Try/Tiny.pm line 99
eval {...} called at
/usr/local/cpanel/3rdparty/perl/524/lib64/perl5/cpanel_lib/Try/Tiny.pm
line 90
Try::Tiny::try(CODE(0xc7c400), Try::Tiny::Catch=REF(0xbe3310)) called at
bin/restartsrv_base.pl line 109
Startup Log No startup log
Memory Information
Used 3.04 GB
Available 12.48 GB
Installed 15.52 GB
Load Information 1.80 1.54 1.40
Uptime 11 days, 10 hours, 18 minutes, and 20 seconds
IOStat Information avg-cpu: %user %nice %system %iowait %steal %idle 26.47
0.67 3.81 31.23 0.05 37.77 Device: tps Blk_read/s Blk_wrtn/s Blk_read
Blk_wrtn xvda 216.64 2090.67 932.57 2064534465 920912136 xvdg 11.34 97.29
103.31 96072106 102014040 xvdh 0.73 18.93 6.78 18696002 6695808 xvdf 2.39
112.79 24.97 111379016 24658896 xvdi 6.53 70.58 526.69 69696360 520108328
Top Processes
PID Owner CPU % Memory % Command
85284 msarts 8.53 0.40 php-fpm: pool msarts_org
85056 factnotfiction 4.44 0.32 php-fpm: pool factnotfiction_com
84481 factnotfiction 3.91 0.32 php-fpm: pool factnotfiction_com
83298 factnotfiction 3.59 0.32 php-fpm: pool factnotfiction_com
83214 factnotfiction 3.30 0.32 php-fpm: pool factnotfiction_com

Configure Monitor Settings:
https://***my_server***:2087/scripts/srvmng#service-chkservd

Configure chkservd:
https://***my_server***:2087/scripts2/tweaksettings?find=
chkservd

Disable HTML notifications:
https://***my_server***:2087/scripts2/tweaksettings?find=
chkservd_plaintext_notify

The system generated this notice on 2017-11-29 at 12:37:02 UTC.

“Service failures (ChkServd)” notifications are currently configured to
have an importance of “High”. You can change the importance or disable this
type of notification in WHM’s Contact Manager at: https://***my_server***
:2087/scripts2/editcontact?event=chkservd

Do not reply to this automated message.
[image: cP]

Copyright© 2017 cPanel, Inc.
The command /scripts/restartsrv_tailwatchd --status throws the next:

Code: 
################
Cpanel::Exception::Timeout/(XID 8k6uua) The system failed to lock the file “/var/run/restartsrv_tailwatchd” after 121 seconds.
 at /usr/local/cpanel/Cpanel/Exception/CORE.pm line 336.
        Cpanel::Exception::create("Timeout", "The system failed to lock the file \x{e2}\x{80}\x{9c}[_1]\x{e2}\x{80}\x{9d} after [quant,_2"..., ARRAY(0x1696ce8)) called at /usr/local/cpanel/Cpanel/Exception.pm line 46
        Cpanel::Exception::__ANON__(__CPANEL_HIDDEN__, __CPANEL_HIDDEN__..., ARRAY(0x1696ce8)) called at /usr/local/cpanel/Cpanel/SafeFile.pm line 710
        Cpanel::SafeFile::_timeout_exception("/var/run/restartsrv_tailwatchd", 121) called at /usr/local/cpanel/Cpanel/SafeFile.pm line 742
        Cpanel::SafeFile::_die_if_file_is_flocked_cuz_already_waited_a_while("/var/run/restartsrv_tailwatchd", 121) called at /usr/local/cpanel/Cpanel/SafeFile.pm line 811
        Cpanel::SafeFile::_lock_wait("/var/run/restartsrv_tailwatchd") called at /usr/local/cpanel/Cpanel/SafeFile.pm line 339
        Cpanel::SafeFile::_safelock("/var/run/restartsrv_tailwatchd") called at /usr/local/cpanel/Cpanel/SafeFile.pm line 538
        Cpanel::SafeFile::_safe_open(undef, ">>", "/var/run/restartsrv_tailwatchd", CODE(0xc0ea38), "safeopen") called at /usr/local/cpanel/Cpanel/SafeFile.pm line 107
        Cpanel::SafeFile::safeopen(undef, ">>", "/var/run/restartsrv_tailwatchd") called at /usr/local/cpanel/Cpanel/RestartSrv/Lock.pm line 25
        Cpanel::RestartSrv::Lock::new("Cpanel::RestartSrv::Lock", "tailwatchd") called at /usr/local/cpanel/Cpanel/ServiceManager/Base.pm line 183
        Cpanel::ServiceManager::Base::lock(Cpanel::ServiceManager::Services::Tailwatchd=HASH(0x16729a0)) called at /usr/local/cpanel/Cpanel/ServiceManager/Base.pm line 459
        Cpanel::ServiceManager::Base::BUILD(Cpanel::ServiceManager::Services::Tailwatchd=HASH(0x16729a0), HASH(0xc4bd58)) called at /usr/local/cpanel/3rdparty/perl/524/lib64/perl5/cpanel_lib/x86_64-linux-64int/Cpanel/Class/Object.pm line 49
        Cpanel::Class::Object::BUILDALL(Cpanel::ServiceManager::Services::Tailwatchd=HASH(0x16729a0), HASH(0xc4bd58)) called at /usr/local/cpanel/3rdparty/perl/524/lib64/perl5/cpanel_lib/x86_64-linux-64int/Cpanel/Class/Object.pm line 41
        Cpanel::Class::Object::new("Cpanel::ServiceManager::Services::Tailwatchd", "service", "tailwatchd") called at /usr/local/cpanel/Cpanel/ServiceManager.pm line 99
        Cpanel::ServiceManager::new("Cpanel::ServiceManager", "service", "tailwatchd") called at bin/restartsrv_base.pl line 62
        main::__ANON__() called at /usr/local/cpanel/3rdparty/perl/524/lib64/perl5/cpanel_lib/Try/Tiny.pm line 99
        eval {...} called at /usr/local/cpanel/3rdparty/perl/524/lib64/perl5/cpanel_lib/Try/Tiny.pm line 90
        Try::Tiny::try(CODE(0xc7d078), Try::Tiny::Catch=REF(0xbe3310)) called at bin/restartsrv_base.pl line 109

################
The /etc/init.d/cpanel restart command throws the same error.
Also I tried to update 'cPanel' server with "/scripts/upcp" command. The cpanel was updated successfully but there still the problem with updating tailwatchd, here is output:

Code: 
################
*** Running restart_tailwatchd_if_updated ***
tailwatchd has newer code than the current running version.  Restarting
Cpanel::Exception::Timeout/(XID ym38rd) The system failed to lock the file <E2><80><9C>/var/run/restartsrv_tailwatchd<E2><80><9D> after 121 seconds.
 at /usr/local/cpanel/Cpanel/Exception/CORE.pm line 336.
        Cpanel::Exception::create("Timeout", "The system failed to lock the file \x{e2}\x{80}\x{9c}[_1]\x{e2}\x{80}\x{9d} after [quant,_2"..., ARRAY(0x1a874c8)) called at /usr/local/cpanel/Cpanel/Exception.pm line 46
        Cpanel::Exception::__ANON__(__CPANEL_HIDDEN__, __CPANEL_HIDDEN__..., ARRAY(0x1a874c8)) called at /usr/local/cpanel/Cpanel/SafeFile.pm line 710
        Cpanel::SafeFile::_timeout_exception("/var/run/restartsrv_tailwatchd", 121) called at /usr/local/cpanel/Cpanel/SafeFile.pm line 742
        Cpanel::SafeFile::_die_if_file_is_flocked_cuz_already_waited_a_while("/var/run/restartsrv_tailwatchd", 121) called at /usr/local/cpanel/Cpanel/SafeFile.pm line 811
        Cpanel::SafeFile::_lock_wait("/var/run/restartsrv_tailwatchd") called at /usr/local/cpanel/Cpanel/SafeFile.pm line 339
        Cpanel::SafeFile::_safelock("/var/run/restartsrv_tailwatchd") called at /usr/local/cpanel/Cpanel/SafeFile.pm line 538
        Cpanel::SafeFile::_safe_open(undef, ">>", "/var/run/restartsrv_tailwatchd", CODE(0xc0ea38), "safeopen") called at /usr/local/cpanel/Cpanel/SafeFile.pm line 107
        Cpanel::SafeFile::safeopen(undef, ">>", "/var/run/restartsrv_tailwatchd") called at /usr/local/cpanel/Cpanel/RestartSrv/Lock.pm line 25
        Cpanel::RestartSrv::Lock::new("Cpanel::RestartSrv::Lock", "tailwatchd") called at /usr/local/cpanel/Cpanel/ServiceManager/Base.pm line 183
        Cpanel::ServiceManager::Base::lock(Cpanel::ServiceManager::Services::Tailwatchd=HASH(0x1a5caa8)) called at /usr/local/cpanel/Cpanel/ServiceManager/Base.pm line 459
        Cpanel::ServiceManager::Base::BUILD(Cpanel::ServiceManager::Services::Tailwatchd=HASH(0x1a5caa8), HASH(0xc4bd58)) called at /usr/local/cpanel/3rdparty/perl/524/lib64/perl5/cpanel_lib/x86_64-linux-64int/Cpanel/Class/Object.pm line 49
        Cpanel::Class::Object::BUILDALL(Cpanel::ServiceManager::Services::Tailwatchd=HASH(0x1a5caa8), HASH(0xc4bd58)) called at /usr/local/cpanel/3rdparty/perl/524/lib64/perl5/cpanel_lib/x86_64-linux-64int/Cpanel/Class/Object.pm line 41
        Cpanel::Class::Object::new("Cpanel::ServiceManager::Services::Tailwatchd", "service", "tailwatchd") called at /usr/local/cpanel/Cpanel/ServiceManager.pm line 99
        Cpanel::ServiceManager::new("Cpanel::ServiceManager", "service", "tailwatchd") called at bin/restartsrv_base.pl line 62
        main::__ANON__() called at /usr/local/cpanel/3rdparty/perl/524/lib64/perl5/cpanel_lib/Try/Tiny.pm line 99
        eval {...} called at /usr/local/cpanel/3rdparty/perl/524/lib64/perl5/cpanel_lib/Try/Tiny.pm line 90
        Try::Tiny::try(CODE(0xc7d078), Try::Tiny::Catch=REF(0xbe3310)) called at bin/restartsrv_base.pl line 109
*** Done restart_tailwatchd_if_updated ***
################
I'm sorry for the long explanation.
Can someone explain how to fix the abowe issue?

Thanks.
 
Last edited by a moderator:
cPanelMichael

cPanelMichael

Administrator
Staff member
Apr 11, 2011
47,880
2,258
463
Hello,

Could you open a support ticket using the link in my signature so we can take a closer look?

Thank you.
 
A

ally1

Member
Nov 13, 2014
12
4
3
cPanel Access Level
Root Administrator
Hello,

Just for those who encountered the same problem, the issue was with eximstats database that had grown too large (about 13 Gb), and tailwatchd was timing out when attempting to start up. The eximstats_db.sqlite3 files was re-initialized and now tailwatchd is up and working.

Thanks cPanel support for the help!

Best Regards,
Alexander Lymar
 
  • Like
Reactions: did-vmonroig
D

did-vmonroig

Well-Known Member
Feb 6, 2012
69
5
58
cPanel Access Level
Root Administrator
ally1 said:
Just for those who encountered the same problem, the issue was with eximstats database that had grown too large (about 13 Gb), and tailwatchd was timing out when attempting to start up. The eximstats_db.sqlite3 files was re-initialized and now tailwatchd is up and working.
Click to expand...
Hello,

Thanks for your report. We had a spam event and same problem started because that database has become huge. Time for cleaning!

Regards,
 
You must log in or register to reply here.
Thread starter Similar threads Forum Replies Date
Z The service “tailwatchd” appears to be down. Operating Systems 1
Samet Chan The service “tailwatchd” appears to be down. Operating Systems 10
N The service “tailwatchd” appears to be down Operating Systems 3
E The service “tailwatchd” appears to be down. Operating Systems 7
M The service “tailwatchd” appears to be down. Operating Systems 4
Similar threads
The service “tailwatchd” appears to be down.
The service “tailwatchd” appears to be down.
The service “tailwatchd” appears to be down
The service “tailwatchd” appears to be down.
The service “tailwatchd” appears to be down.
Top Bottom