Yum hangs following upcp Python update

bloatedstoat

Well-Known Member
Jun 14, 2012
183
24
68
Victoria, Australia
cPanel Access Level
Root Administrator
Hello,

Yesterday yum installed some Python packages during the nightly upcp run.
It could be a red herring and the issue lies with the CLN but clearly conjecture.
I am unable to successfully complete any yum transactions on the cli, any attempt hangs.

Here's what I know. UPCP log reports updates to Python.

49 [2021-04-24 04:21:06 +1000] - Processing command `/usr/local/cpanel/scripts/rpmup`
50 [2021-04-24 04:23:20 +1000] [/usr/local/cpanel/scripts/rpmup] checkyum version 22.3 (excludes: bind-chroot kernel)
51 [2021-04-24 04:23:20 +1000] [/usr/local/cpanel/scripts/rpmup] Loaded plugins: fastestmirror, rhnplugin, universal-hooks
52 [2021-04-24 04:23:20 +1000] [/usr/local/cpanel/scripts/rpmup] Setting up Update Process
53 [2021-04-24 04:23:20 +1000] [/usr/local/cpanel/scripts/rpmup] Loading mirror speeds from cached hostfile
54 [2021-04-24 04:23:20 +1000] [/usr/local/cpanel/scripts/rpmup] * EA4: 27.50.85.8
55 [2021-04-24 04:23:20 +1000] [/usr/local/cpanel/scripts/rpmup] * cpanel-addons-production-feed: 27.50.85.8
56 [2021-04-24 04:23:20 +1000] [/usr/local/cpanel/scripts/rpmup] * cpanel-plugins: 27.50.85.8
57 [2021-04-24 04:23:20 +1000] [/usr/local/cpanel/scripts/rpmup] * cloudlinux-x86_64-server-6: cl-mirror.syd.ventraip.net.au
58 [2021-04-24 04:23:20 +1000] [/usr/local/cpanel/scripts/rpmup] Resolving Dependencies
59 [2021-04-24 04:23:20 +1000] [/usr/local/cpanel/scripts/rpmup] --> Running transaction check
60 [2021-04-24 04:23:20 +1000] [/usr/local/cpanel/scripts/rpmup] ---> Package cloudlinux-release.x86_64 0:6-6.10.2.el6 will be updated
61 [2021-04-24 04:23:20 +1000] [/usr/local/cpanel/scripts/rpmup] ---> Package cloudlinux-release.x86_64 0:6-6.10.3.el6 will be an update
62 [2021-04-24 04:23:20 +1000] [/usr/local/cpanel/scripts/rpmup] ---> Package python.x86_64 0:2.6.6-68.el6_10 will be updated
63 [2021-04-24 04:23:20 +1000] [/usr/local/cpanel/scripts/rpmup] ---> Package python.x86_64 0:2.6.6-69.el6.cloudlinux.els6 will be an update
64 [2021-04-24 04:23:20 +1000] [/usr/local/cpanel/scripts/rpmup] ---> Package python-devel.x86_64 0:2.6.6-68.el6_10 will be updated
65 [2021-04-24 04:23:20 +1000] [/usr/local/cpanel/scripts/rpmup] ---> Package python-devel.x86_64 0:2.6.6-69.el6.cloudlinux.els6 will be an update
66 [2021-04-24 04:23:20 +1000] [/usr/local/cpanel/scripts/rpmup] ---> Package python-libs.x86_64 0:2.6.6-68.el6_10 will be updated
67 [2021-04-24 04:23:20 +1000] [/usr/local/cpanel/scripts/rpmup] ---> Package python-libs.x86_64 0:2.6.6-69.el6.cloudlinux.els6 will be an update
68 [2021-04-24 04:23:20 +1000] [/usr/local/cpanel/scripts/rpmup] ---> Package python-tools.x86_64 0:2.6.6-68.el6_10 will be updated
69 [2021-04-24 04:23:20 +1000] [/usr/local/cpanel/scripts/rpmup] ---> Package python-tools.x86_64 0:2.6.6-69.el6.cloudlinux.els6 will be an update
70 [2021-04-24 04:23:20 +1000] [/usr/local/cpanel/scripts/rpmup] ---> Package screen.x86_64 0:4.0.3-19.el6 will be updated
71 [2021-04-24 04:23:20 +1000] [/usr/local/cpanel/scripts/rpmup] ---> Package screen.x86_64 0:4.0.3-20.el6.cloudlinux.els6 will be an update
72 [2021-04-24 04:23:20 +1000] [/usr/local/cpanel/scripts/rpmup] ---> Package tkinter.x86_64 0:2.6.6-68.el6_10 will be updated
73 [2021-04-24 04:23:20 +1000] [/usr/local/cpanel/scripts/rpmup] ---> Package tkinter.x86_64 0:2.6.6-69.el6.cloudlinux.els6 will be an update
74 [2021-04-24 04:23:20 +1000] [/usr/local/cpanel/scripts/rpmup] --> Finished Dependency Resolution
75 [2021-04-24 04:23:20 +1000] [/usr/local/cpanel/scripts/rpmup]
76 [2021-04-24 04:23:20 +1000] [/usr/local/cpanel/scripts/rpmup]
77 [2021-04-24 04:23:20 +1000] [/usr/local/cpanel/scripts/rpmup] Total download size: 7.2 M
78 [2021-04-24 04:23:20 +1000] [/usr/local/cpanel/scripts/rpmup] Downloading Packages:
79 [2021-04-24 04:23:20 +1000] [/usr/local/cpanel/scripts/rpmup] --------------------------------------------------------------------------------
80 [2021-04-24 04:23:20 +1000] [/usr/local/cpanel/scripts/rpmup] Total 6.9 MB/s | 7.2 MB 00:01
81 [2021-04-24 04:23:20 +1000] [/usr/local/cpanel/scripts/rpmup] Running rpm_check_debug
82 [2021-04-24 04:23:20 +1000] [/usr/local/cpanel/scripts/rpmup] Running Transaction Test
83 [2021-04-24 04:23:20 +1000] [/usr/local/cpanel/scripts/rpmup] Transaction Test Succeeded
84 [2021-04-24 04:23:20 +1000] [/usr/local/cpanel/scripts/rpmup] Running Transaction
85 [2021-04-24 04:23:20 +1000] [/usr/local/cpanel/scripts/rpmup]
86 [2021-04-24 04:23:20 +1000] Updating : python-2.6.6-69.el6.cloudlinux.els6.x86_64 1/14
87 [2021-04-24 04:23:20 +1000] [/usr/local/cpanel/scripts/rpmup]
88 [2021-04-24 04:23:20 +1000] Updating : python-libs-2.6.6-69.el6.cloudlinux.els6.x86_64 2/14
89 [2021-04-24 04:23:20 +1000] [/usr/local/cpanel/scripts/rpmup]
90 [2021-04-24 04:23:20 +1000] Updating : tkinter-2.6.6-69.el6.cloudlinux.els6.x86_64 3/14
91 [2021-04-24 04:23:20 +1000] [/usr/local/cpanel/scripts/rpmup]
92 [2021-04-24 04:23:20 +1000] Updating : python-tools-2.6.6-69.el6.cloudlinux.els6.x86_64 4/14
93 [2021-04-24 04:23:20 +1000] [/usr/local/cpanel/scripts/rpmup]
94 [2021-04-24 04:23:20 +1000] Updating : python-devel-2.6.6-69.el6.cloudlinux.els6.x86_64 5/14
95 [2021-04-24 04:23:20 +1000] [/usr/local/cpanel/scripts/rpmup]
96 [2021-04-24 04:23:20 +1000] Updating : cloudlinux-release-6-6.10.3.el6.x86_64 6/14
97 [2021-04-24 04:23:20 +1000] [/usr/local/cpanel/scripts/rpmup]
98 [2021-04-24 04:23:20 +1000] Updating : screen-4.0.3-20.el6.cloudlinux.els6.x86_64 7/14
99 [2021-04-24 04:23:20 +1000] [/usr/local/cpanel/scripts/rpmup]
100 [2021-04-24 04:23:20 +1000] Cleanup : python-devel-2.6.6-68.el6_10.x86_64 8/14
101 [2021-04-24 04:23:20 +1000] [/usr/local/cpanel/scripts/rpmup]
102 [2021-04-24 04:23:20 +1000] Cleanup : python-tools-2.6.6-68.el6_10.x86_64 9/14
103 [2021-04-24 04:23:20 +1000] [/usr/local/cpanel/scripts/rpmup]
104 [2021-04-24 04:23:20 +1000] Cleanup : cloudlinux-release-6-6.10.2.el6.x86_64 10/14
105 [2021-04-24 04:23:20 +1000] [/usr/local/cpanel/scripts/rpmup]
106 [2021-04-24 04:23:20 +1000] Cleanup : tkinter-2.6.6-68.el6_10.x86_64 11/14
107 [2021-04-24 04:23:20 +1000] [/usr/local/cpanel/scripts/rpmup]
108 [2021-04-24 04:23:20 +1000] Cleanup : python-2.6.6-68.el6_10.x86_64 12/14
109 [2021-04-24 04:23:20 +1000] [/usr/local/cpanel/scripts/rpmup]
110 [2021-04-24 04:23:20 +1000] Cleanup : python-libs-2.6.6-68.el6_10.x86_64 13/14
111 [2021-04-24 04:23:20 +1000] [/usr/local/cpanel/scripts/rpmup]
112 [2021-04-24 04:23:20 +1000] Cleanup : screen-4.0.3-19.el6.x86_64 14/14
113 [2021-04-24 04:23:20 +1000] [/usr/local/cpanel/scripts/rpmup]
114 [2021-04-24 04:23:20 +1000] Verifying : screen-4.0.3-20.el6.cloudlinux.els6.x86_64 1/14
115 [2021-04-24 04:23:20 +1000] [/usr/local/cpanel/scripts/rpmup]
116 [2021-04-24 04:23:20 +1000] Verifying : python-libs-2.6.6-69.el6.cloudlinux.els6.x86_64 2/14
117 [2021-04-24 04:23:20 +1000] [/usr/local/cpanel/scripts/rpmup]
118 [2021-04-24 04:23:20 +1000] Verifying : python-2.6.6-69.el6.cloudlinux.els6.x86_64 3/14
119 [2021-04-24 04:23:20 +1000] [/usr/local/cpanel/scripts/rpmup]
120 [2021-04-24 04:23:20 +1000] Verifying : tkinter-2.6.6-69.el6.cloudlinux.els6.x86_64 4/14
121 [2021-04-24 04:23:20 +1000] [/usr/local/cpanel/scripts/rpmup]
122 [2021-04-24 04:23:20 +1000] Verifying : python-devel-2.6.6-69.el6.cloudlinux.els6.x86_64 5/14
123 [2021-04-24 04:23:20 +1000] [/usr/local/cpanel/scripts/rpmup]
124 [2021-04-24 04:23:20 +1000] Verifying : cloudlinux-release-6-6.10.3.el6.x86_64 6/14
125 [2021-04-24 04:23:20 +1000] [/usr/local/cpanel/scripts/rpmup]
126 [2021-04-24 04:23:20 +1000] Verifying : python-tools-2.6.6-69.el6.cloudlinux.els6.x86_64 7/14
127 [2021-04-24 04:23:20 +1000] [/usr/local/cpanel/scripts/rpmup]
128 [2021-04-24 04:23:20 +1000] Verifying : python-libs-2.6.6-68.el6_10.x86_64 8/14
129 [2021-04-24 04:23:20 +1000] [/usr/local/cpanel/scripts/rpmup]
130 [2021-04-24 04:23:20 +1000] Verifying : python-devel-2.6.6-68.el6_10.x86_64 9/14
131 [2021-04-24 04:23:20 +1000] [/usr/local/cpanel/scripts/rpmup]
132 [2021-04-24 04:23:20 +1000] Verifying : cloudlinux-release-6-6.10.2.el6.x86_64 10/14
133 [2021-04-24 04:23:20 +1000] [/usr/local/cpanel/scripts/rpmup]
134 [2021-04-24 04:23:20 +1000] Verifying : screen-4.0.3-19.el6.x86_64 11/14
135 [2021-04-24 04:23:20 +1000] [/usr/local/cpanel/scripts/rpmup]
136 [2021-04-24 04:23:20 +1000] Verifying : python-2.6.6-68.el6_10.x86_64 12/14
137 [2021-04-24 04:23:20 +1000] [/usr/local/cpanel/scripts/rpmup]
138 [2021-04-24 04:23:20 +1000] Verifying : tkinter-2.6.6-68.el6_10.x86_64 13/14
139 [2021-04-24 04:23:20 +1000] [/usr/local/cpanel/scripts/rpmup]
140 [2021-04-24 04:23:20 +1000] Verifying : python-tools-2.6.6-68.el6_10.x86_64 14/14
141 [2021-04-24 04:23:20 +1000] [/usr/local/cpanel/scripts/rpmup]
142 [2021-04-24 04:23:20 +1000] [/usr/local/cpanel/scripts/rpmup] Complete!
143 [2021-04-24 04:23:20 +1000] [/usr/local/cpanel/scripts/rpmup] checkyum version 22.3 (excludes: bind-chroot)
144 [2021-04-24 04:23:20 +1000] - Finished command `/usr/local/cpanel/scripts/rpmup` in 133.717 seconds
145 [2021-04-24 04:23:20 +1000] 26% complete
146 [2021-04-24 04:23:20 +1000] - Finished in 133.717 seconds

The packages updated successfully and the installs are confirmed in the logs.

Packages Updated:
python-tools-2.6.6-69.el6.cloudlinux.els6.x86_64
python-libs-2.6.6-69.el6.cloudlinux.els6.x86_64
python-2.6.6-69.el6.cloudlinux.els6.x86_64
python-devel-2.6.6-69.el6.cloudlinux.els6.x86_64
cloudlinux-release-6-6.10.3.el6.x86_64


Wind forward 24 hours and there are obvious comm's errors as shown in the next upcp run.

[2021-04-25 04:21:06 +1000] - Processing command `/usr/local/cpanel/scripts/find_and_fix_rpm_issues`
40 [2021-04-25 04:21:26 +1000] [/usr/local/cpanel/scripts/find_and_fix_rpm_issues] Checking that the RPM DB is OK...
41 [2021-04-25 04:21:26 +1000] [/usr/local/cpanel/scripts/find_and_fix_rpm_issues] glibc-2.12-1.212.el6_10.3.x86_64
42 [2021-04-25 04:21:26 +1000] [/usr/local/cpanel/scripts/find_and_fix_rpm_issues] OK: RPM DB is responding to queries
43 [2021-04-25 04:21:26 +1000] [/usr/local/cpanel/scripts/find_and_fix_rpm_issues] Testing if rpm_is_working RPM is installed
44 [2021-04-25 04:21:26 +1000] [/usr/local/cpanel/scripts/find_and_fix_rpm_issues] package rpm_is_working is not installed
45 [2021-04-25 04:21:26 +1000] [/usr/local/cpanel/scripts/find_and_fix_rpm_issues] Testing if it's possible to install a simple RPM
46 [2021-04-25 04:21:26 +1000] [/usr/local/cpanel/scripts/find_and_fix_rpm_issues] Preparing... ##################################################
47 [2021-04-25 04:21:26 +1000] [/usr/local/cpanel/scripts/find_and_fix_rpm_issues] rpm_is_working ##################################################
48 [2021-04-25 04:21:26 +1000] 25% complete
49 [2021-04-25 04:21:26 +1000] - Finished in 19.973 seconds
50 [2021-04-25 04:21:26 +1000] Processing: Running rpmup (yum -y update)
51 [2021-04-25 04:21:26 +1000] - Processing command `/usr/local/cpanel/scripts/rpmup`
52 [2021-04-25 04:34:53 +1000] [/usr/local/cpanel/scripts/rpmup] checkyum version 22.3 (excludes: bind-chroot kernel)
53 [2021-04-25 04:34:53 +1000] [/usr/local/cpanel/scripts/rpmup] Loaded plugins: fastestmirror, rhnplugin, universal-hooks
54 [2021-04-25 04:34:53 +1000] [/usr/local/cpanel/scripts/rpmup] An error has occurred:
55 [2021-04-25 04:34:53 +1000] [/usr/local/cpanel/scripts/rpmup] There was an error communicating with CLN.

56 [2021-04-25 04:34:53 +1000] [/usr/local/cpanel/scripts/rpmup] Setting up Update Process
57 [2021-04-25 04:34:53 +1000] [/usr/local/cpanel/scripts/rpmup] Loading mirror speeds from cached hostfile
58 [2021-04-25 04:34:53 +1000] [/usr/local/cpanel/scripts/rpmup] * EA4: 27.50.85.8
59 [2021-04-25 04:34:53 +1000] [/usr/local/cpanel/scripts/rpmup] * cpanel-addons-production-feed: 27.50.85.8
60 [2021-04-25 04:34:53 +1000] [/usr/local/cpanel/scripts/rpmup] * cpanel-plugins: 27.50.85.8
61 [2021-04-25 04:34:53 +1000] [/usr/local/cpanel/scripts/rpmup] No Packages marked for Update
62 [2021-04-25 04:34:53 +1000] [/usr/local/cpanel/scripts/rpmup] checkyum version 22.3 (excludes: bind-chroot)
63 [2021-04-25 04:34:53 +1000] - Finished command `/usr/local/cpanel/scripts/rpmup` in 807.006 seconds


This error was not reported and I only came across it when examining logs following an attempt to check a kernel update.
I have attempted to run several yum related commands and each hang with no progress.
  • yum update kernel
  • yum clean all
  • yum list all
I never get past Loaded plugins: fastestmirror, rhnplugin, universal-hooks in each case.

The python related calls are timing out and can be seen in /var/log/up2date

[Sun Apr 25 05:13:58 2021] up2date Updating package profile
97 [Sun Apr 25 09:12:41 2021] up2date A socket error occurred: Timeout Exception, attempt #1
98 [Sun Apr 25 09:12:41 2021] up2date Error communicating with server. The message was:
99 Connection timed out on readline
100 [Sun Apr 25 09:12:41 2021] up2date
101 Traceback (most recent call last):
102 File "/usr/share/rhn/up2date_client/rhnreg.py", line 212, in prepareGradualRolloutCertsIfNeeded
103 result = s.up2date.getCLNCertificate(systemId)
104 File "/usr/share/rhn/up2date_client/rhnserver.py", line 50, in __call__
105 return rpcServer.doCall(method, *args, **kwargs)
106 File "/usr/share/rhn/up2date_client/rpcServer.py", line 288, in doCall
107 raise up2dateErrors.CommunicationError(e.args[0])
108 up2date_client.up2dateErrors.CommunicationError: Error communicating with server. The message was:
109 Connection timed out on readline

Needs Restarting
/usr/bin/needs-restarting
3746 : /usr/bin/python/usr/sbin/tuned-d-c/etc/tuned.conf

Any ideas on this please.

Thanks!
 
Last edited by a moderator:

quietFinn

Well-Known Member
Feb 4, 2006
1,327
141
193
Finland
cPanel Access Level
Root Administrator