SOLVED CPANEL-39815 - Not receiving security advisor notifications

cPSloaneB

Developer I
Staff member
Oct 2, 2012
18
5
128
cPanel Access Level
Root Administrator
@WorkinOnIt, as the root user, if you rename the /var/cpanel/security_advisor_history.json file to something else and then run /scripts/check_security_advice_changes --notify once again, does it re-issue all, some, or none of the missing notifications you are expecting?
 

WorkinOnIt

Well-Known Member
Aug 3, 2016
288
52
78
UK
cPanel Access Level
Root Administrator
@cPRex

Thank you for your integrity. I have replied to the ticket to see what they say about a partial refund - although to be honest, the money is not really the issue (although when refunds are involved it does sharpen the focus somewhat, so that might help ;-)

Did you hear anything from your colleagues following the meeting?

I've just discovered another kernel update is ready in the manual security scan (of course, once again, no notification received !)
 

WorkinOnIt

Well-Known Member
Aug 3, 2016
288
52
78
UK
cPanel Access Level
Root Administrator
@WorkinOnIt, as the root user, if you rename the /var/cpanel/security_advisor_history.json file to something else and then run /scripts/check_security_advice_changes --notify once again, does it re-issue all, some, or none of the missing notifications you are expecting?
Thank you @cPSloaneB - yes, that actually worked! I also received the email notification o_O I updated on four servers and they all responded the same.

Of course, it remains to be seen if this will fix future auto-generated notices - I guess will have to wait and see if I still receive the email the next time there are new security advisor notifications ? Do you think this is now fixed - or does this need further steps?

I have no idea why renaming that file fixed it - would you care to explain further?
 
Last edited:

cPRex

Jurassic Moderator
Staff member
Oct 19, 2014
12,481
1,966
363
cPanel Access Level
Root Administrator
Deleting the history file and forcing the run to happen again forces all notifications to be sent again.

I did talk with the development team about this earlier this week, and there is work actively happening on the issue now.
 

cPSloaneB

Developer I
Staff member
Oct 2, 2012
18
5
128
cPanel Access Level
Root Administrator
Of course, it remains to be seen if this will fix future auto-generated notices - I guess will have to wait and see if I still receive the email the next time there are new security advisor notifications ? Do you think this is now fixed - or does this need further steps?

I have no idea why renaming that file fixed it - would you care to explain further?
This by itself will not fix future notifications, unless the file is removed again continually. The Security Advisor currently remembers the internal names of each advice item, a hash of the text of the advice, and the severity of the item in that file, and it carries over this information across runs of the script. Unless the severity of the item was increased since the script last saw the item, it will not issue the same notification twice. Removing the history file makes the script forget about all previous notifications, which may work for you but would be too noisy for other users. (In principle, removing just the information pertaining to the notification you expect to be re-issued will also work.)

At this time, I have submitted a fix which changes this behavior so that the history file only remembers the notices which appeared the last time the script ran. This way, it should notify whenever the state of the item degrades since the last notification; if it stays the same, improves, or disappears, then this should still not trigger a notification, which I am hoping is not too noisy of a policy. Furthermore, it doesn't require changing the data format of the history file.

This change is being targeted for a 104 release. A backport to 102 may be possible, but we would likely want to see whether we get negative feedback for this change in 104, which might make necessary further changes to the kind of data recorded in the history file, so that the Security Advisor could apply more complicated hysteresis to the policy of when to issue notifications.
 
Last edited:

WorkinOnIt

Well-Known Member
Aug 3, 2016
288
52
78
UK
cPanel Access Level
Root Administrator
At this time, I have submitted a fix which changes this behavior so that the history file only remembers the notices which appeared the last time the script ran. This way, it should notify whenever the state of the item degrades since the last notification; if it stays the same, improves, or disappears, then this should still not trigger a notification, which I am hoping is not too noisy of a policy. Furthermore, it doesn't require changing the data format of the history file.

This change is being targeted for a 104 release. A backport to 102 may be possible, but we would likely want to see whether we get negative feedback for this change in 104, which might make necessary further changes to the kind of data recorded in the history file, so that the Security Advisor could apply more complicated hysteresis to the policy of when to issue notifications.
Good one thanks. I am rather surprised that I am the only person who seems to have noticed / reported this matter..... I guess I must be old school !

I think another useful way to handle this is some kind of dashboard notification / flashing icon that says "hey something needs attention!" There is already a notification menu icon in the new layout at top right - I would suggest showing new security state change notices in there would be sensible?
 
Last edited:

WorkinOnIt

Well-Known Member
Aug 3, 2016
288
52
78
UK
cPanel Access Level
Root Administrator
Hi there @cPRex and @cPSloaneB

Today I ran a manual check in the WHM security advisor and it came up with :

Detected 1 service that is running outdated executables: httpd.service
You must take one of the following actions to ensure the system is up-to-date:
  • Restart the listed service using “systemctl restart httpd.service”; then click “Scan Again” to check non-service processes.
  • Reboot the server.
But sadly - no email.... this has not been sent to me as an email notification... In previous emails that I used to receive years ago, this type of notification was indeed sent under the subject line of: "New Security Advisor notifications with High importance" - but I did not receive this - so what now....

[edit] Further more, once I had rebooted the server and re-run the security advisor, it now says a kernel update is available - but once again - no email.....

I am in WHM version 104.0.7
 
Last edited:

cPSloaneB

Developer I
Staff member
Oct 2, 2012
18
5
128
cPanel Access Level
Root Administrator
Today I ran a manual check in the WHM security advisor and it came up with :

Detected 1 service that is running outdated executables: httpd.service
You must take one of the following actions to ensure the system is up-to-date:
  • Restart the listed service using “systemctl restart httpd.service”; then click “Scan Again” to check non-service processes.
  • Reboot the server.
But sadly - no email.... this has not been sent to me as an email notification... In previous emails that I used to receive years ago, this type of notification was indeed sent under the subject line of: "New Security Advisor notifications with High importance" - but I did not receive this - so what now....

[edit] Further more, once I had rebooted the server and re-run the security advisor, it now says a kernel update is available - but once again - no email.....

I am in WHM version 104.0.7
There are still several possible reasons why you may not have received a notification.

As one example, note that the /var/cpanel/security_advisor_history.json file is not updated by the Security Advisor itself but rather by the /scripts/check_security_advice_changes script run during the maintenance phase of the nightly cPanel update. Furthermore, the new behavior only makes an advice item eligible for a new notification if that script notices that the condition went away: if the file says that it saw an item last time, but it's not seeing it this time, it removes it from the file, as if the previous notification never happened. Thus, it's possible that Apache was restarted, but an update to the EasyApache packages returned the issue of Apache needing a restart just prior to the next running of the script. In this case, it would not have noticed a change, so that item would still not be eligible for a new notification.

This is not to say that this is definitely what is happening. We would have to know the specifics from your server. It may be time for another support ticket.

(Something I'm considering as a further improvement is to have /scripts/check_security_advice_changes keep a copy of the previous history file before it writes the new one. Comparing these would be an easier way to know whether the script saw a change (and genuinely failed to notify) or not.)
 
  • Like
Reactions: cPRex

WorkinOnIt

Well-Known Member
Aug 3, 2016
288
52
78
UK
cPanel Access Level
Root Administrator
@cPSloaneB @cPRex

Hi both

I am still not receiving the update email notifications at all. I did a cPanel Security Advisor manual check on one of my 5 servers. It showed the
The system kernel is at version...... message - basically - never received the notification..... and nothing for the other servers either.

I have re-opened the previous ticket.

Request #94427824
 
  • Like
Reactions: cPRex

cPRex

Jurassic Moderator
Staff member
Oct 19, 2014
12,481
1,966
363
cPanel Access Level
Root Administrator
I checked the ticket this morning, and it looks like we recommended renaming or moving the /var/cpanel/security_advisor_history.json file to see if this allows future notifications to be sent. Have you tried doing that yet?
 

cPSloaneB

Developer I
Staff member
Oct 2, 2012
18
5
128
cPanel Access Level
Root Administrator
I just looked at one of the the servers attached to your ticket. What I see makes me believe that the Security Advisor is for certain handing its request for a contact event off to the iContact system, as the messages being printed to the error log are from the iContact template belonging to the "change in Security Advisor state" notification. However, if iContact is generating an error, it is not being recorded anywhere. I'm going to have someone investigate further with this in mind.
 
  • Like
Reactions: cPRex

WorkinOnIt

Well-Known Member
Aug 3, 2016
288
52
78
UK
cPanel Access Level
Root Administrator
I just looked at one of the the servers attached to your ticket... I'm going to have someone investigate further with this in mind.
Hello @cPSloaneB

and @cPRex

I have today opened a further ticket: 94487970

I am still not getting any notifications on my 5 servers. I did a test "manual run" on one of the servers and all the notifications are there - just not getting anything on the emails... This is getting a bit tedious, but I do appreciate your assistance. I am surprised I am the only one that's brought it up! I assume I am not the only one affected !!

Could you change the status of this thread to something other than solved? It's a bit inaccurate.

Thank you
 

cPRex

Jurassic Moderator
Staff member
Oct 19, 2014
12,481
1,966
363
cPanel Access Level
Root Administrator
The original issue we thought this may be, CPANEL-39815, has been solved in version 104. However, there is likely something else going on here, and we'll get that tracked down through the ticket. I'm following along with that now as well.
 

WorkinOnIt

Well-Known Member
Aug 3, 2016
288
52
78
UK
cPanel Access Level
Root Administrator
As I have been saying all along, cPanel notifications are broken - and this has been the case for almost a year.

It is frustrating to have to raise the matter several times, then be told "it's solved" when it clearly isn't.