In Progress [CPANEL-27532] /scripts/modsec_vendor update failed

[jndawson]

All of our servers are reporting identical update failures:

The cPanel & WHM update process failed for the following reason:

Maintenance ended; however, it did not exit cleanly (256). The following events were logged: “scripts/modsec_vendor”. Review the update logs to determine why the update failed.

Update log preview:

...
...
[2019-05-27 02:46:00 -0700] E [/usr/local/cpanel/scripts/modsec_vendor] The “/usr/local/cpanel/scripts/modsec_vendor update --auto” command (process 6160) reported error number 1 when it ended.

Running /scripts/modsec_vendor update --auto:

[ [email protected] ~># /usr/local/cpanel/scripts/modsec_vendor update --auto
info [modsec_vendor] Updates are in progress for all of the installed ModSecurity vendors with automatic updates enabled.
info [modsec_vendor] Restored modsec_cpanel_conf_datastore backup
info [modsec_vendor] The vendor “configserver” is already up to date.
info [modsec_vendor] Restored modsec_cpanel_conf_datastore backup
info [modsec_vendor] The vendor “OWASP3” is already up to date.

There don't seem to be any actual problems. Is this a known issue that is getting addressed?

 

[TDP]

I too am having a very similar issue since the 78.0.24 update. I am using the COMODO LiteSpeed rules. Below is the output from the last update log.

The cPanel & WHM update process failed for the following reason:

Maintenance ended; however, it did not exit cleanly (256). The following events were logged: “scripts/modsec_vendor”. Review the update logs to determine why the update failed.

Update log preview:

...
...
[2019-05-27 03:29:11 -0400] E [/usr/local/cpanel/scripts/modsec_vendor] The “/usr/local/cpanel/scripts/modsec_vendor update --auto” command (process 34159) reported error number 1 when it
+ended.

Sometimes when I run /usr/local/cpanel/scripts/modsec_vendor update --auto it fails to retrieve the rules, but after running it a second, or third time it completes without error. This doesn't happen every time though, sometimes running it once manually it completes without error.

My server is running a fully up to date installation of CloudLinux 7.6, and cPanel/WHM 78.0.24.

 

[Metro2]

I found this thread because I just noticed same happened to me this morning (CloudLinux 6.10 , WHM 78.0.24)

[2019-05-28 01:01:34 -0400] E [/usr/local/cpanel/scripts/modsec_vendor] The “/usr/local/cpanel/scripts/modsec_vendor update --auto” command (process 2805294) reported error number 1 when it ended.

 

[cPanelLauren]

Can you tell me which modsecurity vendors you're using? I'd like to see if there's a common thread.

 

[Metro2]

Can you tell me which modsecurity vendors you're using? I'd like to see if there's a common thread.

ConfigServer 1/1
OWASP 22/22

However, the issue has not occurred since cPanel's upcp cron at 1:00am EST Tuesday May 28 , which ran fine 1am May 29 and 1am May 30.

I might be way off-base, but it almost seems to have coincided with the fact that it was while I was still running WHM 78 while WHM 80 was showing as available in the upper-right of WHM but was not automatically updating to 80 for a couple days, around same time some of us were having the yum update problem in this (possibly related / possibly not) other thread - SOLVED - cPanel update to 80.0.10 fails with YUM repo errors

I hope maybe some of that helps either confirm or dispel some suspicions.

 

[jndawson]

Can you tell me which modsecurity vendors you're using? I'd like to see if there's a common thread.

As noted previously:

info [modsec_vendor] The vendor “configserver” is already up to date.

info [modsec_vendor] The vendor “OWASP3” is already up to date.

The error hadn't repeated for several update cycles, but we got the same error on last night's updates to v.80.0.12.

 

[cPanelLauren]

You're both running the 3rd Party ConfigServer Modsecurity vendor, does the issue persist with this vendor removed?

 

[jndawson]

You're both running the 3rd Party ConfigServer Modsecurity vendor, does the issue persist with this vendor removed?

We're running the same setup on all our servers. We aren't getting the update error on everything every time, and it seems random. The mod_sec rules are up-to-date, so the update error seems anomalous.

 

[Metro2]

You're both running the 3rd Party ConfigServer Modsecurity vendor, does the issue persist with this vendor removed?

In my case, I've been running the 1 ConfigServer Modsec rule for years, and only encountered this one - on May 28th at 1:00am EST during nightly auto upcp , when cPanel releaed update from 78 to 80. Never occurred before, hasn't occurred since, and I keep all servers updated to "release" tier nightly.

 

[sparek-3]

Does cPanel manage the OWASP modsecurity ruleset for cPanel? Or is that managed by someone else. The yaml file appears to be hosted on cPanel

http://httpupdate.cpanel.net/modsecurity-rules/meta_OWASP.yaml

This yaml file doesn't appear to have an entry for ModSecurity 2.9.3. ModSecurity was recently updated to 2.9.3. Is this something that needs to be addressed?

 

[cPanelLauren]

It's also possible there have been intermittent connection issues with the servers hosting the rules, unfortunately, this isn't something we'd track.

 

[fuzzylogic]

Good spotting.
This ruleset would not update for servers with Modsecurity 2.9.3, or if you deleted the ruleset it would not reinstall if you tried.
Would only need copy and paste of entry for 2.9.2 then change to 2.9.3.

This ruleset does not change very often so its not urgent, but better to add this sooner rather than later.

 

[sparek-3]

Yea... the fact that this hasn't been spotted is a bit concerning. Unless maybe I'm using the wrong OWASP vendor? Maybe http://httpupdate.cpanel.net/modsecurity-rules/meta_OWASP.yaml is wrong?

But if it is still valid and the fact that it's taken this long to notice... either nobody is using this ruleset or nobody is paying any attention to upcp failure notices.

Kind of one of those SMH moments... but it's Friday.. who cares!

 

[fuzzylogic]

There is an unresolved recent active thread with some admins noticing Modsec ruleset update failures, you may have discovered the reason.
/scripts/modsec_vendor update failed

 

[fuzzylogic]

Possible cause may be no entry in...
http://httpupdate.cpanel.net/modsecurity-rules/meta_OWASP.yaml
for modsecurity 2.9.3
see thread...
New Thread - OWASP cPanel vendor - modsec 2.9.3

 

[Infopro]

Yea... the fact that this hasn't been spotted is a bit concerning. Unless maybe I'm using the wrong OWASP vendor? Maybe http://httpupdate.cpanel.net/modsecurity-rules/meta_OWASP.yaml is wrong?

But if it is still valid and the fact that it's taken this long to notice... either nobody is using this ruleset or nobody is paying any attention to upcp failure notices.

Kind of one of those SMH moments... but it's Friday.. who cares!

Security doesn't care what day it is. This thread from 2017 may be of some use:
OWASP ModSecurity Core Rule Set v3

It has a link to this file:
http://httpupdate.cpanel.net/modsecurity-rules/meta_OWASP3.yaml

 

[sparek-3]

So... I guess if you installed OWASP ... whatever version that is, non-3.0 ... then you are SOL if you don't find that thread?

Why are there two different OWASP sets? Is OWASP non-3.0 not being updated any longer?

 

[fuzzylogic]

OK I see whats happened.
Sparak-3 is referencing the older version of the cPanel curated OWASP ruleset (and its .yaml file)
The newer version OWASP3 has version 2.9.3 of Modsecurity (and newer versions) in its .yaml file.

The difference is...
OWASP completely renumbered all the rules between these ruleset versions.
The changeover is quick and easy, the pain may come when you get false positives from rules that you have DisabledbyID that become active again when they have a new ID.

That was why both versions were available for a period of time.

 

[sparek-3]

Yea, I'd say that's definitely it.

But I suppose my question is, when was the non 3.0 OWASP ruleset deprecated? Was there a notice about this? I know I'm being a bit facetious in my posts - but honestly I may have missed this notice. But I really wasn't aware that the non-3.0 OWASP ruleset was deprecated, or even if it is.

I will admit, we only have a handful of servers that are using the OWASP ruleset, so maybe that's why I overlooked it. I don't really stay that in tuned with it. But it just seems like there's something amiss that the non 3.0 OWASP ruleset is still there... but not getting ModSecurity 2.9.3 support? Maybe the non-3.0 OWASP ruleset should be taken down? Since... apparently ModSecurity 2.9.3 is THE version of modsecurity now.

 

[kamaok]

Hello guys!
Could you please help me with the same issue regarding update owasp?

2019-06-07 23:40:11 +0100]    - Processing command `/usr/local/cpanel/scripts/modsec_vendor update --auto`

[2019-06-07 23:40:12 +0100]      [/usr/local/cpanel/scripts/modsec_vendor] The system failed to update the vendor from the URL “[URL]http://httpupdate.cpanel.net/modsecurity-rules/meta_OWASP.yaml”[/URL]: The vendor metadata does not contain an entry for your version of ModSecurity, “2.9.3”. The only versions of ModSecurity this rule set supports are “2.8.0”, “2.9.0”, and “2.9.2”.

[2019-06-07 23:40:12 +0100] E    [/usr/local/cpanel/scripts/modsec_vendor] The “/usr/local/cpanel/scripts/modsec_vendor update --auto” command (process 30132) reported error number 1 when it ended.

What should I do to manage with it?
Is it enough only to disable/delete owasp and install and enable owasp3?

[B]# /scripts/modsec_vendor list[/B]
[OWASP3] OWASP ModSecurity Core Rule Set V3.0 (not installed)
 cpanel_provided   1
     description   SpiderLabs OWASP V3 curated ModSecurity rule set
       installed   0
  installed_from   [URL]http://httpupdate.cpanel.net/modsecurity-rules/meta_OWASP3.yaml[/URL]
            name   OWASP ModSecurity Core Rule Set V3.0
       vendor_id   OWASP3
      vendor_url   [URL='https://go.cpanel.net/modsecurityowasp']OWASP ModSecurity CRS - cPanel Knowledge Base - cPanel Documentation[/URL]


[OWASP] OWASP ModSecurity Core Rule Set
         configs   (22)
 cpanel_provided   1
     description   SpiderLabs OWASP curated ModSecurity rule set
         enabled   1
          in_use   22
       inst_dist   OWASP_1501094486
       installed   1
  installed_from   [URL]http://httpupdate.cpanel.net/modsecurity-rules/meta_OWASP.yaml[/URL]
            name   OWASP ModSecurity Core Rule Set
            path   /etc/apache2/conf.d/modsec_vendor_configs/OWASP
      report_url   [URL]https://www.modsecurity.org/rule_issue_report/cPanel/report/new[/URL]
supported_versions   (3)
          update   1
       vendor_id   OWASP
      vendor_url

OWASP ModSecurity CRS - cPanel Knowledge Base - cPanel Documentation

Thanks in advance!