In Progress [COBRA-13435] AutoSSL not renewing domain certificate when some hosts fail DCV

swbrains

Well-Known Member
Sep 13, 2006
262
37
178
I have created a private cPanel ticket for this issue but wanted to post it here so other customers with similar issues could track its progress in case they experience something similar:

I have an account with a domain that the customer has registered and configured at the registrar with the A record pointing to my server. The nameservers point elsewhere because they want to manage their mail externally. This worked fine taking visitors to the site hosted on my server securely with the SSL cert that was installed.

On 1/3/2022, the SSL cert was expected to renew via AutoSSL. The AutoSSL log for this account from 1/3/2022 is included below. My primary company hosting domain is changed to "example.com" and the customer's domain/subdomain is changed to "customer". The primary customer domain for the site would be "customer.com" and the issue occurs with "mail.customer.com" as described in this log:

Code:
Verifying “cPanel (powered by Sectigo)”’s authorization on 12 domains via DNS CAA records …
11:18:59 PM “customer.example.com” is managed.
“www.customer.example.com” is managed.
“mail.customer.example.com” is managed.
“cpanel.customer.example.com” is managed.
“webdisk.customer.example.com” is managed.
“webmail.customer.example.com” is managed.
“cpcontacts.customer.example.com” is managed.
“mail.customer.com” is managed.
“www.customer.com” is managed.
“customer.com” is managed.
“cpcalendars.customer.example.com” is managed.
“autodiscover.customer.example.com” is managed.
All of this user’s 12 domains are managed.
CA authorized: “customer.example.com”
CA authorized: “www.customer.example.com”
CA authorized: “mail.customer.example.com”
CA authorized: “cpanel.customer.example.com”
CA authorized: “webdisk.customer.example.com”
CA authorized: “webmail.customer.example.com”
CA authorized: “cpcontacts.customer.example.com”
CA authorized: “cpcalendars.customer.example.com”
CA authorized: “autodiscover.customer.example.com”
CA authorized: “customer.com”
CA authorized: “www.customer.com”
CA authorized: “mail.customer.com”
“cPanel (powered by Sectigo)” is authorized to issue certificates for 12 of this user’s 12 domains.
11:18:59 PM Performing HTTP DCV (Domain Control Validation) on 12 domains …
11:18:59 PM Local HTTP DCV OK: customer.com
Local HTTP DCV OK: www.customer.com
WARN Local HTTP DCV error (mail.customer.com): The system queried for a temporary file at “http://mail.customer.com/.well-known/pki-validation/60AB477AA88A4AD2AF59E3F3474255E6.txt”, but the web server responded with the following error: 404 (Not Found). A DNS (Domain Name System) or web server misconfiguration may exist. The domain “mail.customer.com” resolved to an IP address “207.204.50.27” that does not exist on this server.
Local HTTP DCV OK: customer.example.com
Local HTTP DCV OK: www.customer.example.com
Local HTTP DCV OK: mail.customer.example.com
Local HTTP DCV OK: cpanel.customer.example.com
Local HTTP DCV OK: webdisk.customer.example.com
Local HTTP DCV OK: webmail.customer.example.com
Local HTTP DCV OK: cpcontacts.customer.example.com
Local HTTP DCV OK: cpcalendars.customer.example.com
Local HTTP DCV OK: autodiscover.customer.example.com
11:18:59 PM Verifying local authority for 1 domain …
11:18:59 PM No local authority: “mail.customer.com”
It seems that AutoSSL/Sectigo couldn't find the "mail.customer.com" host, probably because the user didn't set it up at the registrar or wherever their DNS points to.
The email notification for this failure sent on 12/30/2021 had this message:

"AutoSSL would normally renew this certificate now, but 1 of the website’s secured domains just failed DCV. To provide you with more time to resolve this problem, AutoSSL will defer the renewal until Jan 1, 2022 at 12:00:00 AM UTC. After that time, AutoSSL will request a replacement certificate that excludes any domains that fail DCV. At the time of this notice, the certificate will expire in 3 days, 19 hours, 40 minutes, and 19 seconds."

However, it did not renew the other domains on the certificate, since going to "customer.com" gave an expired SSL message.

I had this happen to another customer about a month ago but it had never happened before over several years of using AutoSSL and customer domains that only have an A record that points to our server. I had to manually run another check for that domain and it corrected itself, but it should do it before the current cert expires so there is no downtime for the customer's site.

It seems something has changed with either AutoSSL or with Sectigo recently. Based on the email message, it would seem that AutoSSL expects to renew the valid parts of the cert regardless of the failure, but ultimately does not do so before the cert expires.

As I did last time, I ran an AutoSSL check manually on just the "customer" user account and it did issue the cert without the missing "mail.customer.com" domain, as it should. The problem is that it didn't do this automatically prior to the old certificate's expiration, leaving the customer without access to their site until my manual intervention.

I'm assuming this manual intervention should not be necessary, particularly provided the email message implying that AutoSSL would provide "a replacement certificate that excludes any domains that fail DCV."

cPanel ticket: https://support.cpanel.net/hc/en-us/requests/94401480
 
Last edited:

swbrains

Well-Known Member
Sep 13, 2006
262
37
178
After cPanel investigation, it appears that Sectigo was responding in the logs that "The “cPanel (powered by Sectigo)” provider cannot currently accept incoming requests. The system will try again later."

But unfortunately, it wasn't "trying again later" soon enough to get a new cert in place prior to the old one expiring.

According to this page:
Sectigo not accepting incoming requests
The article says the cause is either:
1) This may be caused rate limiting at Sectigo. After a significant number of SSL requests from cPanel customers have been submitted to the AutoSSL provider Sectigo, additional requests will be rate-limited. When the rate-limit is in place this message will appear when attempting to run AutoSSL.
2) This may also be due to issues or maintenance at Sectigo.

However, on the WHM AutoSSL provider selection page, it has a chart comparing Sectigo to Let's Encrypt. The Rate Limit column shows "unlimited" for Sectigo and "50" for Let's Encrypt. If Sectigo has an unlimited rate limit, why would the article say that this is a possible cause of not issuing certs?

If there really is no rate limiting, then #2 is the only possible cause of this issue, which then begs the question, why is this happening to so many people more recently (based on these forums). It has happened to two of my customers in the last 5 weeks where AutoSSL failed to renew the cert prior to expiration and the customer's site was issuing SSL errors when they tried to access it.

AutoSSL/Sectigo needs to work in a way where if the request fails, AutoSSL will have enough time before cert expiration to try again, and will actually try again prior to expiration. This failed to occur twice in the past 5 weeks for my customers and apparently for other users here as well. The strange part is that this only started recently. I have used AutoSSL for a few years with Sectigo and never had issues of certs expiring due to these types of errors.

In both cases I experienced, after the customer notified me that their site wasn't working, I was able to run the "AutoSSL Check" on just their account and it did finally issue the cert. But this occurred only with manual intervention after the cert had expired and the customer noticed their site was not accessible via https.
 

Legin76

Well-Known Member
Dec 11, 2007
178
4
68
We are also getting this issue.. When we run a check on the domains waiting we get a message like the one below on all of them. We currently have 4 accounts that have expired ssl certificates that are stuck in the queue.

The provider “cPanel (powered by Sectigo)”’s AutoSSL queue already contains a certificate request for “accountname”’s website “domain.com”. The request’s start time is Jan 4, 2022, 12:35:06 AM UTC.

Two of them use cloudflare but the other two do not.
 
Last edited:

Legin76

Well-Known Member
Dec 11, 2007
178
4
68
Update to this but cloudflare appears to be unrelated. We currently have 45 domains and sub domains with expired certificates all waiting in Sectigo's queue.
 
  • Like
Reactions: cPanelAnthony

quietFinn

Well-Known Member
Feb 4, 2006
1,648
323
438
Finland
cPanel Access Level
Root Administrator
Update to this but cloudflare appears to be unrelated. We currently have 45 domains and sub domains with expired certificates all waiting in Sectigo's queue.
You see those in WHM -> SSL/TLS -> Manage AutoSSL -> Pending Queue ?

We had that kind of problem in one server, and after waiting about 1-2 hours I switched to Let's Encrypt, removed the expired certificates and Let's Encrypt installed them in a few seconds. :rolleyes:
 
  • Like
Reactions: cPanelAnthony

Legin76

Well-Known Member
Dec 11, 2007
178
4
68
You see those in WHM -> SSL/TLS -> Manage AutoSSL -> Pending Queue ?

We had that kind of problem in one server, and after waiting about 1-2 hours I switched to Let's Encrypt, removed the expired certificates and Let's Encrypt installed them in a few seconds. :rolleyes:
Yes.. This is now up to 86 :(
 

Legin76

Well-Known Member
Dec 11, 2007
178
4
68
As a temporary fix I changed from Sectigo to Lets Encrypt, which works for the moment but I suspect it might not for long. It at least gives us time to migrate the sites to a newer server.
 
  • Like
Reactions: AdrianP

DennisMidjord

Well-Known Member
Sep 27, 2016
305
59
78
Denmark
cPanel Access Level
Root Administrator
cPanel definitely changed something in a recent update.
A lot of clients has been contacting us the last few weeks because they received emails about some domains failing DCV. These domains have always failed DCV but the certificate was still issued.
This is no longer the case. Even though AutoSSL says the certificate will be renewed (while excluding the domains that failed DCV), renewal doesn't work.

This is a huge issue since we have a lot of clients who's using an external DNS provider. They haven't created the cpcalendar, webmail, cpcontacts and autodiscover subdomains. cPanel can no longer renew these certificates - we have to exclude the subdomains from AutoSSL before the certificate can be renewed.

The past month, I've basically just been excluding domains from AutoSSL and telling our clients about the issue all day long. What a headache!
 
  • Like
Reactions: swbrains

cPanelAnthony

Administrator
Staff member
Oct 18, 2021
1,046
111
118
Houston, TX
cPanel Access Level
Root Administrator
cPanel definitely changed something in a recent update.
A lot of clients has been contacting us the last few weeks because they received emails about some domains failing DCV. These domains have always failed DCV but the certificate was still issued.
This is no longer the case. Even though AutoSSL says the certificate will be renewed (while excluding the domains that failed DCV), renewal doesn't work.

This is a huge issue since we have a lot of clients who's using an external DNS provider. They haven't created the cpcalendar, webmail, cpcontacts and autodiscover subdomains. cPanel can no longer renew these certificates - we have to exclude the subdomains from AutoSSL before the certificate can be renewed.

The past month, I've basically just been excluding domains from AutoSSL and telling our clients about the issue all day long. What a headache!
Hello! Which specific DCV errors were you seeing in the logs?
 

cPanelAnthony

Administrator
Staff member
Oct 18, 2021
1,046
111
118
Houston, TX
cPanel Access Level
Root Administrator
Either that the DNS record doesn't exist or that it's pointing to an external IP.
We have clients that have had made no changes to DNS for years, and renewal of SSL certificates has always worked. Just not any more.
Thank you for the clarification. As there have been multiple AutoSSL issues, would it be possible to open a ticket with the link in my signature so we can investigate?
 

4u123

Well-Known Member
PartnerNOC
Jan 2, 2006
943
28
178
This has become a serious issue for us now. Happens every day. It's not just domains that have their DNS hosted elsewhere - it's all of them. Every day a client will contact us to say their certificate didn't get renewed automatically. We have found that running the following does seem to provide some improvement but not always...

/usr/local/cpanel/bin/autossl_check --all

Often we will see the error mentioned below...

"The “cPanel (powered by Sectigo)” provider cannot currently accept incoming requests. The system will try again later."

I think there are various other threads on the go about these issues.
 

swbrains

Well-Known Member
Sep 13, 2006
262
37
178
Not a solution, but it may help in the meantime to change the server's AutoSSL cron file (/etc/cron.d/cpanel_autossl) to run the AutoSSL check more frequently. cPanel support suggested this, and although I still see a number of "cannot accept incoming requests" messages in the daily logs, I'm not aware of any cases of SSL certs expiring before renewal on our server since modifying AutoSSL to run more frequently. I set it to run every 6 hours. I guess this quadruples the chances that the AutoSSL checking process will issue the request during a time when Sectigo's server actually can accept requests.

The other workaround is to ensure AutoSSL is configured to NOT replace existing valid certs and switch to Let's Encrypt as your SSL provider, which seems to be more reliable but has the downside of rate limits. If those are an issue for you, you can appeal to LE to increase your specific rate limit. I did request an increase last week but have yet to hear back from them.

The best solution, of course, would be if Sectigo's service could simply be corrected to work as reliably as it did in the past. They have a number of recent reports of certificate issuance delays/issues on their site (Sectigo) and each one is marked as resolved after clearing the backlog. But the same issue appears several times in the past few months, so it seems that it is not truly resolved.
 

Ali Poonawala

Registered
Mar 20, 2016
2
1
50
USA
cPanel Access Level
Root Administrator
I was able to resolve the problem for now by moving to LetsEncrypt as our AutoSSL provider.
Instructions:
Step 1 :
Install Lets Encrypt cpanel plugin
Run the following command in ssh : /usr/local/cpanel/scripts/install_lets_encrypt_autossl_provider

Step 2:
In WHM -> Manage AutoSSL -> Providers tab -> Select Lets Encrypt and click save.

Step 3:
Run AutoSSL For All Users (Alternatively you may just do it for the domain where ssl did not renew)
 
  • Like
Reactions: cPRex

swbrains

Well-Known Member
Sep 13, 2006
262
37
178
I was able to resolve the problem for now by moving to LetsEncrypt as our AutoSSL provider.
Instructions:
A tip for those that do this: Ensure that the AutoSSL option to replace existing valid certs is OFF before running a check-all operation under Lets Encrypt, particularly if you have a lot of accounts. I did this and quickly ran into the Lets Encrypt rate limit. Had to wait a week to try again after turning off the replacement setting. :(
 
Last edited:
  • Like
Reactions: Metro2