Cpanel/WHM not starting, exim defunct...

[adivity]

Hi, im having a few problems with my server today. This morning I attempted to do a cpanel upgrade to the stable version and something seemed to go wrong with the upgrade. WHM and Cpanel were giving off internal server errors so I decided to do a quick restart of the server.

When it came back up, CPanel and WHM were not working at all, and i wasnt even getting internal server errors. Http, FTP, ssh, etc all working.

Exim is having problems as well. When i restart it, it comes up just fine except when users try to check their mail, then the process goes defunct. I tried the following in order:

/scripts/upcp --force
/scripts/eximup --force

/scripts/fixcommonproblems
/scripts/mailperm
/scripts/updateuserdomains
/scripts/restartsrv exim


Everytime someone tries to check their mail, it ends up like this:

702 mailnull 8 0 0 0 0 Z 0.0 0.0 0:00 0 exim <defunct>

Anyone have any ideas for either problem? Thanks!

[designeru]

I'm not sure what your server has... a

# tail -n 30 /var/log/exim_mainlog

could help. But, you can try this:
- update perl to the latest version found on http://layer1.cpanel.net
- then run:
# perl -MCPAN -e shell
> force install DBI
> force install DBD::mysql
- then run /scripts/upcp --force
- then run /scripts/eximup --force
- then check if it's working.

The last 20-30 lines from /usr/local/cpanel/log/error_log could help.
Also, i would scan your server with an antivirus firewall (f-prot i think it's free for linux servers) and some anti-hackers tools (rkhunter).

[adivity]

Well what seems to be happening now is a problem with the email seems to not be allowing cpanel to start:

root@mercury [/etc/init.d]# ./cpanel start
Starting eximstats:                                        [  OK  ]
Starting cPanel services:                                  [  OK  ]
Starting cPanel Log services:                              [  OK  ]
Starting pop3 services:                                    [  OK  ]
Starting cPanel Chat services:
Starting Melange Chat services:
Starting cPanel ssl services: 2006.10.07 14:36:09 LOG3[14472:16384]: error stack: 140B3009 : error:140B3009:SSL routines:SSL_CTX_use_RSAPrivateKey_file:PEM lib
2006.10.07 14:36:09 LOG3[14472:16384]: SSL_CTX_use_RSAPrivateKey_file: 906D06C: error:0906D06C:PEM routines:PEM_read_bio:no start line
                                                           [FAILED]
Starting mailman services: Starting Mailman's master qrunner.

And as it stands right now, email is running but it returns no new mail, while it will allow for sending correctly.

I understand its probably an ssl cert issue, I just dont know what to do about it...any ideas?

[designeru]

As i can see... CPanel is not starting... the email service shoud be start with this command:

# /scripts/restartsrv_exim

After you run the exim restart command, please provide the last 20-30 lines from /var/log/exim_mainlog.

For resolving the WHM issue, please run:

# /scripts/upcp --force
# /scripts/restartsrv_cpanel

... then let us know the output.

[adivity]

Hey, i did as you said, and here are a few lines from the bottom of the exim_mainlog are as follows:

2006-10-07 15:52:51 1GVVgM-0006AU-Vz Message is frozen
2006-10-07 15:52:51 1GVgfv-0004BN-KB Message is frozen
2006-10-07 15:52:51 1GVzCU-000519-Jr Message is frozen
2006-10-07 15:52:51 1GVYwy-00008r-1m Message is frozen
2006-10-07 15:52:51 1GVYwx-00008l-RT Message is frozen
2006-10-07 15:52:51 1GW2Tg-0004c4-9a Message is frozen
2006-10-07 15:52:51 1GVcvk-0003Z5-5u == area51@fusionlangaming.com <AREA51@FUSIONLANGAMING.COM> R=virtual_user_spam T=virtual_userdelivery_spam defer (-52): Retry time not yet reached
2006-10-07 15:52:51 1GVfBk-0000Xy-3g Message is frozen
2006-10-07 15:52:51 1GVW01-00077P-Ud Message is frozen
2006-10-07 15:52:51 1GW5k7-0002yM-JA == silly@fusionlangaming.com R=virtual_user_spam T=virtual_userdelivery_spam defer (-52): Retry time not yet reached
2006-10-07 15:52:51 1GVztN-0006nm-T4 == peterson63@mantramail.com <Peterson63@mantramail.com> R=lookuphost T=remote_smtp defer (-53): retry time not reached for any host
2006-10-07 15:52:51 1GVYwy-000091-SL Message is frozen
2006-10-07 15:52:51 1GW7Fo-0005ox-Hj Message is frozen
2006-10-07 15:52:51 1GVcyI-0003ea-Cw == silly@fusionlangaming.com R=virtual_user_spam T=virtual_userdelivery_spam defer (-52): Retry time not yet reached
2006-10-07 15:52:51 1GVcyY-0003fb-CC Message is frozen
2006-10-07 15:52:51 1GW90u-00016B-0b Message is frozen
2006-10-07 15:52:51 1GVd5O-0003uD-P2 Message is frozen
2006-10-07 15:52:56 1GWL2O-0006ng-C8 <= ylrlmtptj@cmjit.com H=([209.136.189.26]) [209.136.189.26] P=esmtp S=3394 id=0A799E8706B3D03.E01E2D1C25@cmjit.com

Now, when i run /scripts/upcp --force, everything seems to run smoothly until I get to this:

Waiting to bind to port 995 (Failed to open Private Keyerror:0906D06C:PEM routines:PEM_read_bio:no start line)....
Waiting to bind to port 995 (Failed to open Private Keyerror:140B0009:SSL routines:SSL_CTX_use_PrivateKey_file:PEM lib)....

Which ends up going for about 50-100 lines.

doing a /scripts/restartsrv_cpanel didnt work, so i did a 'service cpanel restart' instead which yeilded:

root@mercury [/scripts]# service cpanel restart
Stopping cPanel services:                                  [  OK  ]
Stopping pop3 services:                                    [  OK  ]
Stopping cPanel log services:                              [  OK  ]
Stopping cPanel Chat services:                             [FAILED]
Stopping Melange Chat services:                            [FAILED]
Stopping InterChange services:                             [FAILED]
Stopping cPanel ssl services:                              [  OK  ]

Stopping mailman services: Shutting down Mailman's master qrunner

Starting eximstats:                                        [  OK  ]
Starting cPanel services:                                  [  OK  ]
Starting cPanel Log services:                              [  OK  ]
Starting pop3 services:                                    [  OK  ]
Starting cPanel Chat services:
Starting Melange Chat services:
Starting cPanel ssl services: 2006.10.07 16:16:41 LOG3[31089:16384]: error stack: 140B3009 : error:140B3009:SSL routines:SSL_CTX_use_RSAPrivateKey_file:PEM lib
2006.10.07 16:16:41 LOG3[31089:16384]: SSL_CTX_use_RSAPrivateKey_file: 906D06C: error:0906D06C:PEM routines:PEM_read_bio:no start line
                                                           [FAILED]
Starting mailman services: Starting Mailman's master qrunner.

/cpanel and /whm still dont work, so hopefully this will better help describe the problem...

[designeru]

So... you have http://domain.com:2086 ... right ?

If yes, please go there, then SSL/TLS -> Reset Server Certificates -> cPanel/WHM Server

Then restart WHM and check.

[adivity]

No i dont. cpanel and whm are not running, or are not being allowed to bind to those ports. So /whm and /cpanel do not work, neither does :2082/:2086

Is there a way to reset the server certificate via ssh?

[adivity]

anyone know how to reset the server certificate via ssh? mwhm wont come up

[angelina_holy]

Since your cpanel and whm is not working properly , try this

Run script ./cpsrvd

ALso I noticed you are getting message frozen error ,


Exim will not bounce emails if the original message looks like a bounce. Instead the message is frozen, awaiting the SysAdmin's attention. To bounce the message with a message to the original user, and thereby removing it from the queue, use -Mg messageid. If you have a bunch of frozen messages you want to remove, the following command will do it:
# exiqgrep -zi | xargs exim -Mrm

[adivity]

Well the frozen emails are gone, but it hasnt seemed to help anything just yet.

I ran the cpsrvd, but all it returns is:

root@mercury [~]# /usr/local/cpanel/cpsrvd
==> cpsrvd 10.9.6 started
Waiting to bind to port 2082 (Address already in use)....
Waiting to bind to port 2082 (Address already in use)....
Waiting to bind to port 2082 (Address already in use)....
Waiting to bind to port 2082 (Address already in use)....
Waiting to bind to port 2082 (Address already in use)....
Waiting to bind to port 2082 (Address already in use)....
Waiting to bind to port 2082 (Address already in use)....
Waiting to bind to port 2082 (Address already in use)....
Waiting to bind to port 2082 (Address already in use)....
...

and keeps going until i kill it.

So what im thinking is something has to be using the cpanel/whm ports as well as the pop3 port. The server certificate needs to be reset, but i dont know how to do it through ssh.

anyone have any other ideas?

[brumie]

i have the same defunct problem
looks like bugs

[daveformerlyof]

Well the frozen emails are gone, but it hasnt seemed to help anything just yet.

So what im thinking is something has to be using the cpanel/whm ports as well as the pop3 port. The server certificate needs to be reset, but i dont know how to do it through ssh.

anyone have any other ideas?

Check netstat -anp to see what's on those ports. Check ps auxwww to see if duplicate cPanel processes are running. If those don't answer the issue, contact your datacenter or cPanel support.