cPanel 11.28.45 - upcp and ipaliases (not restarting)

[MikeDVB]

We had a server that had a reported outage tonight, we nailed it down to happening during the /scripts/upcp that runs nightly and it looks as though cPanel restarted networking without restarting ipaliases causing any secondary routed IP addresses to fail.

Has anybody else experienced this issue? In talking with a couple of other providers - they've seen it a couple of times but it appears to be rare but with the type of outage this can cause even once is too often in my opinion.

 

[VeZoZ]

We've seen it on several machines now when updating cPanel. It shows ipaliases being restarted but I'm not sure what exactly goes wrong. Although my question is why is it even messing with the network when you run an update? It seems like a strange and unnecessary thing to be doing.

 

[MikeDVB]

http://www.screen-shot.net/2010-11-25_0305.png
http://www.screen-shot.net/2010-11-25_0305_001.png

Ultimately I think that the ipaliases start/restart/whatever should be removed from the update script.

 

[cPanelNick]

Please check the netmasks on the ips in /etc/ips. This is usually the a symptom of a misconfigured netmask.

 

[MikeDVB]

Netmasks are 255.255.255.0. Ticket ID is 1047278 if you want to take a look - in speaking with about a dozen providers ... I'm not the only one to see this as of 11.28.

This was a fresh CentOS5 install with a fresh cPanel 11.28 installation - all IP addresses besides the primary on eth0 were added via WHM.

At this point it looks like we're going to have to remove the cron for /scripts/upcp until this is resolved. I still see no reason why upcp runs anything having to do with ip aliasing...

 

[VeZoZ]

11.25:

Checking for main ip changes......Done
Checking mySQL to ensure we can connect......Done
Sending usage data (if enabled)...Usage data is required!
...Done
Validating cPanel system users...Homedir: /var/cpanel/userhomes/cpanel
...Done
Updating email whitelists......Done
Checking and Repairing System Package Setup......Done
Running Platform Specific Optimizations......Done
Checking and Repairing Nameserver Config......Done
Setting up ip startup......Done
Configuring editors......Done
Creating symlinks for common binaries......Done
Setting Clock......Done
Running Env Auto Repair...quota test: check passed (the repquota signal was 0 [ok])
Requesting script ... Done
Auto Repair is running...Running Auto Repair routines
Finished running Auto Repair routines
...Auto Repair is done.
...Done

11.28:

Checking for main ip changes......Done
Checking mySQL to ensure we can connect......Done
Sending usage data (if enabled)...Anonymous usage data accepted [2741].
...Done
Validating cPanel system users...Homedir: /var/cpanel/userhomes/cpanel
...Done
Updating email whitelists......Done
Checking and Repairing System Package Setup......Done
Running Platform Specific Optimizations......Done
Checking and Repairing Nameserver Config......Done
Setting up ip startup...enable was successful
[ipaliases] Bringing up eth1:1 [ OK ]... 
(it goes on all ip's obviously)
Configuring editors......Done
Creating symlinks for common binaries......Done
Setting Clock......Done
Running Env Auto Repair...quota test: check passed
Requesting script ... Done
Auto Repair is running...Running Auto Repair routines
Finished running Auto Repair routines
...Auto Repair is done.
...Done

 

[MikeDVB]

Thank you for your suggestion. I will take a note of that and will report to my supervisor.

Coming back to the issue you are having, do i have your permission to run "upcp" to reproduce the issue you are having with ipaliases? Please advise.

Honestly I feel like this is being blown off... I guess the only option at this point until somebody at cPanel decides that running ipaliases on every /scripts/upcp isn't a good idea is to disable the cron job and run it manually and then check to make sure the IPs bound the way they were supposed to.

I really shouldn't have to actively disable features of cPanel so that our servers don't experience issues.

 

[MikeDVB]

Not sure what is going on with cPanel's MySQL but my last post disappeared so I'm going to re-post it (I happened to save a copy of the thread just before this happened).

Edit: Looks like the MySQL servers re-converged.

 

[MikeDVB]

The code from 11.25 /scripts/upcp:

if ( -x '/sbin/chkconfig' ) {
        $output .= cPScript::WHMAPI::setstatus('Setting up ip startup');
        cPScript::SafeRun::Errors::saferunnoerror( '/sbin/chkconfig', '--add',     'ipaliases' );
        cPScript::SafeRun::Errors::saferunnoerror( '/sbin/chkconfig', '--level',   '35', 'ipaliases', 'on' );
        cPScript::SafeRun::Errors::saferunnoerror( '/sbin/chkconfig', 'ipaliases', '35' );
        $output .= cPScript::WHMAPI::setstatusdone();
    }

The code from 11.28 /scripts/upcp:

    $output .= Cpanel::WHMAPI::setstatus('Setting up ip startup');

    if ( -x '/scripts/cpservice' ) {
        $output .= Cpanel::WHMAPI::status_cmd( '/scripts/cpservice', 'ipaliases', 'enable', '35' );
        my $ipaliases_start = Cpanel::SafeRun::Errors::saferunallerrors('/scripts/cpservice', 'ipaliases', 'start' );
        $ipaliases_start =~ s/S[^\:]+:\s*File\s*exists\s*\[\s*FAILED\s*\]/\[ UP \]/g; # if it is already in the routing table we do not need to throw an err$
        print $ipaliases_start;
        $output .= $ipaliases_start;
    }
    else {
        $output .= Cpanel::WHMAPI::setstatus('/scripts/cpservice is missing or not executable.');
    }

    $output .= Cpanel::WHMAPI::setstatusdone();

Uggghhh.... Why change this? In 11.25 it made sure that the service was set to start on boot - in 11.28 for some reason it physically tries starting it every time the update runs... This makes no sense.

What this function did in 11.25 and earlier was make sure that the service ipaliases was set to start on boot (which makes sense)... 11.28 for whatever reason "Setting up ip startup" (i.e. setting it to start on boot) isn't really that at all.

 

[cPanelNick]

Hi Mike. You are definitely not being blown off. I am very sorry that we gave you that impression. Your ticket was already escalated by the time I looked at it this morning. It is a holiday (Thanksgiving) in the states today so responses may be a bit slower then usual.

 

[MikeDVB]

I was referring to the feeling I got from that one specific response. It was escalated because I asked for it and I appreciate it.

Thank you.

 

[VeZoZ]

We've had to make so /scripts/upcp does not run at all unless a technician does it. It's just to unpredictable as some times ipaliases has issues even with proper netmasks and such. It does not happen frequently but when you have enough machines it does happen once in a while just from the fact you have enough machines running it once a day.

There does not appear to be a reason to be doing this every time upcp runs. If cPanel now thinks it should be ran we need a way to opt out. It's causing problems for our fleet of machines all with properly assigned IP's.

 

[MikeDVB]

We've had to make so /scripts/upcp does not run at all unless a technician does it. It's just to unpredictable as some times ipaliases has issues even with proper netmasks and such. It does not happen frequently but when you have enough machines it does happen once in a while just from the fact you have enough machines running it once a day.

We've done the same.

 

[sebby]

Not running 11.28.45 just yet but how can you fix this (i.e. restart ipaliases) when it is happening?

I understand that the main IP is not affected so you still can access the server?

Was this issue also reported in 11.28.48+?

 

[MikeDVB]

I've not yet been notified that it's been fixed - I do believe that they reverted the way /scripts/upcp works with IPs until an official fix is in place.

 

[WebHostPro]

I think I ran into the same problem today. I put in a ticket but on my server it will not take any additional IP's just the main IP will work.

I can't imagine it crashing the existing additional dedicated IP's that would be a disaster!

When i try to run ip ipaliases I get this error on all additional IP's

Routing 66.249.xx.xx SIOCADDRT: No such device

/etc/ips shows the correct information.

 

[Indianets]

I had the exact same issue on one of the servers. As soon as I realized that the IP's are not responding i did

service ipaliases restart

and it readded all the virtual interfaces.

- Vijay

 

[WebHostPro]

I had the exact same issue on one of the servers. As soon as I realized that the IP's are not responding i did

service ipaliases restart

and it readded all the virtual interfaces.

- Vijay

Thanks, I tried that but the error is still there.

But Cpanel found the error and there etho 0 was not turned on so there was no alternative port to run the additional IP's on.

I turned it on in the basic WHM config section and now it works.

Thanks!

 

[alxcds]

Was this ever resolved?

 

[MikeDVB]

Was this ever resolved?

Yes, it was fixed in newer revisions of cPanel.