The Community Forums

Interact with an entire community of cPanel & WHM users!
  1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

FreeBSD locking up

Discussion in 'General Discussion' started by kryogen, Aug 15, 2004.

  1. kryogen

    kryogen Member
    PartnerNOC

    Joined:
    Feb 3, 2004
    Messages:
    18
    Likes Received:
    0
    Trophy Points:
    1
    Hello,

    I have two servers which run FreeBSD 4.x and 5.x. When I put Cpanel on either one, they randomly lock up within a 1-2 week period. The systems become non-responsive and I am forced to reboot via console.

    I'm wondering if anyone here has had this problem using FreeBSD w/Cpanel and what might be the cause. Should I disable backups? I don't auto update - everything is set to manual.
     
  2. easyhoster1

    easyhoster1 Well-Known Member

    Joined:
    Sep 25, 2003
    Messages:
    659
    Likes Received:
    0
    Trophy Points:
    16
  3. kryogen

    kryogen Member
    PartnerNOC

    Joined:
    Feb 3, 2004
    Messages:
    18
    Likes Received:
    0
    Trophy Points:
    1
    Thanks. I had actually looked at that thread and tried the suggestions in it however it doesn't appear to resolve the issue long term.

    Depending on how much RAM you have in the machine, in my case 512MB, my FreeBSD systems with Cpanel installed on them gradually increase in wired memory. I've been recording the usage and so far have this:

    date 8/15/04 - 97MB wired

    date 8/16/04 - 105MB wired

    date 8/17/04 - 109MB wired

    If I reinstall FreeBSD without Cpanel, the wired memory stays at around 67MB and does not increase. As soon as I install Cpanel the wired memory begins increasing slightly and the server will eventually lock up. So far it's every couple of weeks.

    I have set updates in WHM to "manual" but noticed that cpanel runs /scripts/upcp anyway and then stops or continues based on what your settings are. Could the increase in memory be related to the backup procedure? I have that running nightly. I guess I could try disabling it for a few days.
     
  4. easyhoster1

    easyhoster1 Well-Known Member

    Joined:
    Sep 25, 2003
    Messages:
    659
    Likes Received:
    0
    Trophy Points:
    16
    After you add the cron, you need to reboot. We have machines with 512 RAM and they stay under 80 MB Wired now.

    The other thing we have done is configure the backup to run every other day and set the update to "never update" unless we are doing manual updates, then we reset to manual to get the latest. However, upcp will run every night even with those settings checked to never update. Our wired memory issue is no longer a problem with the cron added.
     
  5. kryogen

    kryogen Member
    PartnerNOC

    Joined:
    Feb 3, 2004
    Messages:
    18
    Likes Received:
    0
    Trophy Points:
    1
    Sweet! I will reboot this beast tonight. Thanks.
     
  6. kryogen

    kryogen Member
    PartnerNOC

    Joined:
    Feb 3, 2004
    Messages:
    18
    Likes Received:
    0
    Trophy Points:
    1
    Just a follow up: my problems with increase in wired memory appear to be caused by the Cpanel backup scripts (/scripts/cpbackup). I'm currently backing up about 10GB worth of data.

    Not sure why I didn't try this before but I manually ran /scripts/upcp today. The result was no increase in wired memory so upcp doesn't appear to be causing any issues.

    Next, I manually ran /scripts/cpbackup. I did this two times. Each time the servers wired memory increased by about 5-10MB. The backup configuration is very simple, I'm just sending things to a second drive installed on the system as /backup.

    I guess what I'll probably end up doing is disabling Cpanel backups and just use my own. I'm not sure how the rest of the Cpanel/FreeBSD users here are able to use Cpanel backups without wired memory increasing gradually over time. I'd be interested if some could share their backup configurations - perhaps they are doing incremental or something.

    I've got a really simple perl script that I use for backups on my other FreeBSD systems - works great. If any BSD/Cpanel users want it feel free to PM me. I'll have to modify it to grab all the Cpanel config files and other junk.
     
  7. iko

    iko Well-Known Member
    PartnerNOC

    Joined:
    Jan 29, 2004
    Messages:
    56
    Likes Received:
    0
    Trophy Points:
    6
    PM it to me please. Thanks :)
     
  8. kryogen

    kryogen Member
    PartnerNOC

    Joined:
    Feb 3, 2004
    Messages:
    18
    Likes Received:
    0
    Trophy Points:
    1
    Actually, removing cpanel backups didn't work either, it helped prolong the inevitable.

    I've started a ticket with Cpanel on this issue. It's gone on for far too long and it's looking like the developers really don't care about how stable their own product is on the FreeBSD platform. I've tried FreeBSD 4.8, 4.9, 5.0 and 5.1.

    I have noticed that of the two systems I'm currently running, one is busier than the other and the busy server will lock up on average once per week while the less busy server can go up to 30 days before running out of wired memory. This looks to me like a particular service is causing the problem, maybe one of the Cpanel apache modules? Unfortunately both servers are production and I can't start turning services off or disabling modules.

    I can honestly say that some systems that aren't busy which have at least 1GB of memory in them can probably stay up for 60-90 days without issue.
     
  9. StevenC

    StevenC Well-Known Member

    Joined:
    Jan 1, 2004
    Messages:
    254
    Likes Received:
    0
    Trophy Points:
    16
    are you sure you do not have a hardware problem 90% of the time thats the case.
     
  10. kryogen

    kryogen Member
    PartnerNOC

    Joined:
    Feb 3, 2004
    Messages:
    18
    Likes Received:
    0
    Trophy Points:
    1
    Originally the first thing I did was start swapping out hardware on the first Cpanel/FreeBSD system I built. That was my guess too however after replacing everything the problem still existed, then after building two more Cpanel/FreeBSD boxes the same behavior occurred on those as well.

    I finally found a bug in the cpanellogd daemon that when run on a FreeBSD system, it somehow never lets go of shared memory (Wired) whenever it processes a lot of logs that are large in size. Unfortunately you can't duplicate this problem on a test box or a box with only 1 or 2 moderately used sites. It must be duplicated on a machine that has a lot of sites with fairly large log files. This is probably why the problem has existed for so long.

    I did find one other user on this forum who suggested that cpanellogd was driving up his shared memory but a bug report was never filed. I can't duplicate the problem under RedHat Linux.
     
Loading...

Share This Page