The Community Forums

Interact with an entire community of cPanel & WHM users!
  1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

regular server crash - any ideas...

Discussion in 'General Discussion' started by 4u123, Feb 11, 2006.

  1. 4u123

    4u123 Well-Known Member
    PartnerNOC

    Joined:
    Jan 2, 2006
    Messages:
    765
    Likes Received:
    1
    Trophy Points:
    18
    Can anyone help me identify why this is running out of memory ?

    Message log shows...

    Feb 11 07:41:11 SERVERNAME kernel: oom-killer: gfp_mask=0x1d2
    Feb 11 07:41:29 SERVERNAME kernel: DMA per-cpu:
    Feb 11 07:41:46 SERVERNAME kernel: cpu 0 hot: low 2, high 6, batch 1
    Feb 11 07:42:36 SERVERNAME kernel: cpu 0 cold: low 0, high 2, batch 1
    Feb 11 07:42:47 SERVERNAME kernel: Normal per-cpu:
    Feb 11 07:43:17 SERVERNAME kernel: cpu 0 hot: low 32, high 96, batch 16
    Feb 11 07:43:41 SERVERNAME kernel: cpu 0 cold: low 0, high 32, batch 16
    Feb 11 07:43:49 SERVERNAME kernel: HighMem per-cpu: empty
    Feb 11 07:44:19 SERVERNAME kernel:
    Feb 11 07:44:42 SERVERNAME kernel: Free pages: 1636kB (0kB HighMem)
    Feb 11 07:45:35 SERVERNAME kernel: Active:251 inactive:110681 dirty:0 writeback:110734 unstable:0 free:409 slab:10564 mapped:4845 pagetables:3367
    Feb 11 07:45:38 SERVERNAME kernel: DMA free:20kB min:20kB low:40kB high:60kB active:0kB inactive:11124kB present:16384kB
    Feb 11 07:45:44 SERVERNAME kernel: protections[]: 0 0 0
    Feb 11 07:45:45 SERVERNAME kernel: Normal free:1616kB min:692kB low:1384kB high:2076kB active:1004kB inactive:431600kB present:499648kB
    Feb 11 07:45:46 SERVERNAME kernel: protections[]: 0 0 0
    Feb 11 07:45:48 SERVERNAME kernel: HighMem free:0kB min:128kB low:256kB high:384kB active:0kB inactive:0kB present:0kB
    Feb 11 07:45:50 SERVERNAME kernel: protections[]: 0 0 0
    Feb 11 07:45:50 SERVERNAME kernel: DMA: 1*4kB 0*8kB 1*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 20kB
    Feb 11 07:45:50 SERVERNAME kernel: Normal: 166*4kB 83*8kB 0*16kB 3*32kB 3*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 1616kB
    Feb 11 07:45:50 SERVERNAME kernel: HighMem: empty
    Feb 11 07:46:09 SERVERNAME kernel: Swap cache: add 831251, delete 720514, find 6713034/6771598, race 0+9
    Feb 11 07:47:59 SERVERNAME kernel: Out of Memory: Killed process 4572 (mysqld).
    Feb 11 07:48:24 SERVERNAME kernel: Out of Memory: Killed process 4622 (mysqld).
    Feb 11 07:48:31 SERVERNAME kernel: Out of Memory: Killed process 4624 (mysqld).
    Feb 11 07:48:51 SERVERNAME kernel: Out of Memory: Killed process 4638 (mysqld).
    Feb 11 07:48:58 SERVERNAME kernel: Out of Memory: Killed process 4639 (mysqld).
    Feb 11 07:49:37 SERVERNAME kernel: Out of Memory: Killed process 4640 (mysqld).
    Feb 11 07:49:42 SERVERNAME kernel: Out of Memory: Killed process 4641 (mysqld).
    Feb 11 07:49:50 SERVERNAME kernel: Out of Memory: Killed process 4642 (mysqld).
    Feb 11 07:50:00 SERVERNAME kernel: Out of Memory: Killed process 4643 (mysqld).
    Feb 11 07:50:31 SERVERNAME kernel: Out of Memory: Killed process 4644 (mysqld).
    Feb 11 07:52:16 SERVERNAME pure-ftpd: (?@IP.IP.IP.IP) [INFO] New connection from IP.IP.IP.IP
    Feb 11 07:54:34 SERVERNAME kernel: Out of Memory: Killed process 4645 (mysqld).
    Feb 11 07:55:33 SERVERNAME kernel: Out of Memory: Killed process 4646 (mysqld).
    Feb 11 07:57:13 SERVERNAME kernel: Out of Memory: Killed process 4647 (mysqld).
    Feb 11 07:57:14 SERVERNAME kernel: Out of Memory: Killed process 4648 (mysqld).
    Feb 11 07:57:14 SERVERNAME kernel: Out of Memory: Killed process 4649 (mysqld).
    Feb 11 07:57:14 SERVERNAME kernel: Out of Memory: Killed process 4650 (mysqld).
    Feb 11 07:57:16 SERVERNAME kernel: Out of Memory: Killed process 4651 (mysqld).
    Feb 11 07:57:41 SERVERNAME kernel: Out of Memory: Killed process 4652 (mysqld).
    Feb 11 07:57:31 SERVERNAME pure-ftpd: (?@IP.IP.IP.IP) [INFO] Logout.
    Feb 11 07:57:43 SERVERNAME kernel: Out of Memory: Killed process 4653 (mysqld).
    Feb 11 08:00:35 SERVERNAME kernel: Out of Memory: Killed process 4654 (mysqld).
    Feb 11 08:00:39 SERVERNAME kernel: Out of Memory: Killed process 4655 (mysqld).
    Feb 11 08:00:39 SERVERNAME freshclam[3639]: Received signal: wake up
    Feb 11 08:00:42 SERVERNAME kernel: Out of Memory: Killed process 4656 (mysqld).
    Feb 11 08:00:44 SERVERNAME pure-ftpd: (?@IP.IP.IP.IP) [INFO] New connection from IP.IP.IP.IP
    Feb 11 08:00:45 SERVERNAME crond(pam_unix)[21809]: session opened for user root by (uid=0)
    Feb 11 08:00:45 SERVERNAME crond(pam_unix)[21865]: session opened for user tvacnnet by (uid=0)
    Feb 11 08:00:45 SERVERNAME crond(pam_unix)[21859]: session opened for user root by (uid=0)
    Feb 11 08:00:46 SERVERNAME kernel: Out of Memory: Killed process 4663 (mysqld).
    Feb 11 08:00:47 SERVERNAME freshclam[3639]: ClamAV update process started at Sat Feb 11 08:00:46 2006
    Feb 11 08:00:49 SERVERNAME pure-ftpd: (?@IP.IP.IP.IP) [INFO] Logout.
    Feb 11 08:01:42 SERVERNAME crond(pam_unix)[22055]: session opened for user root by (uid=0)
    Feb 11 08:01:58 SERVERNAME kernel: oom-killer: gfp_mask=0xd2
    Feb 11 08:02:05 SERVERNAME crond(pam_unix)[22162]: session opened for user root by (uid=0)
    Feb 11 08:02:05 SERVERNAME crond(pam_unix)[22161]: session opened for user root by (uid=0)
    Feb 11 08:02:05 SERVERNAME freshclam[3639]: main.cvd is up to date (version: 35, sigs: 41649, f-level: 6, builder: tkojm)
    Feb 11 08:02:29 SERVERNAME crond(pam_unix)[21801]: session closed for user root

    LOTS MORE CROND SESSIONS HERE SAME AS ABOVE (didnt paste them all)


    Feb 11 08:11:51 SERVERNAME pure-ftpd: (?@IP.IP.IP.IP) [INFO] New connection from IP.IP.IP.IP
    Feb 11 08:16:25 SERVERNAME crond(pam_unix)[21852]: session closed for user root
    Feb 11 08:24:51 SERVERNAME pure-ftpd: (?@IP.IP.IP.IP) [INFO] New connection from IP.IP.IP.IP
    Feb 11 08:31:19 SERVERNAME pure-ftpd: (?@IP.IP.IP.IP) [INFO] New connection from IP.IP.IP.IP
    Feb 11 08:31:42 SERVERNAME kernel: DMA per-cpu:
    Feb 11 08:32:06 SERVERNAME crond(pam_unix)[22417]: session opened for user root by Feb 11 08:32:43 SERVERNAME crond(pam_unix)[21795]: session closed for user root
    Feb 11 08:32:46 SERVERNAME crond(pam_unix)[21859]: session closed for user root
    Feb 11 08:33:07 SERVERNAME crond(pam_unix)[21837]: session closed for user root
    Feb 11 08:33:17 SERVERNAME crond(pam_unix)[21864]: session closed for user root
    Feb 11 08:33:20 SERVERNAME crond(pam_unix)[21798]: session closed for user root
    Feb 11 08:33:38 SERVERNAME crond(pam_unix)[21857]: session closed for user root
    Feb 11 08:33:47 SERVERNAME crond(pam_unix)[21820]: session closed for user root
    Feb 11 08:33:46 SERVERNAME crond(pam_unix)[21804]: session closed for user root
    Feb 11 08:33:47 SERVERNAME crond(pam_unix)[21789]: session closed for user root
    Feb 11 08:33:49 SERVERNAME crond(pam_unix)[21807]: session closed for user root
    Feb 11 08:33:49 SERVERNAME crond(pam_unix)[21856]: session closed for user root
    Feb 11 08:33:49 SERVERNAME crond(pam_unix)[21822]: session closed for user root
    Feb 11 08:33:52 SERVERNAME crond(pam_unix)[21793]: session closed for user root
    Feb 11 08:34:16 SERVERNAME crond(pam_unix)[23400]: session opened for user root by (uid=0)
    Feb 11 08:34:47 SERVERNAME crond(pam_unix)[21860]: session closed for user root
    Feb 11 09:05:28 SERVERNAME syslogd 1.4.1: restart.

    Can a mysql process eat up all the memory like this ? Is a user abusing mysql ?

    I guess the reason for the number of crond entries is related to the frequency we are running sim and antidos...

    SHELL=/bin/bash
    PATH=/sbin:/bin:/usr/sbin:/usr/bin
    MAILTO=root
    HOME=/

    # run-parts
    01 * * * * root run-parts /etc/cron.hourly
    02 4 * * * root run-parts /etc/cron.daily
    22 4 * * 0 root run-parts /etc/cron.weekly
    42 4 1 * * root run-parts /etc/cron.monthly
    */5 * * * * root /usr/local/sim/sim -q >> /dev/null 2>&1
    */2 * * * * root /etc/apf/ad/antidos -a >> /dev/null 2>&1

    Box has only 512MB but this isnt normally a problem.

    Any ideas? I'm still not an expert on this yet.
     
  2. mctDarren

    mctDarren Well-Known Member

    Joined:
    Jan 6, 2004
    Messages:
    664
    Likes Received:
    2
    Trophy Points:
    18
    Location:
    New Jersey
    cPanel Access Level:
    Root Administrator
    They might not be purposefully abusing it, but they might be running something that is too intensive for the box. They are probably just running a script that is very sql intensive - check to see what it is that script is doing. 512MB can be used up pretty quick on a mysql intensive box but do a search on 'optimizing mysql' and see if that helps.
     
Loading...

Share This Page