Named shows High Load constantly...

[niccell]

Hello All,

I'm a long time 'looker', but I rarely post.

The last several days my 'named' process has gone up and up. Here are the stats from top:

GOOD:

top - 23:19:31 up 60 days, 23:17, 1 user, load average: 0.43, 1.06, 1.26
Tasks: 143 total, 3 running, 136 sleeping, 0 stopped, 4 zombie
Cpu0 : 9.0% us, 4.0% sy, 0.0% ni, 86.1% id, 0.5% wa, 0.5% hi, 0.0% si
Cpu1 : 10.9% us, 2.0% sy, 0.0% ni, 86.6% id, 0.5% wa, 0.0% hi, 0.0% si
Cpu2 : 7.5% us, 2.5% sy, 0.0% ni, 90.0% id, 0.0% wa, 0.0% hi, 0.0% si
Cpu3 : 5.0% us, 1.5% sy, 0.0% ni, 93.5% id, 0.0% wa, 0.0% hi, 0.0% si
Mem: 2074724k total, 1962028k used, 112696k free, 161620k buffers
Swap: 4096532k total, 3832k used, 4092700k free, 932304k cached

PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
18841 named 19 0 67496 13m 1956 S 35 0.7 81:12.98 named
19852 mailnull 17 0 9228 2916 1764 R 2 0.1 0:00.05 exim
19859 nobody 16 0 0 0 0 Z 1 0.0 0:00.02 httpd <defunct>
19849 nobody 15 0 43184 32m 1968 S 0 1.6 0:00.01 httpd
19857 nobody 17 0 43376 33m 2776 S 0 1.6 0:00.01 httpd
1 root 16 0 1740 500 468 S 0 0.0 1:07.68 init
2 root RT 0 0 0 0 S 0 0.0 0:10.09 migration/0
3 root 34 19 0 0 0 S 0 0.0 0:12.18 ksoftirqd/0
4 root RT 0 0 0 0 S 0 0.0 0:11.55 migration/1
5 root 34 19 0 0 0 S 0 0.0 0:03.00 ksoftirqd/1
6 root RT 0 0 0 0 S 0 0.0 0:18.17 migration/2
7 root 34 19 0 0 0 S 0 0.0 0:05.61 ksoftirqd/2
8 root RT 0 0 0 0 S 0 0.0 0:43.47 migration/3
9 root 34 19 0 0 0 S 0 0.0 0:03.61 ksoftirqd/3

BAD:

top - 23:25:26 up 60 days, 23:23, 1 user, load average: 2.04, 1.29, 1.25
Tasks: 132 total, 3 running, 126 sleeping, 0 stopped, 3 zombie
Cpu0 : 10.9% us, 6.5% sy, 0.0% ni, 81.1% id, 1.5% wa, 0.0% hi, 0.0% si
Cpu1 : 15.0% us, 2.5% sy, 0.0% ni, 82.5% id, 0.0% wa, 0.0% hi, 0.0% si
Cpu2 : 28.7% us, 5.4% sy, 0.0% ni, 62.4% id, 3.0% wa, 0.5% hi, 0.0% si
Cpu3 : 23.8% us, 4.5% sy, 0.0% ni, 65.8% id, 5.4% wa, 0.5% hi, 0.0% si
Mem: 2074724k total, 1970860k used, 103864k free, 167512k buffers
Swap: 4096532k total, 3832k used, 4092700k free, 950852k cached

PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
18841 named 19 0 67496 13m 1956 S 51 0.7 84:13.95 named
9322 mailnull 18 0 80144 71m 3328 R 13 3.5 0:35.69 MailScanner
20331 nobody 16 0 48984 37m 3224 S 2 1.8 0:00.92 httpd
20524 nobody 15 0 43184 32m 2004 S 1 1.6 0:00.08 httpd
19926 root 16 0 3188 1000 768 R 1 0.0 0:01.38 top
20124 nobody 15 0 57660 45m 3108 S 1 2.2 0:03.62 httpd
20406 nobody 15 0 46652 36m 2940 S 0 1.8 0:00.34 httpd
1 root 16 0 1740 500 468 S 0 0.0 1:07.69 init
2 root RT 0 0 0 0 S 0 0.0 0:10.09 migration/0
3 root 34 19 0 0 0 S 0 0.0 0:12.18 ksoftirqd/0
4 root RT 0 0 0 0 S 0 0.0 0:11.55 migration/1
5 root 34 19 0 0 0 S 0 0.0 0:03.00 ksoftirqd/1
6 root RT 0 0 0 0 S 0 0.0 0:18.17 migration/2
7 root 34 19 0 0 0 S 0 0.0 0:05.61 ksoftirq

It goes fine for a while, then the load spikes to 1.5 - 2.5, then back down like an all day roller coaster. It's not an errant account, it's not spam (clean), the server is not rooted/exploited that I can see. I'm also wondering if MailScanner isn't causing the issue. My partner doesn't want to remove it, but I don't like the way it brings up the load on occasion....

Any help at all is appreciated. Thank you very much in advance.

 

[nyjimbo]

You could enable named logging in named.conf and restart it, but keep an eye on the logs as they can grow pretty quickly. If you see a big spike and then watch it end you could then disable the logging, restart named and then go view the logs to see what named was responding to. Could be just about anything because named WILL answer requests for dns services from the outside world if you dont have your named.conf tweaked properly.

What version of named/bind are you running?. Lots of exploits and DOS attacks out there for older versions.

 

[niccell]

Hello!

named (9.2.4)

Not sure if that's the most current or not..

 

[niccell]

Hello,

My apologies, but I don't know how to turn on DNS logging, and I've searched the forum......

Any help please??

Thank you in advance for helping a DNS newbie...

 

[niccell]

Well, I've looked..

Recursive is closed (it only allows trusted).

named still wildly fluctuates between 10-130

Any DNS experts out there?

Thank you in advance

 

[gribozavr]

Edit your named.conf and restart named.

logging
{
    channel debug {
      file "data/debug.log" versions 3 size 5m;
      severity debug 2;
      print-category yes;
      print-severity yes;
      print-time yes;
    };
    category queries {
      debug;
    };
};

This will enable query log and put it into data/debug.log.* files, they won't grow more than 15Mb, so it is safe to leave it on.

 

[niccell]

That did the trick!

Thanks!

I was able to find the issue and deal with it effectively.

Thanks to all who assisted!

 

[nyjimbo]

Thanks!

I was able to find the issue and deal with it effectively.

Thanks to all who assisted!

For future users, can you post just a little on what you did to fix your problem. Believe me, someone else will find your solution useful at some point.

 

[niccell]

Solution:

Here is the 'fix'...

Please enter the following code in the /etc/named.conf file

logging
{
channel debug {
file "data/debug.log" versions 3 size 5m;
severity debug 2;
print-category yes;
print-severity yes;
print-time yes;
};
category queries {
debug;
};
};

Then I went to the debug.log and opened it up. IP's of the DOS person were there. I blocked them with my firewall and the issue went away.

My apologies for not posting this earlier....I was concerned the same idiot who did the DOS was reading this and would swap IP's or do something equally nasty....