The Community Forums

Interact with an entire community of cPanel & WHM users!
  1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

Too many conn's with FIN_WAIT2 and TIME_WAIT status

Discussion in 'General Discussion' started by guschi2k, Feb 4, 2006.

  1. guschi2k

    guschi2k Member

    Joined:
    Sep 11, 2003
    Messages:
    11
    Likes Received:
    0
    Trophy Points:
    1
    Hi everyone,

    like the topic says, my server has tons of connections in FIN_WAIT2 and TIME_WAIT status and I have no idea whats causing it :( It's a new box and it has been like that pretty much from the beginning (past 6 days). I think the CPU is not overloaded and always only between 0.2 and 0.5.
    I have set up a 10 minute cronjob a few hours ago to log the output of netstat -tn into files and the largest file has 60+ Kbytes.

    I would be very thankful, if someone has an idea about what is happening here.

    Thanks a lot! :)

    guschi


    Heres the current first 100 of 600+ lines of the netstat -tn output.
    Code:
    tcp   0      0 my.srv.ip.addy:80   203.211.195.215:3142    SYN_RECV
    tcp   0      0 my.srv.ip.addy:80   60.42.127.20:4560       SYN_RECV
    tcp   0      0 my.srv.ip.addy:80   221.88.152.54:4560      SYN_RECV
    tcp   0      0 my.srv.ip.addy:80   210.237.14.150:4532     SYN_RECV
    tcp   0      0 my.srv.ip.addy:80   218.251.76.8:2178       SYN_RECV
    tcp   0      0 my.srv.ip.addy:80   221.191.189.71:4265     ESTABLISHED
    tcp   0      0 my.srv.ip.addy:80   125.192.197.27:3315     TIME_WAIT
    tcp   0      0 my.srv.ip.addy:110  84.162.196.224:38804    TIME_WAIT
    tcp   0      0 my.srv.ip.addy:80   202.208.54.182:49216    FIN_WAIT2
    tcp   0      0 my.srv.ip.addy:110  84.162.196.224:38806    TIME_WAIT
    tcp   0      0 my.srv.ip.addy:80   219.55.15.104:3768      FIN_WAIT2
    tcp   0      0 my.srv.ip.addy:80   219.203.30.72:2420      FIN_WAIT2
    tcp   0      0 my.srv.ip.addy:80   221.119.164.248:2877    ESTABLISHED
    tcp   0      0 my.srv.ip.addy:80   203.140.182.157:2225    TIME_WAIT
    tcp   0      0 my.srv.ip.addy:80   220.63.48.159:4742      ESTABLISHED
    tcp   0      0 my.srv.ip.addy:80   60.42.159.105:3381      ESTABLISHED
    tcp   0      0 my.srv.ip.addy:80   220.163.51.87:4141      FIN_WAIT2
    tcp   0      0 my.srv.ip.addy:80   220.151.65.5:1849       FIN_WAIT2
    tcp   0      0 my.srv.ip.addy:80   81.158.171.99:50610     ESTABLISHED
    tcp   0      0 my.srv.ip.addy:80   219.50.79.33:1721       FIN_WAIT2
    tcp   0      0 my.srv.ip.addy:80   81.158.171.99:50611     ESTABLISHED
    tcp   0      0 my.srv.ip.addy:80   219.55.78.26:2437       TIME_WAIT
    tcp   0      0 my.srv.ip.addy:80   81.158.171.99:50608     ESTABLISHED
    tcp   0      0 my.srv.ip.addy:80   59.171.84.157:33147     FIN_WAIT2
    tcp   0      0 my.srv.ip.addy:80   80.127.43.83:16609      ESTABLISHED
    tcp   0      0 my.srv.ip.addy:80   81.158.171.99:50609     ESTABLISHED
    tcp   0      0 my.srv.ip.addy:80   221.119.164.248:2868    TIME_WAIT
    tcp   0      0 my.srv.ip.addy:80   222.9.19.236:3817       FIN_WAIT2
    tcp   0      0 my.srv.ip.addy:80   219.129.119.92:3147     FIN_WAIT2
    tcp   0      0 my.srv.ip.addy:80   210.235.253.75:1724     TIME_WAIT
    tcp   0      0 my.srv.ip.addy:80   219.98.9.146:3303       FIN_WAIT2
    tcp   0      0 my.srv.ip.addy:80   202.233.243.253:2809    TIME_WAIT
    tcp   0      0 my.srv.ip.addy:110  84.162.196.224:38808    TIME_WAIT
    tcp   0      0 my.srv.ip.addy:80   202.142.195.71:2024     FIN_WAIT2
    tcp   0      0 my.srv.ip.addy:80   210.237.7.173:3162      TIME_WAIT
    tcp   0      0 my.srv.ip.addy:80   219.115.60.160:2034     ESTABLISHED
    tcp   0      0 my.srv.ip.addy:80   219.38.200.141:3454     FIN_WAIT2
    tcp   0      0 my.srv.ip.addy:80   210.194.108.208:52066   TIME_WAIT
    tcp   0      0 my.srv.ip.addy:80   202.173.101.8:63523     FIN_WAIT2
    tcp   0      0 my.srv.ip.addy:80   220.4.66.73:1266        FIN_WAIT2
    tcp   0      0 my.srv.ip.addy:80   60.34.222.139:2165      FIN_WAIT2
    tcp   0      0 my.srv.ip.addy:80   219.51.197.11:3075      FIN_WAIT2
    tcp   0     26 my.srv.ip.addy:80   61.198.172.149:1278     FIN_WAIT1
    tcp   0      0 my.srv.ip.addy:80   61.127.151.251:64018    FIN_WAIT2
    tcp   0      0 my.srv.ip.addy:80   218.127.110.81:3256     FIN_WAIT2
    tcp   0      0 my.srv.ip.addy:80   221.85.155.3:1076       TIME_WAIT
    tcp   0      0 my.srv.ip.addy:80   222.209.145.71:3837     FIN_WAIT2
    tcp   0      0 my.srv.ip.addy:80   218.139.33.91:4107      FIN_WAIT2
    tcp   0      0 my.srv.ip.addy:80   61.208.191.7:1938       FIN_WAIT2
    tcp   0      0 my.srv.ip.addy:80   58.70.55.20:3743        TIME_WAIT
    tcp   0      0 my.srv.ip.addy:80   222.172.138.243:3024    FIN_WAIT2
    tcp   0      0 my.srv.ip.addy:80   219.35.86.186:1226      FIN_WAIT2
    tcp   0      1 my.srv.ip.addy:80   219.114.63.107:58589    FIN_WAIT1
    tcp   0      0 my.srv.ip.addy:80   218.44.21.86:3227       FIN_WAIT2
    tcp   0      0 my.srv.ip.addy:80   60.45.68.200:1451       ESTABLISHED
    tcp   0      0 my.srv.ip.addy:80   219.112.60.188:61178    FIN_WAIT2
    tcp   0      0 my.srv.ip.addy:80   220.27.22.242:2037      FIN_WAIT2
    tcp   0      0 my.srv.ip.addy:80   220.41.10.87:4226       FIN_WAIT2
    tcp   0      0 my.srv.ip.addy:80   210.149.190.139:36777   TIME_WAIT
    tcp   0      0 my.srv.ip.addy:80   125.172.3.119:2350      TIME_WAIT
    tcp   0      0 my.srv.ip.addy:80   221.184.217.199:4783    TIME_WAIT
    tcp   0      0 my.srv.ip.addy:80   219.31.182.14:3152      FIN_WAIT2
    tcp   0      0 my.srv.ip.addy:80   220.111.59.172:4336     FIN_WAIT2
    tcp   0      0 my.srv.ip.addy:80   219.167.255.2:38827     TIME_WAIT
    tcp   0      0 my.srv.ip.addy:80   58.88.54.27:3460        TIME_WAIT
    tcp   0      0 my.srv.ip.addy:80   210.253.89.118:51235    TIME_WAIT
    tcp   0      0 my.srv.ip.addy:80   219.55.78.26:2449       TIME_WAIT
    tcp   0      0 my.srv.ip.addy:80   219.21.141.207:62042    FIN_WAIT2
    tcp   0      0 my.srv.ip.addy:110  84.162.196.224:38794    TIME_WAIT
    tcp   0      0 my.srv.ip.addy:80   220.221.115.21:1655     FIN_WAIT2
    tcp   0      0 my.srv.ip.addy:80   202.156.6.51:20580      TIME_WAIT
    tcp   0      0 my.srv.ip.addy:80   218.129.140.140:28083   TIME_WAIT
    tcp   0      0 my.srv.ip.addy:80   219.33.241.6:3608       TIME_WAIT
    tcp   0      0 my.srv.ip.addy:80   218.115.100.1:33752     FIN_WAIT2
    tcp   0      0 my.srv.ip.addy:80   60.238.159.121:2759     FIN_WAIT2
    tcp   0      0 my.srv.ip.addy:80   222.10.117.46:1945      ESTABLISHED
    tcp   0      0 my.srv.ip.addy:80   218.106.76.247:4070     FIN_WAIT2
    tcp   0      0 my.srv.ip.addy:80   59.87.124.114:3217      ESTABLISHED
    tcp   0      0 my.srv.ip.addy:80   61.24.185.143:1304      TIME_WAIT
    tcp   0      0 my.srv.ip.addy:80   218.41.180.115:62756    TIME_WAIT
    tcp   0      0 my.srv.ip.addy:80   222.146.86.157:1900     FIN_WAIT2
    tcp   0      0 my.srv.ip.addy:80   my.srv.ip.addy:46631     TIME_WAIT
    tcp   0      0 my.srv.ip.addy:80   219.56.115.150:1513     FIN_WAIT2
    tcp   0      0 my.srv.ip.addy:80   211.19.160.3:2932       TIME_WAIT
    tcp   0      0 my.srv.ip.addy:80   my.srv.ip.addy:46632     TIME_WAIT
    tcp   0      0 my.srv.ip.addy:80   220.14.90.158:3313      FIN_WAIT2
    tcp   0      0 my.srv.ip.addy:80   my.srv.ip.addy:46633     TIME_WAIT
    tcp   0      0 my.srv.ip.addy:80   210.147.194.247:2972    TIME_WAIT
    tcp   0      0 my.srv.ip.addy:80   58.94.76.33:3062        FIN_WAIT2
    tcp   0      0 my.srv.ip.addy:80   my.srv.ip.addy:46634     TIME_WAIT
    tcp   0      0 my.srv.ip.addy:80   220.109.170.13:1037     TIME_WAIT
    tcp   0      0 my.srv.ip.addy:80   60.37.238.17:49437      ESTABLISHED
    tcp   0      0 my.srv.ip.addy:80   68.99.189.37:2108       ESTABLISHED
    tcp   0      0 my.srv.ip.addy:80   my.srv.ip.addy:46635     TIME_WAIT
    tcp   0      0 my.srv.ip.addy:80   221.65.9.143:4095       FIN_WAIT2
    tcp   0      0 my.srv.ip.addy:80   221.16.86.232:4502      FIN_WAIT2
    tcp   0      0 my.srv.ip.addy:80   218.230.157.235:61103   FIN_WAIT2
    tcp   0      0 my.srv.ip.addy:80   219.33.241.6:3606       TIME_WAIT
    tcp   0      0 my.srv.ip.addy:80   210.156.29.64:2049      FIN_WAIT2
    tcp   0      0 my.srv.ip.addy:80   125.0.83.241:3741       TIME_WAIT
    
     
  2. madaboutlinux

    madaboutlinux Well-Known Member

    Joined:
    Jan 24, 2005
    Messages:
    1,052
    Likes Received:
    2
    Trophy Points:
    38
    Location:
    Earth
    You will get the description of FIN_WAIT2 & TIME_WAIT in the man page of netstat. Type 'man netstat' in the shell. You can minimize those FIN_WAIT2 & TIME_WAIT states by doing the below things :-

    echo 1 > /proc/sys/net/ipv4/tcp_syncookies

    Put following in /etc/sysctl.conf
    # Enable TCP SYN cookie protection
    net.ipv4.tcp_syncookies = 1

    # Decrease the time default value for tcp_fin_timeout connection
    net.ipv4.tcp_fin_timeout = 30

    # Turn off the tcp_window_scaling
    net.ipv4.tcp_window_scaling = 0

    # Turn off the tcp_sack
    net.ipv4.tcp_sack = 0

    Then execute the command :-
    # /sbin/sysctl -p

    You can also execute the following commands to minimize the syn attack in the future :-
    iptables -A INPUT -p tcp --tcp-flags SYN,FIN SYN,FIN -j DROP
    iptables -A INPUT -p tcp --tcp-flags SYN,RST SYN,RST -j DROP
    iptables -A INPUT -p tcp --tcp-flags FIN,RST FIN,RST -j DROP
    iptables -A INPUT -p tcp --tcp-flags ACK,FIN FIN -j DROP
     
  3. guschi2k

    guschi2k Member

    Joined:
    Sep 11, 2003
    Messages:
    11
    Likes Received:
    0
    Trophy Points:
    1
    Awesome, now it seems to stay below 100 connections! :)

    Thanks so much Shekhar :)
     
  4. djblamire

    djblamire Well-Known Member

    Joined:
    May 3, 2003
    Messages:
    250
    Likes Received:
    0
    Trophy Points:
    16
    Thank you very much for that.

    Out of interest, what do the about iptables actually block on ?? - I've used them before to block a specific ip address.

    Thanks again
    Daniel
     
  5. madaboutlinux

    madaboutlinux Well-Known Member

    Joined:
    Jan 24, 2005
    Messages:
    1,052
    Likes Received:
    2
    Trophy Points:
    38
    Location:
    Earth

    It means inspect the SYN and FIN flags and if they are both set, drop the packet. That is just look at the SYN and FIN flags to see if the rule matches. The first pair(SYN,FIN) are the flags to inspect, the second pair after the space is the state of the flags to test.

    The SYN and Fin flags are used when establishing and terminating a TCP connection, respectively.
    The ACK flag is set any time, implying that the receiver should pay attention to it.
    The RST flag signifies that the receiver wants to abort the connection.
     
  6. djblamire

    djblamire Well-Known Member

    Joined:
    May 3, 2003
    Messages:
    250
    Likes Received:
    0
    Trophy Points:
    16
    Thanks for your reply and explanation.

    So these wouldn't be used during a normal session ?? - ie, would this block any genuine connections too ??

    Thanks again
    Daniel
     
  7. madaboutlinux

    madaboutlinux Well-Known Member

    Joined:
    Jan 24, 2005
    Messages:
    1,052
    Likes Received:
    2
    Trophy Points:
    38
    Location:
    Earth
    It should not block the genuine connections as long as you do as specified in my first post.
     
  8. djblamire

    djblamire Well-Known Member

    Joined:
    May 3, 2003
    Messages:
    250
    Likes Received:
    0
    Trophy Points:
    16
    Thanks again for your reply :)

    Daniel
     
Loading...

Share This Page