The Community Forums

Interact with an entire community of cPanel & WHM users!
  1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

Guide: Spreading bandwidth across two nics - Bonding, CentOS 6.4 (Final)

Discussion in 'Workarounds and Optimization' started by Cemtey, Jul 14, 2013.

  1. Cemtey

    Cemtey Well-Known Member

    Joined:
    Aug 19, 2007
    Messages:
    69
    Likes Received:
    0
    Trophy Points:
    6
    TipsAndTricks/BondingInterfaces - CentOS Wiki

    cat /etc/redhat-release
    CentOS release 6.4 (Final)

    vi /etc/ips
    Code:
    194.204.54.42:255.255.255.248:194.204.54.47
    194.204.54.43:255.255.255.248:194.204.54.47
    194.204.54.44:255.255.255.248:194.204.54.47
    194.204.54.45:255.255.255.248:194.204.54.47
    vi /etc/resolv.conf
    Code:
    nameserver 8.8.4.4
    nameserver 8.8.8.8
    nameserver 4.2.2.1
    nameserver 4.2.2.2
    nameserver 4.2.2.3
    nameserver 4.2.2.4
    vi /etc/modprobe.d/bonding.conf
    Code:
    alias bond0 bonding
    options bond0 miimon=80 mode=6
    vi /etc/sysconfig/network-scripts/ifcfg-lo
    Code:
    DEVICE=lo
    IPADDR=127.0.0.1
    NETMASK=255.0.0.0
    NETWORK=127.0.0.0
    # If you're having problems with gated making 127.0.0.0/8 a martian,
    # you can change this to something else (255.255.255.255, for example)
    BROADCAST=127.255.255.255
    ONBOOT=yes
    NAME=loopback
    vi /etc/sysconfig/network-scripts/ifcfg-bond0
    Code:
    DEVICE=bond0
    IPADDR=194.204.54.41
    NETMASK=255.255.255.248
    NETWORK=194.204.54.40
    BROADCAST=194.204.54.47
    GATEWAY=194.204.54.46
    ONBOOT=yes
    BOOTPROTO=none
    USERCTL=no
    vi /etc/sysconfig/network-scripts/ifcfg-eth0
    Code:
    DEVICE=eth0
    ONBOOT=yes
    BOOTPROTO=none
    USERCTL=no
    MASTER=bond0
    SLAVE=yes
    vi /etc/sysconfig/network-scripts/ifcfg-eth1
    Code:
    DEVICE=eth1
    ONNBOOT=yes
    BOOTPROTO=none
    USERCTL=no
    MASTER=bond0
    SLAVE=yes
    cat /proc/net/bonding/bond0
    service network restart
    service ipaliases restart

    Following is done in WHM

    Home » Server Configuration » Basic cPanel & WHM Setup
    Pick bond0 from dropdown list.
    It should be currently only one there.
    Save Changes

    Home » Account Functions » Change Site’s IP Address
    Put main shared IP to all dedicated IPs temporarily to be able to remove them from interfaces.

    Home » IP Functions » Rebuild the IP Address Pool
    Click Proceed to mark previously taken dedicated IP addresses as Free IPs

    Home » IP Functions » Show or Delete Current IP Addresses
    You have list of IP addresses as
    Code:
    194.204.54.41 bond0
    194.204.54.42 eth0:cp1
    194.204.54.43 eth0:cp2
    194.204.54.44 eth0:cp3
    194.204.54.45 eth0:cp4
    ...
    Delete eth0:cp1, eth0:cp2.. etc because they are attached to wrong interface.

    Home » IP Functions » Add a New IP Address
    Code:
    194.204.54.40/29
    Your IP addresses here
    Code:
    255.255.255.248
    Your subnet mask here, use subnet-calculator.com if not sure.

    Go check back at
    Home » IP Functions » Show or Delete Current IP Addresses
    You shuld now have list of IP addresses as
    Code:
    194.204.54.41 bond0
    194.204.54.42 bond0:cp1
    194.204.54.43 bond0:cp2
    194.204.54.44 bond0:cp3
    194.204.54.45 bond0:cp4
    ...
    Home » IP Functions » Change a Site’s IP Address
    Change back sites dedicated IP addresses and your done.

    Just for checking reasons or reboot is rec
    service network restart
    service ipaliases restart

    Additional info on mode,
    Additional info on miimon,
     
  2. cPanelMichael

    cPanelMichael Forums Analyst
    Staff Member

    Joined:
    Apr 11, 2011
    Messages:
    30,811
    Likes Received:
    671
    Trophy Points:
    113
    cPanel Access Level:
    Root Administrator
  3. Cemtey

    Cemtey Well-Known Member

    Joined:
    Aug 19, 2007
    Messages:
    69
    Likes Received:
    0
    Trophy Points:
    6
    [SOLUTION] Troubleshooting to NTP deleting interface:

    Phenomenon: Main bonded ipaddress crash (randomly between 1-24 hours) and ipaliases stay online.

    Description: Message in logfile about NTPD deleting interface happens.

    Solution,
    Swap ETH0 to be secondary NIC and ETH1 to be main or other way around if you have currently ETH1 as main and ETH0 secondary.

    So previous configuration is correct but if such issue comes up try following which should solve the case.

    First of make sure which of NICs bond0 is using with,
    ifconfig -a bond0

    Check for MAC address and then run,
    ifconfig -a eth1
    ifconfig -a eth0

    Make note which one is currently in use.

    Code:
    grep 194.204.54.41 /var/log/messages
    Code:
    Jul 15 18:19:58 est-et ntpd[4367]: Deleting interface #5 bond0, 194.204.54.41#123, interface stats: received=220, sent=223, dropped=0, active_time=9935 secs
    Jul 15 18:20:02 est-et ntpd[4367]: Listening on interface #11 bond0, 194.204.54.41#123 Enabled
    Jul 15 18:22:25 est-et named[9584]: no longer listening on 194.204.54.41#53
    Jul 15 18:24:17 est-et named[2081]: listening on IPv4 interface bond0, 194.204.54.41#53
    Jul 15 18:24:29 est-et ntpd[4352]: Listening on interface #5 bond0, 194.204.54.41#123 Enabled
    Jul 15 18:32:09 est-et ntpd[4352]: Deleting interface #5 bond0, 194.204.54.41#123, interface stats: received=22, sent=22, dropped=1, active_time=460 secs
    Jul 15 18:32:13 est-et ntpd[4352]: Listening on interface #11 bond0, 194.204.54.41#123 Enabled
    Code:
    grep 194.204.54.41 /var/log/messages
    Code:
    Jul 15 18:19:58 est-et ntpd[4367]: Deleting interface #5 bond0, 194.204.54.41#123, interface stats: received=220, sent=223, dropped=0, active_time=9935 secs
    Jul 15 18:20:02 est-et ntpd[4367]: Listening on interface #11 bond0, 194.204.54.41#123 Enabled
    Jul 15 18:22:25 est-et named[9584]: no longer listening on 194.204.54.41#53
    Jul 15 18:24:17 est-et named[2081]: listening on IPv4 interface bond0, 194.204.54.41#53
    Jul 15 18:24:29 est-et ntpd[4352]: Listening on interface #5 bond0, 194.204.54.41#123 Enabled
    Jul 15 18:32:09 est-et ntpd[4352]: Deleting interface #5 bond0, 194.204.54.41#123, interface stats: received=22, sent=22, dropped=1, active_time=460 secs
    Jul 15 18:32:13 est-et ntpd[4352]: Listening on interface #11 bond0, 194.204.54.41#123 Enabled
    vi /etc/udev/rules.d/70-persistent-net.rules
    Code:
    # PCI device 0x8086:0x1096 (e1000e) (custom name provided by external tool)
    SUBSYSTEM=="net", ACTION=="add", DRIVERS=="?*", ATTR{address}=="00:e0:81:b3:20:7c", ATTR{type}=="1", KERNEL=="eth*", NAME="eth0"
    
    # PCI device 0x8086:0x1096 (e1000e) (custom name provided by external tool)
    SUBSYSTEM=="net", ACTION=="add", DRIVERS=="?*", ATTR{address}=="00:e0:81:b3:20:7d", ATTR{type}=="1", KERNEL=="eth*", NAME="eth1"
    
    Note: Find out what NIC you have, currently this one is (e1000e)

    vi /etc/modprobe.d/bonding.conf
    Code:
    alias bond0 bonding
    options bond0 miimon=100 mode=6
    alias eth1 e1000e
    alias eth0 e1000e
    
    Note: Type alias eth0 after alias eth1 if you had eth0 in use before.

    vi /etc/sysconfig/network-scripts/ifcfg-bond0
    Code:
    DEVICE=bond0
    IPADDR=194.204.54.41
    NETMASK=255.255.255.248
    #NETWORK=194.204.54.40
    BROADCAST=194.204.54.47
    GATEWAY=194.204.54.46
    ONBOOT=yes
    BOOTPROTO=none
    USERCTL=no
    
    vi /etc/sysconfig/network-scripts/ifcfg-eth0
    Code:
    DEVICE=eth0
    HWADDR=00:e0:81:b3:20:7c
    ONBOOT=yes
    BOOTPROTO=none
    USERCTL=no
    MASTER=bond0
    SLAVE=yes
    
    vi /etc/sysconfig/network-scripts/ifcfg-eth1
    Code:
    DEVICE=eth1
    HWADDR=00:e0:81:b3:20:7d
    ONNBOOT=yes
    BOOTPROTO=none
    USERCTL=no
    MASTER=bond0
    SLAVE=yes
    
    service network restart
    service ipaliases restart

    Just to check that bond0 now runs on eth1 instead of eth0 by default, looking at MAC addresses.
    ifconfig -a bond0
    ifconfig -a eth1
    ifconfig -a eth0
     
    #3 Cemtey, Jul 16, 2013
    Last edited: Jul 16, 2013
  4. Cemtey

    Cemtey Well-Known Member

    Joined:
    Aug 19, 2007
    Messages:
    69
    Likes Received:
    0
    Trophy Points:
    6
    Additional info:

    cat /proc/net/bonding/bond0
    Code:
    Bonding Mode: adaptive load balancing
    Primary Slave: none
    Currently Active Slave: eth0
    MII Status: up
    MII Polling Interval (ms): 100
    Up Delay (ms): 0
    Down Delay (ms): 0
    
    Slave Interface: eth0
    MII Status: up
    Speed: 100 Mbps
    Duplex: full
    Link Failure Count: 0
    Permanent HW addr: 00:e0:81:b3:20:7c
    Slave queue ID: 0
    
    Slave Interface: eth1
    MII Status: up
    Speed: 100 Mbps
    Duplex: full
    Link Failure Count: 0
    Permanent HW addr: 00:e0:81:b3:20:7d
    Slave queue ID: 0
    Following would change eth0/eth1 to be main device (temporarily, until network restart)

    cat /proc/net/bonding/bond0
    echo -eth1 > /sys/class/net/bond0/bonding/slaves
    echo +eth1 > /sys/class/net/bond0/bonding/slaves

    echo -eth0 > /sys/class/net/bond0/bonding/slaves
    echo +eth0 > /sys/class/net/bond0/bonding/slaves


    Following command will change bond0 MAC address without having to reboot
    ifconfig bond0 hw ether 00:e0:81:b3:20:7d

    Just markup what eth0 and eth1 I have as MAC addresses
    Code:
    HWADDR=00:e0:81:b3:20:7c
    HWADDR=00:e0:81:b3:20:7d
    ifconfig -a bond0
    ifconfig -a eth0
    ifconfig -a eth1


    Testing, but as result it is not firm solution.
    service network restart
    service ipaliases restart


    Adding primary value into bond0 configuration
    /etc/sysconfig/network-scripts/ifcfg-bond0
    Code:
    GATEWAY
    BONDING_OPTS="mode=6 miimon=100 primary=eth1"
    ONBOOT
    ifconfig bond0 hw ether 00:e0:81:b3:20:7d
    service network restart
    service ipaliases restart


    Checking if it works
    cat /proc/net/bonding/bond0
    Code:
    Bonding Mode: adaptive load balancing
    Primary Slave: eth1 (primary_reselect always)
    Currently Active Slave: eth1
    MII Status: up
    MII Polling Interval (ms): 100
    Up Delay (ms): 0
    Down Delay (ms): 0
    
    Slave Interface: eth0
    MII Status: up
    Speed: 100 Mbps
    Duplex: full
    Link Failure Count: 0
    Permanent HW addr: 00:e0:81:b3:20:7c
    Slave queue ID: 0
    
    Slave Interface: eth1
    MII Status: up
    Speed: 100 Mbps
    Duplex: full
    Link Failure Count: 0
    Permanent HW addr: 00:e0:81:b3:20:7d
    Slave queue ID: 0
    
    ifconfig -a bond0
    ifconfig -a eth0
    ifconfig -a eth1


    Second test,
    service network restart
    service ipaliases restart


    Perfect!
     
    #4 Cemtey, Jul 27, 2013
    Last edited: Jul 27, 2013

Share This Page