/etc/modprobe.d/bonding is run at boot time before /etc/network/interfaces.
/etc/network/interfaces stores the network configuration.
Active-Backup Mode: The server uses only one link, the other link still receives broadcasts etc, but never transmits a frame and so the switch CAM table will show no MAC address on the backup link's port. As both links use the same MAC this is a good thing, so no switch config required..
Active-Active Mode: The server uses both links (flow based agaggregation, link hashing policy can be changed). This requires both the server and the switch to negotiate a logical trunk bonding the links together so they are both active and share the same physical MAC address. This is done using the LACP (802.3ad) protocol.
Switch LACP Configuration;
Cisco
default interface range gigabitethernet 1 - 2
interface range gigabitethernet 1 - 2
channel-group 1 mode active
interface port-channel 1
switchport mode access
switchport access vlan 10
switchport mode trunk
switchport trunk allowed vlan 50,51,52
switchport trunk native vlan 50
The black and orange commands are for connecting to a normal server (with only a single L2 connection to VLAN 10).
The black and blue commands are used if the server is a virtual hypervisor for example, and the link needs to carry multiple VLANs to the server. This could also be another switch.
HP
interface <x>
trunk 1-2 trk1 lacp
vlan 10 untagged trk1
vlan 50 untagged trk1
vlan 51 tagged trk1
vlan 51 tagged trk1
Juniper
interfaces {
ge-0/0/0 {
ether-options {
802.3ad ae1;
}
}
ge-0/0/1 {
ether-options {
802.3ad ae1;
}
}
ae1 {
description "LAG Interface";
native-vlan-id 201;
aggregated-ether-options {
lacp {
active;
periodic slow;
}
}
unit 0 {
family ethernet-switching {
interface-mode trunk;
vlan {
members all;
}
}
}
}
(Section to be finished..)
Active - Backup Bonded Interfaces, with VLAN tagging
Set /etc/modprobe.d/bonding so it looks like;
alias netdev-bond0 bonding
options bonding primary_reselect=1 num_grat_arp=5 mode=active-backup miimon=100
This /etc/modprobe.d/bonding.conf ensures the "bonding" driver is loaded at boot where;
primary_reselect=1 (use physical link with better speed and duplex)
num_grat_arp=25 (number of Gratutitous ARPs sent after a link change - ensures the top of rack switch updates its CAM table with the new port for the server's MAC)
mode=active-backup (mode is immutable)
miimon=100 (check the Ethernet carrier on the link is still alive ever 100ms).
Set /etc/network/interfaces so it looks like;
auto lo
iface lo inet loopback
allow-hotplug eth0 # Run eth0 stanza every event change(link up etc)
auto eth0 # Run eth0 stanza on boot
allow-bond0 eth0 # Allow bond0 events to control (More Info needed on this)
iface eth0 inet manual # Begin eth0 stanza
bond-master bond0 # Register eth0 in the slaves list for bond0
allow-hotplug eth1
auto eth1
allow-bond0 eth1
iface eth1 inet manual
bond-master bond0
auto bond0 # Run bond0 stanza on boot
iface bond0 inet manual # Begin bond0 stanza
bond-mode active-backup # Mode
bond-miimon 100 # Check link carrier every 100ms
bond-num_grat_arp 5 # Send 5 GARPs after link change
bond-downdelay 200 # Wait 200ms after slave destroyed/de-registered
bond-updelay 200 # Wait 200ms after slave up (driver stable)
bond-slaves none # DO NOT take Control/Start PHYs (they are ready)
use_carrier 1 # Force to use new Mii detect
primary_reselect 1 # Use Physicals interface with best speed/duplex
auto bond0.16 # Explicit bond0 subinterface (no decouple)
iface bond0.16 inet static # Begin bond0.16 stanza
address 10.0.16.101 # IP Address
netmask 255.255.255.0 # Subnet Mask
network 10.0.16.0 # Network ID
broadcast 10.0.16.255 # Broadcast
gateway 10.0.16.1 # default gateway (0.0.0.0/0)
vlan-raw-device bond0 # Attach to bond0 (not needed with .16 notation, but safer to ensure binding)
Notes;
The design of this config is; the physical NICs register themselves as available slaves on their own, and each time a cable is connected they will re-register themselves as available to the bond. The bond then starts, but does not try to add any more slaves due to "bond-slaves none".
The alternative common design is to not define the physical interfaces at all, and define them under the bond with "bond-slaves eth0 eth1". However sometimes the physical interfaces can detatch from the bond's slave list, and without defining the hotplug property on the physical interface, they will never be re-enslaved by the bond.
If you set "bond-slaves eth0 eth1" as well as defining the physicals as above, during boot, the physical interfaces will come up, and then when the bond0 is loaded, the physicals will go down again and then come back up again (without their previous properties)!
The modprobe.d/bonding.conf file is mandatory with this design, it is optional if defining the salves under the bond0. This is simply that the bonding driver must already be loaded by modprobe.d/bonding.conf before the physical interfaces are processed, else an error will be thrown as "bond-master bond0" will not be recognised.
The bonding driver is only loaded automatically by the bond0 stanza, however in this improved design we want to register the physical interfazces with extra redundancy paraameters before the bond0 is loaded.
As a rule of thumb, set 'manual' for interfaces which do not hold an IP, and 'static' for those who do. Or 'dhcp' for interfaces which should run dhclient
Active - Backup Bonded Interfaces, with VLANs and Bridges
Active - Active Bonded Interfaces, with Bridges
Active - Active Bonded Interfaces, with VLANs and Bridges
(to be finsihed)
