Load balanced, redundant network

configuration for Linux using ECMP,

Quagga, BGP and OSPF
Consider this diagram:

• r1 and r2 are routers (possibly running FreeBSD & Quagga, as described here, or
perhaps Cisco 7204/7206)
• Each router is connected to both ISPs. Hopefully, each ISP also has each
connection handled by a unique switch and router.
• The two switches each service a unique network segment, possibly using private
IP addresses (192.168.1.0/24 and 192.168.2.0/24)
• The organisation has a block of real IP addresses which it want's external hosts to
interact with. Each internal host is assigned one of the real addresses on it's
loopback interface with a /32 netmask, so that no NAT is necessary.
 • The servers are each running Linux. The Linux kernel is compiled with Equal
Cost Multi-Path routing enabled (configuration option
CONFIG_IP_ROUTE_MULTIPATH=y). This means that the kernel will permit
multiple default gateways in the routing table, and will load balance outgoing
traffic across them.
• Routing entries, including the default gateway addresses, are not specified
manually on any of the servers or routers. Instead, Quagga is running on every
server. On the internal servers, Quagga uses OSPF to discover the router
addresses, and then creates the default routes.
• If either route becomes unavailable, because of router failure, switch failure or
network card/cable failure, or during maintenance, the OSPF protocol will detect
the failure within 5 seconds. When failure is detected, the route is automatically
deleted from the routing table, so the kernel won't continue sending packets via
that route.
• Using OSPF to detect failure is ideal, because OSPF relies on packets sent at layer
3 - the IP layer. If, for instance, the cable between the 2950 and router r2 was
removed, all other hosts connected to the 2950 would still feel that there was an
electrical connection through the switch, and would continue sending some of
their outbound packets through that switch. However, OSPF will detect this
situation correctly.

Configuration details
We assume each host is running Debian Linux (except the routers). The real IP of the host
is to be substituted where you see A.B.C.D in the examples.

Install Quagga on each host with the commands:

apt-get update
apt-get install quagga iproute

Put the following in /etc/network/interfaces:

auto lo
iface lo inet loopback
up ip addr add dev lo A.B.C.D/32 scope global

# notice that we use `manual' rather than `static', so that we can

# over-ride the scope parameter
auto eth0
iface eth0 inet manual
up ip link set dev eth0 up
up ip addr add dev eth0 192.168.1.10/24 scope link

auto eth1
iface eth1 inet manual
up ip link set dev eth1 up
up ip addr add dev eth1 192.168.2.10/24 scope link
Now put the following in /etc/quagga/zebra.conf:

hostname www1
password changeme
enable password changeme

interface lo
ip address 127.0.0.1/8
ip address A.B.C.D/32 (this is your server's real IP)

interface eth0
ip address 192.168.1.10/24
multicast

interface eth1
ip address 192.168.2.10/24
multicast

!log file /var/log/quagga/zebra.log

This is /etc/quagga/ospfd.conf

hostname www1
password changeme
enable password changeme

interface eth0
no ip ospf authentication-key
ip ospf hello-interval 2
ip ospf dead-interval 5

interface eth1
no ip ospf authentication-key
ip ospf hello-interval 2
ip ospf dead-interval 5

router ospf
ospf router-id A.B.C.D
network 192.168.1.0/24 area 0
network 192.168.2.0/24 area 0

!log file /var/log/quagga/ospfd.log

Modify /etc/quagga/daemons.conf: set zebra=yes and ospfd=yes

Testing
Once configured, reboot your host.

Type ip route and you should see a list of routes showing multiple default gateways.
Try unplugging one of the routers - then check the routing table on one of the servers.
After 5 seconds, the references to the unplugged router should be gone from the routing
tables.

Troubleshooting
• Make sure Equal Cost Multi-path is enabled in each Linux kernel
• Make sure Multicast is enabled in the kernel. Check that the network cards and
drivers support multicast - some don't
• Make sure that you don't have iptables blocking the OSPF packets. OSPF sends
packets using IP protocol 89 (not TCP or UDP). This allows OSPF:

iptables --insert INPUT -s 192.168.0.0/16 --protocol ospf -j

ACCEPT

http://www.readytechnology.co.uk/open/bgp/loadbalanced.html