Emil CHERICHES

Emil CHERICHES

High Availability Load Balancer

August 12, 2015 Emil C
1 Comment

Suppose you really need a highly available service. Normally you would put a load balancer in front of a server farm. But what happens when the load balancer fails? Well, we could have to load balancers, one which is actually active and the second one kicking in when first one fails.

We could have two servers or two VMs that are in the same VLAN, they both have their own IP, they are able to communicate/broadcast to each other and there’s a third IP that is shared between the two servers/VMs. One of them is MASTER the other one is BACKUP.

For a test setup I used CentOS 7, you could use your own favorite distro. I’ve set-up on my test machines with 192.168.6.110/24 and 192.168.6.120/24 as the machine’s IPs. I’m going to set two clustered/highly available IPs 192.168.7.100/24 and 192.168.7.110/24. Notice they’re from different classes, machine’s IPs could be anything as long as the two machines can communicate with each other. The default gateway on the two machines should be the one for the clustered IP.

I first installed some basic tools, on both machines

yum install net-tools wget vim mtr

Then, on both I installed the daemon that handles the IP sharing/changing.

yum install keepalived

You could use any software you prefer as a load balancer, it could either be nginx, haproxy, pound etc. In order for your preferred load balancer software to be able to bind/listen to the shared IP you need some kernel configuration. For this we add to /etc/sysctl.conf the folowing line

net.ipv4.ip_nonlocal_bind=1

then “sysctl -p”  will activate the setting.

For the demonstration sake I just stopped and disabled firewalld, though, for production machines you should consider implementing a firewall (honestly I prefer the classic iptables-services for CentOS 7).

Then we need to configure the keepalived daemon. On the first machine “/etc/keepalived/keepalived.conf” file looks like this:

global_defs {
   router_id LVS_DEVEL
}

vrrp_instance VI_1 {
    state MASTER
    interface eth0
    virtual_router_id 52
    priority 100
    advert_int 1
    authentication {
        auth_type PASS
        auth_pass 1111
    }
    virtual_ipaddress {
        192.168.7.100/24
        192.168.7.101/24
    }

    virtual_routes {
        default via 192.168.7.1 dev eth0
        }
}

on the second:

global_defs {
   router_id LVS_DEVEL
}

vrrp_instance VI_1 {
    state BACKUP
    interface eth0
    virtual_router_id 52
    priority 99
    advert_int 1
    authentication {
        auth_type PASS
        auth_pass 1111
    }
    virtual_ipaddress {
        192.168.7.100/24
        192.168.7.101/24
    }

    virtual_routes {
        default via 192.168.7.1 dev eth0
        }
}

then on both machines we need to enable and start the daemon:

systemctl enable keepalived.service
systemctl start keepalived.service

Now we can ping one of the clustered IP, it’s alive.

To further check ip addr show would like this on the first machine:

2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
    link/ether 52:54:00:62:b1:58 brd ff:ff:ff:ff:ff:ff
    inet 192.168.6.110/24 brd 192.168.6.255 scope global eth0
       valid_lft forever preferred_lft forever
    inet 192.168.7.100/24 scope global eth0
       valid_lft forever preferred_lft forever
    inet 192.168.7.101/24 scope global secondary eth0
       valid_lft forever preferred_lft forever
    inet6 fe80::5054:ff:fe62:b158/64 scope link
       valid_lft forever preferred_lft forever

and like this, on the second one:

2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
    link/ether 52:54:00:02:88:bd brd ff:ff:ff:ff:ff:ff
    inet 192.168.6.120/24 brd 192.168.6.255 scope global eth0
       valid_lft forever preferred_lft forever
    inet6 fe80::5054:ff:fe02:88bd/64 scope link
       valid_lft forever preferred_lft forever

Then ip route show would look like this, on the active node:

default via 192.168.7.1 dev eth0  proto static  metric 100
192.168.6.0/24 dev eth0  proto kernel  scope link  src 192.168.6.110  metric 100
192.168.7.0/24 dev eth0  proto kernel  scope link  src 192.168.7.100

and like this on the second one:

192.168.6.0/24 dev eth0  proto kernel  scope link  src 192.168.6.120  metric 100

a tail of /var/log/messages would look like this:

Aug 12 21:17:41 KA1 systemd: Started LVS and VRRP High Availability Monitor.
Aug 12 21:17:41 KA1 Keepalived_vrrp[2010]: Netlink reflector reports IP 192.168.6.110 added
Aug 12 21:17:41 KA1 Keepalived_vrrp[2010]: Netlink reflector reports IP fe80::5054:ff:fe62:b158 added
Aug 12 21:17:41 KA1 Keepalived_healthcheckers[2009]: Netlink reflector reports IP 192.168.6.110 added
Aug 12 21:17:41 KA1 Keepalived_vrrp[2010]: Registering Kernel netlink reflector
Aug 12 21:17:41 KA1 Keepalived_healthcheckers[2009]: Netlink reflector reports IP fe80::5054:ff:fe62:b158 added
Aug 12 21:17:41 KA1 Keepalived_healthcheckers[2009]: Registering Kernel netlink reflector
Aug 12 21:17:41 KA1 Keepalived_vrrp[2010]: Registering Kernel netlink command channel
Aug 12 21:17:41 KA1 Keepalived_vrrp[2010]: Registering gratuitous ARP shared channel
Aug 12 21:17:41 KA1 Keepalived_vrrp[2010]: Opening file '/etc/keepalived/keepalived.conf'.
Aug 12 21:17:41 KA1 Keepalived_vrrp[2010]: Configuration is using : 62972 Bytes
Aug 12 21:17:41 KA1 Keepalived_vrrp[2010]: Using LinkWatch kernel netlink reflector...
Aug 12 21:17:41 KA1 Keepalived_vrrp[2010]: VRRP sockpool: [ifindex(2), proto(112), unicast(0), fd(10,11)]
Aug 12 21:17:41 KA1 Keepalived_healthcheckers[2009]: Registering Kernel netlink command channel
Aug 12 21:17:41 KA1 Keepalived_healthcheckers[2009]: Opening file '/etc/keepalived/keepalived.conf'.
Aug 12 21:17:41 KA1 Keepalived_healthcheckers[2009]: Configuration is using : 6203 Bytes
Aug 12 21:17:41 KA1 Keepalived_healthcheckers[2009]: Using LinkWatch kernel netlink reflector...
Aug 12 21:17:42 KA1 Keepalived_vrrp[2010]: VRRP_Instance(VI_1) Transition to MASTER STATE
Aug 12 21:17:42 KA1 Keepalived_vrrp[2010]: VRRP_Instance(VI_1) Received lower prio advert, forcing new election
Aug 12 21:17:43 KA1 Keepalived_vrrp[2010]: VRRP_Instance(VI_1) Entering MASTER STATE
Aug 12 21:17:43 KA1 Keepalived_vrrp[2010]: VRRP_Instance(VI_1) setting protocol VIPs.
Aug 12 21:17:43 KA1 Keepalived_vrrp[2010]: VRRP_Instance(VI_1) setting protocol Virtual Routes
Aug 12 21:17:43 KA1 Keepalived_vrrp[2010]: VRRP_Instance(VI_1) Sending gratuitous ARPs on eth0 for 192.168.7.100
Aug 12 21:17:43 KA1 Keepalived_vrrp[2010]: VRRP_Instance(VI_1) Sending gratuitous ARPs on eth0 for 192.168.7.101
Aug 12 21:17:43 KA1 Keepalived_healthcheckers[2009]: Netlink reflector reports IP 192.168.7.100 added
Aug 12 21:17:43 KA1 Keepalived_healthcheckers[2009]: Netlink reflector reports IP 192.168.7.101 added
Aug 12 21:17:43 KA1 NetworkManager[436]: <info>  Policy set 'eth0' (eth0) as default for IPv4 routing and DNS.
Aug 12 21:17:48 KA1 Keepalived_vrrp[2010]: VRRP_Instance(VI_1) Sending gratuitous ARPs on eth0 for 192.168.7.100
Aug 12 21:17:48 KA1 Keepalived_vrrp[2010]: VRRP_Instance(VI_1) Sending gratuitous ARPs on eth0 for 192.168.7.101

There you have it, not that hard 🙂

If you switch off the first machine you’ll notice that the clustered IP will be raised automatically by the second node. When the first machine comes back online it goes back to it.

linux ClusterHigh AvailabilityVRRP
Previous Post

Error: Can't open display: localhost:10.0

Next Post

Debconf 15

  1. Pingback: High Availability Load Balancer | Emil’s bits | LUG Mureş
Leave a Reply Cancel reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.

Recent Posts
  • Flash Tasmota on the Sonoff Mini – Linux user experience
  • DokuWiki a simple solution to sorting documentation
  • I discovered Wireguard
  • Using Let’s Encrypt with CentOS (for now)
  • Let’s Encrypt is Trusted
Categories
  • from the web
  • just blog
  • linux
    • Mail Server Series
  • News
  • phones
  • Phones & Tablets
  • programing
  • security
  • Smart Home
  • Tips & Tricks
  • Uncategorized
  • Web development
  • Windows
Blogroll
  • cheriches.fr
Subscribe by Email
Recent Posts
  • Flash Tasmota on the Sonoff Mini – Linux user experience
  • DokuWiki a simple solution to sorting documentation
  • I discovered Wireguard
  • Using Let’s Encrypt with CentOS (for now)
  • Let’s Encrypt is Trusted
Categories
  • from the web (3)
  • just blog (1)
  • linux (23)
    • Mail Server Series (7)
  • News (1)
  • phones (1)
  • Phones & Tablets (2)
  • programing (1)
  • security (2)
  • Smart Home (1)
  • Tips & Tricks (15)
  • Uncategorized (3)
  • Web development (1)
  • Windows (1)
Blogroll
  • cheriches.fr
Tags cloud
adb ADS aircrack-ng Android Apache apt-get CentOS Chrome ClamAV Cluster CSS Debconf debian DKIM Dovecot EasyRSA EPEL Firefox Firewall GNOME High Availability iptables KVM linux MariaDB Milter MySQL Nginx php Postfix Postfix Admin Proxy_ARP Python release Roundcube Samba SELinux Sonoff SpamAssassin ssh Tasmota ubuntu VRRP windows Youtube
Recent Comments
  • Frank on Dedicated Mail Server Hosting Multiple Domains – Step 3 – Postfix and Dovecot
  • Andrew Schott on Dedicated Mail Server Hosting Multiple Domains – Step 3 – Postfix and Dovecot
  • Emil C on Dedicated Mail Server Hosting Multiple Domains – Step 3 – Postfix and Dovecot
  • Andrew Schott on Dedicated Mail Server Hosting Multiple Domains – Step 3 – Postfix and Dovecot
  • Emil C on Dedicated Mail Server Hosting Multiple Domains – Step 3 – Postfix and Dovecot
Proudly powered by WordPress | Theme: Fmi by Forrss.
Manage Cookie Consent
We use cookies to optimize our website and our service.
Functional cookies Always active
The technical storage or access is strictly necessary for the legitimate purpose of enabling the use of a specific service explicitly requested by the subscriber or user, or for the sole purpose of carrying out the transmission of a communication over an electronic communications network.
Preferences
The technical storage or access is necessary for the legitimate purpose of storing preferences that are not requested by the subscriber or user.
Statistics
The technical storage or access that is used exclusively for statistical purposes. The technical storage or access that is used exclusively for anonymous statistical purposes. Without a subpoena, voluntary compliance on the part of your Internet Service Provider, or additional records from a third party, information stored or retrieved for this purpose alone cannot usually be used to identify you.
Marketing
The technical storage or access is required to create user profiles to send advertising, or to track the user on a website or across several websites for similar marketing purposes.
Manage options Manage services Manage vendors Read more about these purposes
View preferences
{title} {title} {title}