Linux LVS Load balance on Fedora 5: My experience
Reference: http://www.austintek.com/LVS/LVS-HOWTO/
Before doing LVS, readers are required to read the HowTo and MiniHowTo of this site. This site explains the basic and the way load balance work.
My architecture is as follow:
Router(192.168.1.1)
|---------------------|----------------------------|
| | |
Client Load balancer (LB) Real Server (RS)
192.168.1.5 eth0/DIP:192.168.1.2 192.168.1.4
VIP:192.168.1.3
Client - Windows machine
Load balancer/Real Server - Fedora 5 - Kernel 2.6.18
Points to note:
LB - I use 1 network card only, i.e. I hosted 2 IPs on 1 network card.
I used the basic configuration script created by LVS site:
http://www.linuxvirtualserver.org/software/index.html
For those who don't know:
DIP = Director IP: Director can be seen as load balancer, this is the IP of this load balancer.
VIP = Virtual IP: This is the IP which can be seen as an "access point" to outside world. Client are only interested in and made requests to this IP.
Real server (RS): Server(s) which process the requests from LB.
This is a LVS-NAT architecture.
Process start:
1) Enable modprobe on LB:
modprobe ip_vs_dh
modprobe ip_vs_ftp
modprobe ip_vs
modprobe ip_vs_lblc
modprobe ip_vs_lblcr
modprobe ip_vs_lc
modprobe ip_vs_nq
modprobe ip_vs_rr
modprobe ip_vs_sed
modprobe ip_vs_sh
modprobe ip_vs_wlc
modprobe ip_vs_wrr
modprobe is in /sbin, so to make them available when system is booted, add the above command in /etc/rc.local or create script and put it under /etc/init.d.
2. Install ipvsadm: This is probably the most important step. This is an application which maintains how packets/traffics are collected and forwarded. To install this, run the following:
~#> yum -y install ipvsadm
3. Download the configuration script of LVS from the site below:
http://www.austintek.com/LVS/
As the time of this writing, the version is 0.9.4.
4. Untar and cd to the extracted folder, find a file called lvs_nat_one_NIC_one_network and edit this file:
#----------lvs_nat.conf------------------------------------
LVSCONF_FORMAT=1.1
LVS_TYPE=VS_NAT
INITIAL_STATE=on
CLEAR_IPVS_TABLES=yes
#
#VIP line format - device[:alias] IP netmask broadcast
#To help avoid namespace collisions with other VIPs, I set alias=last number of VIP (here 110).
VIP=eth0:0 192.168.1.3 255.255.255.0 192.168.1.255
#
#DIP line format - device[:alias] IP network netmask broadcast
DIP=eth0 192.168.1.1 192.168.1.0 255.255.255.0 192.168.1.255
#
#DIRECTOR_GW - packets with src_addr=VIP, dst_addr=0/0 are sent to DIRECTOR_GW
#to be forwarded to the outside world.
#The script will not neccesarily set up the DIRECTOR_GW as the director's default gw.
DIRECTOR_GW=192.168.1.1 #<==This is probably the most interesting part, I set it to my router
#
#SERVICE line format - proto port scheduler IP:port[,weight] [IP:port[,weight]]
#SERVICE=t ftp rr RS1:ftp,1 RS2:ftp,2
#SERVICE=t telnet rr RS1:telnet RS2:telnet
SERVICE=t 80 wrr 192.168.1.4:80
#
SERVER_NET_DEVICE=eth0
#VS-NAT real-servers do not have a VIP, i.e. there is no SERVER_VIP_DEVICE
#SERVER_VIP_DEVICE=
#SERVER_GW is not user configurable with VS-NAT. script sets SERVER_GW = DIP
#SERVER_GW=
#----------end lvs_nat.conf---------------------------------
5. On real server, add a routing table rules:
~#> route add -net 192.168.1.0 netmask 255.255.255.0 gw 192.168.1.2
This step is the second importance in the whole setup process. According to the LVS-mini-HOWTO in linux-vs.org
7.6 The client says "connection refused"
7.7. connection hangs; ipvsadm shows entries in InActConn, but none in ActiveConn
The connection is refused because:
1. You are setting the gateway of your RS incorrectly.
or
1. When client first connect to VIP, VIP will reply and forward the request to RS
2. RS replied DIRECTLY TO CLIENT, BUT NOT PASS THROUGH THE LOAD BALANCER.
The important point is that, RS SHOULD NOT REPLY THE MESSAGE DIRECTLY TO CLIENT, BUT SHOULD GO BACK THE MESSAGE TO LB FIRST, then LB will forward the reply from RS to client! Otherwise, client will just have the "connection refused" error, or timeout error in browser.
Hope somebody can find this helpful
Before doing LVS, readers are required to read the HowTo and MiniHowTo of this site. This site explains the basic and the way load balance work.
My architecture is as follow:
Router(192.168.1.1)
|---------------------|----------------------------|
| | |
Client Load balancer (LB) Real Server (RS)
192.168.1.5 eth0/DIP:192.168.1.2 192.168.1.4
VIP:192.168.1.3
Client - Windows machine
Load balancer/Real Server - Fedora 5 - Kernel 2.6.18
Points to note:
LB - I use 1 network card only, i.e. I hosted 2 IPs on 1 network card.
I used the basic configuration script created by LVS site:
http://www.linuxvirtualserver.org/software/index.html
For those who don't know:
DIP = Director IP: Director can be seen as load balancer, this is the IP of this load balancer.
VIP = Virtual IP: This is the IP which can be seen as an "access point" to outside world. Client are only interested in and made requests to this IP.
Real server (RS): Server(s) which process the requests from LB.
This is a LVS-NAT architecture.
Process start:
1) Enable modprobe on LB:
modprobe ip_vs_dh
modprobe ip_vs_ftp
modprobe ip_vs
modprobe ip_vs_lblc
modprobe ip_vs_lblcr
modprobe ip_vs_lc
modprobe ip_vs_nq
modprobe ip_vs_rr
modprobe ip_vs_sed
modprobe ip_vs_sh
modprobe ip_vs_wlc
modprobe ip_vs_wrr
modprobe is in /sbin, so to make them available when system is booted, add the above command in /etc/rc.local or create script and put it under /etc/init.d.
2. Install ipvsadm: This is probably the most important step. This is an application which maintains how packets/traffics are collected and forwarded. To install this, run the following:
~#> yum -y install ipvsadm
3. Download the configuration script of LVS from the site below:
http://www.austintek.com/LVS/
As the time of this writing, the version is 0.9.4.
4. Untar and cd to the extracted folder, find a file called lvs_nat_one_NIC_one_network and edit this file:
#----------lvs_nat.conf------------------------------------
LVSCONF_FORMAT=1.1
LVS_TYPE=VS_NAT
INITIAL_STATE=on
CLEAR_IPVS_TABLES=yes
#
#VIP line format - device[:alias] IP netmask broadcast
#To help avoid namespace collisions with other VIPs, I set alias=last number of VIP (here 110).
VIP=eth0:0 192.168.1.3 255.255.255.0 192.168.1.255
#
#DIP line format - device[:alias] IP network netmask broadcast
DIP=eth0 192.168.1.1 192.168.1.0 255.255.255.0 192.168.1.255
#
#DIRECTOR_GW - packets with src_addr=VIP, dst_addr=0/0 are sent to DIRECTOR_GW
#to be forwarded to the outside world.
#The script will not neccesarily set up the DIRECTOR_GW as the director's default gw.
DIRECTOR_GW=192.168.1.1 #<==This is probably the most interesting part, I set it to my router
#
#SERVICE line format - proto port scheduler IP:port[,weight] [IP:port[,weight]]
#SERVICE=t ftp rr RS1:ftp,1 RS2:ftp,2
#SERVICE=t telnet rr RS1:telnet RS2:telnet
SERVICE=t 80 wrr 192.168.1.4:80
#
SERVER_NET_DEVICE=eth0
#VS-NAT real-servers do not have a VIP, i.e. there is no SERVER_VIP_DEVICE
#SERVER_VIP_DEVICE=
#SERVER_GW is not user configurable with VS-NAT. script sets SERVER_GW = DIP
#SERVER_GW=
#----------end lvs_nat.conf---------------------------------
5. On real server, add a routing table rules:
~#> route add -net 192.168.1.0 netmask 255.255.255.0 gw 192.168.1.2
This step is the second importance in the whole setup process. According to the LVS-mini-HOWTO in linux-vs.org
7.6 The client says "connection refused"
7.7. connection hangs; ipvsadm shows entries in InActConn, but none in ActiveConn
The connection is refused because:
1. You are setting the gateway of your RS incorrectly.
or
1. When client first connect to VIP, VIP will reply and forward the request to RS
2. RS replied DIRECTLY TO CLIENT, BUT NOT PASS THROUGH THE LOAD BALANCER.
The important point is that, RS SHOULD NOT REPLY THE MESSAGE DIRECTLY TO CLIENT, BUT SHOULD GO BACK THE MESSAGE TO LB FIRST, then LB will forward the reply from RS to client! Otherwise, client will just have the "connection refused" error, or timeout error in browser.
Hope somebody can find this helpful
Comments