What is Load Balancer?

Load Balancer evenly distributes incoming traffic/load among webservers/workers that are defined in a load-balanced set.
Users connect to the public IP of the load balancer, and web-servers are reachable over private IP from load balancer.
Ex: Ngnix, Amazon ELB
load balancer

Advantages of LB

1. Failover

If Web-Server1 goes offline, all the traffic will be routed to Web-Server2. This prevents the website from going offline.
If the website traffic grows rapidly, and two servers are not enough to handle the traffic, new web servers can be added to pool

Types of LB

Type Description
ALB/Layer-7/Application LB/Also called Reverse Proxy LB looks into Layer-7 Application packet(Eg: HTTP/HTTPS) and performs load balancing based on Header content
Layer 4 LB / Transport Layer LB Operate at transport layer(TCP, UDP, TLS). Packet is routed based on Src,dst Ports(without looking into packet).

Problem: Small Sized HTTP Response via LB
  Issue: Incoming request passes via LB and response is also sent via LB. if response is small(2MB) & if LB is serving larger sized requests, this small response has to wait unneccesarily.
  Solution= DSR (Direct Server Return Mode): backends will answer directly to the clients, without passing through LB
Layer 3 LB / NLB(Network LB) / VPN LB LB decision is made based on IP Address
IPVS(IP Virtual Server) It is a Linux kernel feature that acts as a Layer 4 load balancer. ie it's a built-in tool within the Linux kernel for load balancing

Places where LB can be kept

Place Description
Between Client(browser) and Webserver LB will send pkt to free webserver
Between App-servers and cache we can have 100s of cache servers, those needed load balancing
Between cache and DB servers

Scheduling Algorithms for LB

Name Description
1. Round Robin Send request one after other.
2. Least Connections Sends requests to the server with the lowest number of active connections
3. Least time Sends requests to the server selected by a formula that combines the fastest response time and fewest active connections
4. Hash Distributes requests based on a key you define, such as the client IP address or the request URL
5. IP HASH: (HTTP only) Distributes requests based on the first three octets of the client IP address
6. Random with 2 Choices Picks two servers at random and sends the request to the one that is selected by then applying the Least Connections algorithm
7. DNS Round Robin with Load Balancing Assign multiple IPs(IP1, IP2, IP3) to 1 domain.
When a DNS server receives a query for that domain, it responds with one of the available IP addresses in a rotating manner.
Each time an IP address is returned, it is placed at the end of the list, ensuring that all servers receive approximately equal traffic over time.
8. Sub domain DNS Delegation with Round Robin When we have multiple subdomains (hr.example.com, payroll.example.com) inside main domain(example.com). subdomains have their own nameservers. Now when request comes in for resource inside hr.example.com primary nameserver(example.com) will forward to subdomains nameservers.
9. Client side random load balancing Deliver a list of server IPs to the client, and then to have client randomly select the IP from the list on each connection. This essentially relies on all clients generating similar loads.
10. Server side load balancers Load balancer binds and listens on port. Forwards the request to all backend servers, which ever responds 1st. it caters the request.
11. IP ADDRESS based If request comes from IP Address within range from (x to y) forward packet to backend server-1, if request comes from IP Address range from (a to c) forward packet to backend server-2 and so on
12. Layer 5 Aware Look into layer-5 protocol(Eg: HTTP). They can look into HTTP-header can decide what to do with packet whether to send to server-1 or 2