Load Balancing

Distributing work across multiple servers to enable scalability, reliability, and optimal resource utilization.

OVERVIEW

The Concept

Load balancing distributes incoming traffic and computational load across multiple servers to optimize resource utilization, maximize throughput, minimize response time, and avoid overload on any single server.

Without it, performance degrades

Without load balancing, all requests go to a single server. When traffic spikes, that server becomes overloaded, responses slow down, and users experience timeouts and failures.

Where it's critical

  • Web applications with millions of concurrent users
  • APIs serving multiple client applications
  • Microservices architectures
  • Database query distribution
  • Real-time streaming platforms
Distribute the work fairly so no single server bears all the burden.
ALGORITHMS

Load Balancing Strategies

Round Robin

Requests are distributed sequentially to each server in rotation. Simple but assumes all servers have equal capacity.

Least Connections

Directs new requests to the server handling the fewest active connections. Better for long-lived connections.

Weighted Round Robin

Assigns different weights to servers based on capacity. More powerful servers get more requests.

IP Hash

Routes requests based on client IP address. Ensures a client always connects to the same server (session persistence).

Random

Randomly selects a server for each request. Stateless and works well when servers are homogeneous.

TERMINOLOGY

Key Concepts

Design considerations

  • Single point of failure: load balancer itself must be redundant
  • State management: servers should be stateless when possible
  • Latency overhead: load balancer adds minimal but measurable delay
  • Sticky sessions: needed for stateful applications
  • Cost: load balancers add infrastructure expense
Good load balancing is invisible to users—requests flow smoothly regardless of server load.
DESIGN

Trade-offs

Benefits

  • Improved availability through redundancy
  • Better resource utilization across servers
  • Increased throughput and response time
  • Transparent failover to healthy servers
  • Easy horizontal scaling

Challenges

  • Adds complexity to system architecture
  • Load balancer itself becomes critical infrastructure
  • Session state management complexity
  • Network latency from extra hop
  • Requires monitoring and maintenance

Common Patterns

  • HAProxy or NGINX for application load balancing
  • Cloud providers (AWS ELB, Google Cloud LB, Azure LB)
  • DNS-based load balancing for geographic distribution
  • Layer 4 (transport) vs Layer 7 (application) balancing
  • Redundant load balancers with failover
APPLICATIONS

Real-World Examples

E-Commerce Platform

During sales, load balancer distributes traffic across dozens of application servers to handle peak load.

Social Media Feed

Load balancer routes millions of concurrent feed requests across geographically distributed servers.

Microservices API Gateway

Incoming API requests are load balanced across multiple gateway instances and then to backend services.

Interactive Simulation

Load Balancing Live Lab

Adjust traffic, server count, and balancing strategy to see how request distribution, throughput, and queue pressure change in real time.

Requests / tick

24

Throughput

0

Success rate

100.0%

Failed serverNone

Current utilization

0%

Capacity44 / tick

S1

weight 1 • online

0 assigned

0 queued

S2

weight 2 • online

0 assigned

0 queued

S3

weight 2 • online

0 assigned

0 queued

S4

weight 3 • online

0 assigned

0 queued

Average latency

0 ms

Queue pressure and server capacity drive delays.

Queue backlog

0

Requests waiting because servers are at capacity.

Total dropped

0

Requests dropped when no healthy server could accept them.

<Ameh/>