A website may appear to run on one address, but the server handling a visitor’s request does not have to be the same machine. As traffic increases, sending every request to a single server can create a bottleneck. That server may run out of CPU, memory, available connections, or other resources while other servers sit unused.
Load balancing addresses this problem by distributing incoming network requests across multiple servers. The basic idea is straightforward: instead of asking one server to handle everything, a load balancer helps spread the work among several available servers. This can improve how an application handles traffic and can make it easier to keep a service available when one server has a problem.
However, load balancing is more than simply sending requests to different machines. The method used to select a server, the health of each server, the type of application, and whether a user’s session needs to remain on a particular server can all affect the design.
What Is Load Balancing?
Load balancing is the process of distributing incoming requests or network connections across multiple servers or other backend resources.
A typical setup might look like this:
Visitors → Load Balancer → Server A, Server B, Server C
The visitor generally connects to the public address of the application. The load balancer receives the request and decides which backend server should process it.
For example, imagine an online store running on three application servers. If hundreds of customers are browsing products at the same time, the load balancer can distribute their requests rather than directing every request to one machine.
The servers can then share the processing work. This arrangement is sometimes called a server pool, backend pool, or server cluster, depending on the technology and architecture involved.
Why Send Traffic to Multiple Servers?
A single server has finite resources. Adding more servers does not automatically make an application faster, but it can provide additional capacity when the application is designed to use them. Consider a simple example.
Suppose an application normally receives 300 requests per second. One server may be capable of handling that workload comfortably. If traffic grows substantially, that same server may begin approaching its CPU, memory, connection, or application limits.
- An organisation could replace it with a much larger machine, which is known as vertical scaling.
- Another approach is to add additional application servers and distribute requests between them. This method is horizontal scaling.
Load balancing is commonly used with horizontal scaling because something needs to decide where incoming requests should go.
Load balancing can help with several architectural goals.
A properly designed system can use load balancing to:
- distribute work across multiple servers
- add capacity by introducing additional servers
- detect servers that are unavailable
- reduce dependence on one application server
- support maintenance with less disruption
- handle changing traffic levels
- provide a single entry point for multiple backend systems
These benefits depend on the complete architecture. A load balancer cannot fix an application that has a database bottleneck, poorly designed code, or insufficient capacity elsewhere.
How a Load Balancer Chooses a Server
The load balancer needs a method for deciding where each incoming request should go. Different algorithms are useful in different situations.
Round Robin
With round robin, requests are distributed through the available servers in sequence.
For example:
- Request 1 → Server A
- Request 2 → Server B
- Request 3 → Server C
- Request 4 → Server A
This approach is simple and works well when the servers have similar capabilities and requests generally require similar amounts of work. It can become less suitable when one request takes a fraction of a second while another takes several seconds to complete.
Weighted Round Robin
A weighted version allocates some servers a larger share of traffic. For example, if Server A has considerably more processing capacity than Server B, the configuration can assign different weights to them. The stronger server can therefore receive more traffic while the smaller server receives less. This can be useful when backend machines are not identical.
Least Connections
The least-connections method considers how many active connections each server currently has. A new connection can be directed toward a server with fewer active connections. This can be useful when connections vary significantly in duration. A server handling several long-lived connections may have a very different workload from one handling many short requests.
Other Selection Methods
Some systems use additional information when choosing a backend, such as response time, server health, client information, or application-specific rules. The right method depends on the workload rather than on a universal rule that applies to every application.
Health Checks Prevent Traffic From Going to Failed Servers
One of the most useful functions of many load balancers is health checking.
Imagine that three application servers are available:
- Server A is working normally.
- Server B has stopped responding.
- Server C is working normally.
If the load balancer continues sending traffic to Server B, users may receive errors even though other servers are available. A health check gives the load balancer a way to determine whether a backend is responding as expected. A simple health check might request a specific endpoint from an application. More advanced systems can check conditions relevant to the application. When a server fails its health checks, the load balancer can temporarily remove it from the pool. Once the server becomes healthy again, it may be returned to service depending on the configuration.
This is one reason load balancing is closely associated with high availability. The load balancer itself does not make an application immune to failures, but it can help route traffic away from unhealthy backend servers.
A Simple Example: A Busy Web Application
Consider a news website that initially runs on one application server. During normal periods, the server handles requests without difficulty. When a major event causes a sudden increase in visitors, CPU usage and active connections rise. The organisation adds two more application servers.
The new arrangement becomes:
Visitors → Load Balancer → Application Server 1
** → Application Server 2**
** → Application Server 3**
The public-facing application address can remain the same while the backend capacity changes. The load balancer distributes incoming requests among the available servers. If Application Server 2 becomes unhealthy, the load balancer can stop directing new traffic to it while the other servers continue handling requests. This does not mean users will be unaware of a failure. Existing connections, application state, databases, caches, and other dependencies can still cause problems. Load balancing solves one part of the availability problem rather than the entire problem.
Load Balancing Does Not Mean Every Request Takes the Same Path
It is tempting to imagine that a visitor connects to Server A and stays there permanently. In many architectures, that is not necessary. Different requests from the same user can potentially reach different backend servers.
For example:
- Request 1 → Server A
- Request 2 → Server C
- Request 3 → Server B
This works particularly well when application servers are designed to be stateless. A stateless application does not depend on one particular server remembering everything about a user’s previous requests. Important session information can instead be stored in shared infrastructure such as a database or dedicated session store, depending on the application design.
What about user sessions?
Some applications do need requests from a user to return to the same server. A technique known as session persistence or sticky sessions can sometimes be used for this purpose.
However, sticky sessions introduce trade-offs. They can make traffic distribution less flexible because a user may remain associated with one backend server even when another server has more available capacity. For applications that can avoid server-local session state, keeping backend servers interchangeable simplifies scaling and failure handling.
Layer 4 and Layer 7 Load Balancing
Load balancing can operate at different layers of a network architecture.
Layer 4 Load Balancing
Layer 4 load balancing works with transport-level information, commonly involving protocols such as TCP or UDP. The load balancer can make decisions using information such as network addresses and ports without necessarily understanding the details of the application request. This can be useful when high-performance connection distribution is the primary requirement.
Layer 7 Load Balancing
Layer 7 load balancing operates at the application layer. For web traffic, this approach can allow decisions based on information such as the requested hostname, URL path, HTTP headers, or other application-level details.
For example, an organisation might route:
example.com/images/ to one group of servers
and:
example.com/api/ to another group.
This gives application architects more control over traffic routing, although it can also make the configuration more complex.
Load Balancing and Databases Are Not the Same Thing
A common misunderstanding is that adding a load balancer automatically distributes every part of an application. It does not. Suppose three web servers are behind a load balancer, but all three depend on a single database server. The web tier may have plenty of capacity while the database becomes the bottleneck.
The same issue can occur with:
- storage systems
- external APIs
- authentication services
- message queues
- caches
- network connections
A useful architecture therefore considers the entire request path.
User → Load Balancer → Application → Database
Improving only the first part may not solve a bottleneck further down the chain.
What Happens During Server Maintenance?
Load balancing can also make routine maintenance easier. Suppose an organisation needs to update one application server. Instead of continuing to send new traffic to that server during the update, the server can be removed from the active pool or placed into a suitable draining state, depending on the platform. The other servers continue handling traffic.
After maintenance, the server can be checked and returned to the pool. This approach can reduce disruption, but it still requires careful planning. Before taking the server offline, you may need to handle existing connections, long-running requests, background tasks, and application state.
When Load Balancing Is Not the Complete Answer
Adding a load balancer introduces another component that requires configuration, monitoring, security, and maintenance. It can also introduce problems if configured poorly.
For example, an incorrectly configured health check might consider a server healthy when the application is actually unable to serve useful requests. An unsuitable routing algorithm might also distribute work poorly. Another consideration is the load balancer itself.
If an architecture has one load balancer and that device or service becomes unavailable, it can become a single point of failure. Production systems that require high availability may therefore use redundant load-balancing infrastructure or a managed service designed to avoid this dependency.
Cost is another factor. For a small website with modest traffic, adding several application servers and a dedicated load-balancing layer may create unnecessary complexity. The architecture should match the actual workload.
A Practical Way to Think About Load Balancing
Instead of thinking of a load balancer as simply a traffic splitter, consider it to be a traffic coordinator. It sits between incoming clients and a group of backend resources. Its job can include:
- Receiving incoming connections or requests.
- Identifying suitable backend servers.
- Selecting a server according to configured rules.
- Monitoring backend health.
- Avoiding unhealthy servers when appropriate.
- Sending traffic to available backends.
- Supporting changes in the backend pool.
This distinction matters because the load balancer is making decisions based on the state and design of the system.
Questions to Ask Before Adding Load Balancing
Before introducing a load-balancing layer, an organisation should understand what problem it is trying to solve.
Useful questions include:
- Is the current server actually reaching a capacity limit?
- Which resource is the bottleneck: CPU, memory, connections, storage, or something else?
- Can the application run correctly on multiple servers?
- Where is session information stored?
- What happens if one backend server fails?
- How are backend servers monitored?
- Does the database have enough capacity?
- Does the application require connection persistence?
- How will servers be added or removed?
- Is the load balancer itself highly available?
These questions prevent a common architectural mistake: adding a load balancer without addressing the actual limiting component.
Conclusion
Load balancing provides a way to place multiple servers behind a shared entry point and distribute incoming traffic among them. Depending on its configuration, it can help an application use additional capacity, avoid sending requests to unhealthy servers, and support maintenance with less disruption.
The important part is the architecture around it. A load balancer cannot compensate for every bottleneck, and simply adding more servers does not guarantee better performance. The application, database, storage, sessions, monitoring, and failover strategy all need to work together. When those pieces are designed appropriately, load balancing becomes an important part of building networked applications that can handle changing workloads without relying entirely on a single backend server.
FAQs
1. What is load balancing in simple terms?
Load balancing distributes incoming network requests or connections across multiple servers instead of sending all of them to one server. The goal is to share the workload and help the application use its available backend capacity effectively.
2. Does load balancing speed up a website?
It can improve performance when a single server is overloaded and additional servers have available capacity. However, it does not automatically speed up every request. Network latency, application code, databases, external services, and other components can still limit performance.
3. What happens if one server fails?
A load balancer with appropriate health checks can detect that a backend server is unhealthy and stop sending new traffic to it. The remaining servers can continue handling requests if they have enough capacity.
4. Is load balancing necessary for every website?
No. A small website with modest traffic may not need multiple application servers or a load-balancing layer. It becomes more relevant when an application needs additional capacity, redundancy, or controlled traffic distribution.
5. Can a load balancer manage traffic distribution between servers in different locations?
Yes, depending on the technology and architecture. Traffic can potentially be distributed across servers in different data centres or geographic regions. However, this introduces additional considerations such as latency, DNS behaviour, data consistency, failover design, and regional dependencies.