Load balancing
Load balancing is the distribution or adjustment of work across multiple resources to manage demand and reduce overload or imbalance.
Kubernetes in Action, Second Edition
Load balancing is the distribution of incoming work across multiple application instances or other processing targets.
- How it is used
- Distributes incoming traffic across application replicas so clients can use a Service rather than choose individual pods.
- How deep it goes
- Works through a three-pod request example and distinguishes internal Service balancing from external node balancing without detailing balancing algorithms.
- Examples
- Uses a three-replica scaling command, responses naming different pods, and a LoadBalancer Service manifest.
- What you can do after
- Operate/configure it with kubectl and a Service manifest, then distinguish pod distribution from external node distribution.
Read how Kubernetes in Action, Second Edition explains load balancing →
Amazon Web Services in Action, Third Edition: An in-depth guide to AWS
Load balancing is the distribution of requests or workloads across multiple computing resources to improve availability and performance.
- How it is used
- Distributes web requests across virtual machines to improve availability and decouple clients from individual servers.
- How deep it goes
- Explains request forwarding, health checks, target groups, console inspection, and load balancer selection without detailing routing algorithms.
- Examples
- Uses a WordPress deployment with two web servers and a web shop that continues serving customers after a virtual-machine failure.
- What you can do after
- Operate/configure it through AWS Elastic Load Balancing by inspecting a load balancer, its target group, target health, and monitoring metrics.
Build Python Web Apps with Streamlit
A load balancer is an infrastructure component that distributes incoming network traffic across multiple service instances and can route traffic away from unhealthy instances.
- How it is used
- Deploys Streamlit applications across multiple EC2 instances and routes user traffic through a shared entry point.
- How deep it goes
- Explains request flow, instance failures, deployment choices, and HTTPS listeners, but defers detailed load-balancer internals.
- Examples
- Uses an Elastic Beanstalk deployment, an application load-balancer selection, a single-instance alternative, and an HTTPS listener configuration.
- What you can do after
- Operate/configure a load-balanced Elastic Beanstalk environment and attach an HTTPS listener and certificate.
Read how Build Python Web Apps with Streamlit explains load balancing →
Tech Startup Toolkit
Load balancing is the distribution of incoming workload across multiple servers to prevent any one server from exceeding its capacity.
- How it is used
- Frames website load balancing as a startup product that prevents crashes during high-demand interactive web use.
- How deep it goes
- Explains the operational purpose through outage and customer-demand cases, without detailing traffic-routing mechanics.
- Examples
- Uses Victoria’s Secret’s Super Bowl crash, Fidelity’s 401(k) access, and WebSpective’s web-server array.
- What you can do after
- Evaluate/understand when simultaneous visitor demand makes distributing work across servers commercially valuable.
Acing the Frontend Interview
Load balancing distributes requests or workloads across multiple servers or processing instances.
- How it is used
- Positions load balancing as the traffic-routing mechanism that makes horizontal scaling practical.
- How deep it goes
- Explains the reverse-proxy role and request routing, while leaving detailed routing and failover tuning to advanced practice.
- Examples
- Uses a reverse-proxy request flow, multiple backend instances, and implementations such as NGINX, HAProxy, and cloud ELBs.
- What you can do after
- Evaluate how a load balancer distributes traffic across instances during horizontal scaling interviews.
Read how Acing the Frontend Interview explains load balancing →
Generative AI in Action
Load balancing distributes computational requests or workloads across multiple resources to improve utilization, performance, and availability.
- How it is used
- Distributes GenAI traffic across infrastructure to support scalable, responsive, and available production services.
- How deep it goes
- Explains traffic distribution, auto-scaling, redundancy, and monitoring but does not specify balancing algorithms or implementation procedures.
- Examples
- Uses cloud deployments, multi-region model services, enterprise internal customers, and ensemble architectures as application contexts.
- What you can do after
- Operate/configure production services with load balancing, auto-scaling, monitoring, caching, and redundancy aligned to demand and availability needs.
Road to Kubernetes
Load balancing is the distribution of incoming traffic across one or more servers capable of handling it.
- How it is used
- Scales applications by forwarding incoming requests across multiple reachable servers or container instances.
- How deep it goes
- Defines reverse-proxy forwarding, demonstrates upstream configuration, and names algorithms while deferring deeper algorithm configuration.
- Examples
- Uses NGINX upstream files, repeated Compose services, localhost ports, remote IP addresses, and Python and Node.js applications.
- What you can do after
- Apply it via NGINX and Docker Compose to distribute requests across repeated application services.
Designing AI Systems
Load-based routing distributes requests among providers or execution paths according to their current workload or capacity.
- How it is used
- Routes model requests across providers to increase throughput and avoid rate limits.
- How deep it goes
- Defines least-load selection and counter lifecycle, while stopping short of provider-specific implementation code.
- Examples
- Uses multiple providers, multiple provider accounts, and rate-limit-induced fallback as operational examples.
- What you can do after
- Build it from scratch by tracking per-provider active requests and dispatching each request to the least-loaded provider.
Server-Side WebAssembly
Load balancing is the distribution of application requests or work across multiple running instances or hosts.
- How it is used
- Uses replicas and request routing to improve concurrent throughput and prepare Wasm applications for production deployment.
- How deep it goes
- Explains replica counts, weighted host placement, benchmarking, and outage simulation, but does not detail the routing algorithm.
- Examples
- Demonstrates one versus ten replicas, weighted East and West Coast placement, benchmark commands, inventory inspection, and host shutdown.
- What you can do after
- Operate/configure wasmCloud scaling with spreadscaler, deploy replicas, benchmark concurrency, inspect placement, and test host failure.
Refactoring to Rust
A load balancer is a system component that distributes incoming requests or tasks among multiple running service instances.
- How it is used
- Coordinates incoming work across running services after horizontal scaling creates multiple application instances.
- How deep it goes
- Defines distribution logic and duplicate-work safeguards, but defers detailed implementation as an architectural concern.
- Examples
- Uses two background Python processes and contrasts distributing work across servers with adding resources to one machine.
- What you can do after
- Evaluate when distributing work across servers fits better than adding resources to existing hardware.
Event Processing in Action
The distribution or adjustment of work among available computing resources to manage workload.
- How it is used
- Frames load balancing as an EPA-assignment optimization for adapting partitioning to changing workloads and resources.
- How deep it goes
- Distinguishes partitioning reevaluation from load shedding but leaves general decisions about when and how to rebalance unresolved.
- Examples
- Uses resource changes, shifting event loads, and event discarding under insufficient capacity as cases.
- What you can do after
- Understand why changing resources and event loads can prompt repartitioning, and recognize event discarding as load shedding.
Read how Event Processing in Action explains load balancing →
Transformers in Action
Load balancing is the distribution of tokens or computation across experts so that no experts are overloaded while others remain underused.
- How it is used
- Prevents sparse MoE routing from overusing some experts while leaving others idle.
- How deep it goes
- Explains top-k routing and introduces capacity and balancing mechanisms, while the minimal gate stops short of enforcing them.
- Examples
- Uses a top-k gating listing and a GShard-style top-2 gate with capacity and random-routing options.
- What you can do after
- Apply it via top-k or GShard-style gating to distribute tokens across selected experts.
Grokking Concurrency
Load balance is the degree to which computational work is distributed evenly across available workers or processing resources.
- How it is used
- Distributes parallel tasks among workers while balancing independent work against the global information that can improve worker utilization.
- How deep it goes
- Explains heuristic grouping and mapping, then describes central dynamic scheduling; it notes that optimal allocation cannot always be guaranteed in advance.
- Examples
- Uses a central scheduler that assigns map and reduce tasks to workers processing files of unknown number and size.
- What you can do after
- Understand how a central scheduler assigns tasks and why dynamic allocation may help when workloads are unknown.
Learn Concurrent Programming with Go
Load balancing is the distribution of work across multiple workers or processing resources to avoid concentrating demand on one worker.
- How it is used
- Uses channels and worker pools to distribute concurrent workloads and exploit multicore hardware.
- How deep it goes
- Explains shared work queues, worker lifecycles, overload rejection, and HTTP implementation, but defers detailed race analysis.
- Examples
- Uses payroll groups and a static-file HTTP server with three workers, busy responses, and parallel curl load.
- What you can do after
- Build it from scratch with Go goroutines, channels, worker pools, and bounded-load rejection.
Read how Learn Concurrent Programming with Go explains load balancing →
Apache Kafka in Action
Load balancing is the distribution of work or resource consumption across system components to avoid substantial imbalance.
- How it is used
- Balances Kafka broker workloads by redistributing partitions according to resource consumption.
- How deep it goes
- Explains monitoring, analysis, review, execution, and automatic detection, while warning about movement-related load.
- Examples
- Uses uneven CPU, memory, and network consumption to motivate moving partitions between brokers.
- What you can do after
- Operate/configure Cruise Control to detect uneven broker load, review balancing proposals, and execute partition movements safely.
Learn Docker in a Month of Lunches, Second Edition
Load balancing is the distribution of requests or traffic across multiple service instances.
- How it is used
- Distributes traffic among scaled Docker Compose service containers through DNS responses.
- How deep it goes
- Demonstrates scaling and DNS inspection, but stops short of application-side response-selection logic.
- Examples
- Uses three scaled iotd containers, nslookup output, and container removal followed by restoration.
- What you can do after
- Operate Docker Compose scaling and inspect returned service addresses from inside a running container.
Read how Learn Docker in a Month of Lunches, Second Edition explains load balancing →
Building Distributed Applications that Work
The distribution of incoming traffic across multiple service instances or resources.
- How it is used
- Scales out a service in an Aspire application by routing requests across multiple instances.
- How deep it goes
- Explains replica configuration and shared versus internal addresses, but does not detail the routing algorithm.
- Examples
- Demonstrates three webfrontend replicas sharing a front-facing address while listening on distinct internal ports.
- What you can do after
- Operate/configure Aspire service replicas by setting their count with WithReplicas in AppHost.cs.
Read how Building Distributed Applications that Work explains load balancing →
Also covered in
- Think Distributed Systems
- Acing the CCNA Exam: Volume 1 Fundamentals and Protocols
- The Cloud at Your Service: The when, how, and why of enterprise cloud computing
- Platform Engineering on Kubernetes
- 100 SQL Server Mistakes and How to Avoid Them
- ScyllaDB in Action
- Grokking Algorithms, Second Edition
- Microsoft Azure in Action
- Build a DeepSeek Model (From Scratch)In early access (MEAP)
- MongoDB 8.0 in Action, Third Edition
- Acing the Certified Kubernetes Administrator Exam, Second Edition
- Writing for Developers: Blogs that get read
What to know first
- HTTPHTTP is an application-layer protocol that defines request-and-response communication for transferring resource representations and related metadata between networked systems.
- IP addressAn IP address is a numerical network-layer address identifying an interface or endpoint for communication over an Internet Protocol network.
- scalabilityScalability is the ability of a system or process to accommodate growth in workload, size, or demand while maintaining acceptable operation.
- availabilityAvailability is the degree or proportion of time that a system or service is operational and accessible when required.
Related concepts
- serviceimplemented by · A Kubernetes Service is an API resource that provides stable network access and service discovery for a changing set of Pods or other specified destinations.
- Kubernetesused by · Kubernetes is an open-source platform for orchestrating containerized workloads and automating their deployment, scaling, and management.
- Gateway Load Balancing Protocolused by · Gateway Load Balancing Protocol is a Cisco first hop redundancy protocol that provides gateway redundancy while distributing traffic across multiple routers.
- NGINXused by · NGINX is open-source web-server and reverse-proxy software that serves HTTP content, forwards requests, and can load-balance network traffic.
- serveruses · A server is a computer system or software process that provides data, resources, or services to other systems, typically by responding to requests.
- horizontal scalingused by · Horizontal scaling is increasing or decreasing system capacity by adding or removing computing nodes or service instances that share the workload.
Questions about load balancing
What is load balancing?
Load balancing is the distribution or adjustment of work across multiple resources to manage demand and reduce overload or imbalance.
Which Manning books explain load balancing?
Kubernetes in Action, Second Edition, Amazon Web Services in Action, Third Edition: An in-depth guide to AWS, Build Python Web Apps with Streamlit, Tech Startup Toolkit, Acing the Frontend Interview, Generative AI in Action, Road to Kubernetes, Designing AI Systems and 9 more.
What should I know before learning load balancing?
Manning books that teach load balancing usually introduce HTTP, IP address, scalability and availability first.
What is load balancing also called?
Load balancing is also called load balance.
These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.
















