Scalability
Scalability is the ability of a system, process, or method to accommodate increasing workload, size, or demand while maintaining acceptable operation.
Sutskever's List
Scaling is the practice of increasing a model's size, parameter count, training data, or computation to improve performance.
- How it is used
- Frames improvements in language models as a resource-allocation problem while testing claims that more scale alone explains progress.
- How deep it goes
- Explains empirical loss curves and parameter–data recipes, then qualifies them with undertraining and double descent rather than deriving the laws formally.
- Examples
- Compares GPT-2’s fluency, GPT-3’s token-to-parameter ratio, and ResNet test error as width increases.
- What you can do after
- Evaluate/understand when more parameters, data, or compute are likely to help and why an undertrained larger model may lose.
Amazon Web Services in Action, Third Edition: An in-depth guide to AWS
Scalability is the ability of a system or infrastructure to increase or decrease capacity to handle changing workload demands.
- How it is used
- Uses scalability to match web-application and background-worker capacity to changing demand without maintaining peak capacity continuously.
- How deep it goes
- Works through Auto Scaling group settings, scaling signals, and application architectures; does not prescribe one metric for every workload.
- Examples
- Demonstrates a CPU-scaled WordPress blog, queue-length-scaled screenshot workers, and ECS tasks scaled by CPU utilization.
- What you can do after
- Apply it via AWS Auto Scaling groups and CloudWatch to adjust EC2 capacity, while choosing decoupling and storage that tolerate replacement.
Natural Language Processing in Action, Second Edition
Scalability is the ability of a computational system or model to increase capacity, data volume, or workload while remaining practically trainable or usable.
- How it is used
- Explains how transformer capacity grows and tests whether larger language models improve task performance.
- How deep it goes
- Explains stackable layers and parallel matrix operations, then inspects benchmark scaling categories without deriving compute requirements.
- Examples
- Uses pandas benchmark counts and GPT and Llama arithmetic outputs to contrast greater capacity with reliable performance.
- What you can do after
- Evaluate/understand it by checking task-level gains and generated answers instead of assuming larger models reason reliably.
Read how Natural Language Processing in Action, Second Edition explains scalability →
Machine Learning System Design
Scalability is the ability of a system to maintain acceptable operation as workload, data volume, or users increase.
- How it is used
- Frames capacity decisions for growing training datasets and variable inference demand without sacrificing required performance.
- How deep it goes
- Defines vertical and horizontal training approaches and sketches serving controls, but does not provide distributed-training implementation steps.
- Examples
- Uses contrasting dataset types, retail traffic spikes, a batch forecasting design and a cached real-time inference service.
- What you can do after
- Operate/configure scaling choices by matching capacity to expected load and monitoring whether production performance holds.
Read how Machine Learning System Design explains scalability →
Generative AI in Action
Scalability is the ability of a system to accommodate increasing workloads, users, or traffic while maintaining acceptable operation.
- How it is used
- Frames production GenAI architecture around changing demand, reliable service, and acceptable performance.
- How deep it goes
- Provides deployment and operational measures, including load balancing, auto-scaling, stress testing, and monitoring, without specifying capacity thresholds.
- Examples
- Uses growing RAG indexes and model-routing architecture to illustrate increasing data volumes and demand spikes.
- What you can do after
- Operate/configure it through deployment selection, load balancing, auto-scaling, monitoring, and performance optimization.
Bootstrapping Microservices, Second Edition: With Docker, Kubernetes, GitHub Actions, and Terraform
The ability of an application or its data-management components to accommodate increasing workload or size by expanding resources or capacity.
- How it is used
- Frames how a growing microservices application can increase capacity without enlarging every service together.
- How deep it goes
- Outlines cluster, service, and database scaling options, but stops short of hands-on scaling procedures or advanced performance techniques.
- Examples
- Uses service replication and a database moved from a shared server to a dedicated server as concrete growth scenarios.
- What you can do after
- Evaluate/understand when to scale a cluster, service, or database, and why to test risky changes away from production.
LLMs in Production
Autoscaling is the automatic adjustment of computing resources or service instances in response to workload or capacity conditions.
- How it is used
- Keeps LLM services responsive under changing request volume while releasing replicas when demand falls.
- How deep it goes
- Explains replica adjustment through native HPA signals and identifies tools needed for GPU metrics, but does not show configuration steps here.
- Examples
- Uses daytime and nighttime demand, traffic bursts, and GPU utilization to illustrate when and how replica counts should change.
- What you can do after
- Evaluate/understand it as a reliability and cost measure, and distinguish native CPU-and-memory scaling from GPU-metric scaling for LLM services.
Kubernetes for Developers
Scaling is the adjustment of application instances or computing capacity to match changing workload demand.
- How it is used
- Adjusts application and cluster capacity as traffic rises or falls, reducing intervention during spikes and expense during quiet periods.
- How deep it goes
- Explains separate Pod and node scaling, HPA targets and limits, and a load test without detailing autoscaler internals.
- Examples
- Uses a Deployment replica setting, a CPU-targeted HPA manifest, and an Apache Bench load test against a CPU-intensive endpoint.
- What you can do after
- Operate/configure it with Deployment replicas and an HPA, then inspect Pods for unmet node capacity.
Starting Data Analytics with Generative AI and Python
Scalability is the ability of a system or method to maintain or improve effective operation as workload or data volume increases.
- How it is used
- Guides data-analysis systems through workload growth while preserving timely and effective results.
- How deep it goes
- Defines performance measures, distinguishes optimization from scaling, and surveys hardware, software, and deployment strategies.
- Examples
- Uses Brazilian e-commerce, patient monitoring, financial markets, observatories, hospitals, and cloud migration to contrast scaling priorities.
- What you can do after
- Evaluate data-analysis scalability with throughput, bandwidth, response time, availability, reliability, and workload-specific scaling choices.
Read how Starting Data Analytics with Generative AI and Python explains scalability →
Effective Platform Engineering
Scalability is the ability of a system or organization to handle increasing workload or scope while maintaining required performance and operation.
- How it is used
- Guides intentional architectural and organizational changes as engineering platforms support more teams, clusters, services, and capabilities.
- How deep it goes
- Connects team boundaries, event-driven releases, dynamic pipelines, distributed deployers, APIs, cost, and user experience, without implementation-level detail.
- Examples
- Uses multiple control planes, lifecycle pipelines, distributed cluster deployers, release events, contract tests, and evolutionary scaling strategies.
- What you can do after
- Evaluate and understand platform scaling choices through team boundaries, event-driven releases, dynamic pipelines, distributed deployers, and cost assessments.
Read how Effective Platform Engineering explains scalability →
Kubernetes in Action, Second Edition
Horizontal scaling is the addition or removal of application instances to adjust capacity to changing workload levels.
- How it is used
- Uses Kubernetes to adjust application capacity by replicating Pods as workload levels change.
- How deep it goes
- Explains replica commands, Deployment state, Pod units, Service routing, and application constraints; defers advanced scaling mechanisms.
- Examples
- Demonstrates `kubectl scale`, three ready replicas, Pod placement, and Service requests reaching different application instances.
- What you can do after
- Operate/configure a Kubernetes Deployment with `kubectl scale` and verify replica readiness and Service-level request distribution.
Read how Kubernetes in Action, Second Edition explains scalability →
Build AI-Enhanced Web Apps
Scalability is the ability of a system to maintain acceptable operation as workload, data volume, or user demand increases.
- How it is used
- Frames production deployment decisions around traffic spikes, heavy AI workloads, and avoiding service downtime.
- How deep it goes
- Provides traffic-based deployment checks and workload mitigations, but does not specify capacity calculations or a complete scaling implementation.
- Examples
- Uses request-per-day deployment tiers and uploaded-document parsing and embedding as concrete workload examples.
- What you can do after
- Operate/configure it by matching deployment checks to expected traffic and moving heavy document work off the request path.
Kafka for Architects
Scalability is the ability of a system to maintain or improve its performance as workload or resource demands increase.
- How it is used
- Frames capacity decisions for Kafka consumers, brokers, deployment models, and high-throughput event-driven systems.
- How deep it goes
- Explains consumer limits, lag monitoring, worker-pool trade-offs, and broker rebalancing costs; defers optimal partition-count selection.
- Examples
- Contrasts streaming-platform events with low-volume e-commerce orders, and uses a key-sharded worker pool to illustrate consumer parallelism.
- What you can do after
- Evaluate/understand it by identifying consumer bottlenecks, interpreting lag, and accounting for partition movement before adding capacity.
ScyllaDB in Action
Scalability is the ability of a system to accommodate increasing workload or data volume by expanding its resources while maintaining acceptable performance.
- How it is used
- Positions adding nodes as the practical response to data-store limits and growing capacity demands.
- How deep it goes
- Explains horizontal scaling through cluster distribution, node bootstrapping, and fleet-maintenance trade-offs without detailing configuration commands.
- Examples
- Contrasts a compute- and memory-limited relational server with a ScyllaDB cluster that adds and bootstraps nodes.
- What you can do after
- Operate/configure it by adding cluster nodes and accounting for bootstrap and maintenance work.
Microsoft Azure in Action
Scalability is the ability of a system to accommodate changing workloads by increasing or decreasing resources.
- How it is used
- Uses scalability to keep websites and processing workloads responsive as demand changes without buying hardware for peak traffic.
- How deep it goes
- Explains capacity limits, autoscaling thresholds, instance bounds, and a database throughput cap; defers detailed scale-set operation.
- Examples
- Illustrates traffic spikes with a hosted website and holiday database demand with a Cosmos DB autoscaling command.
- What you can do after
- Apply it via Azure CLI by enabling Cosmos DB autoscaling and setting an explicit maximum throughput.
Road to Kubernetes
Scalability is the ability of a system or process to accommodate increasing workload or resource demands while maintaining acceptable operation.
- How it is used
- Uses replication and traffic distribution to support growing application workloads in containerized deployments.
- How deep it goes
- Progresses from repeated Compose services and NGINX routing to Kubernetes replica declarations, stopping short of broader scaling management.
- Examples
- Demonstrates repeated Compose services, NGINX upstreams, and a 15-replica NGINX Kubernetes Deployment.
- What you can do after
- Apply it via Docker Compose, NGINX, or kubectl to run multiple instances and distribute requests across available containers.
Deep Learning with R, Third Edition
Scalability is the ability of a system or method to handle increasing amounts of data or computation without becoming impractical.
- How it is used
- Frames deep learning as practical for growing datasets, workloads, models, and deployment demands.
- How deep it goes
- Explains hardware, batching, budget, data, and deployment tradeoffs, while stopping short of formal scaling laws or derivations.
- Examples
- Uses small-batch training, GPT-3's compute budget, cheaper deployment hardware, and matrix-based geometric scaling as examples.
- What you can do after
- Evaluate whether to scale data, model size, or computation while accounting for hardware cost, data quality, and generalization limits.
Read how Deep Learning with R, Third Edition explains scalability →
Machine Learning Platform Engineering
The ability of a system to maintain suitable operation as workload volume or resource demand increases.
- How it is used
- Protects production ML usability by preparing inference services for varying, heavy, and spiky workloads.
- How deep it goes
- Explains deployment concerns, load testing, monitoring, and auto-scaling without detailing specific scaling algorithms or capacity plans.
- Examples
- Uses REST APIs, Docker deployment, AWS or GCP hosting, load tests, RPS metrics, HTTP status codes, and production lifecycle diagrams.
- What you can do after
- Operate/configure production inference by load-testing services, enabling auto-scaling, and monitoring operational and ML-specific performance.
Read how Machine Learning Platform Engineering explains scalability →
C++ Concurrency in Action, Second Edition
Scalability is the ability of a system or algorithm to increase its capacity as available resources increase.
- How it is used
- Frames scalability as a performance criterion for concurrency, guiding design and testing as processor counts increase.
- How deep it goes
- Explains serial-versus-parallel limits through Amdahl’s law and identifies waiting and contention as additional constraints.
- Examples
- Compares single- and 24-core performance, serial-work limits, and processing multiple files concurrently.
- What you can do after
- Evaluate scalability by comparing performance across processor configurations and checking whether serial work or contention limits gains.
Read how C++ Concurrency in Action, Second Edition explains scalability →
Platform Engineering on Kubernetes
Scaling is the process of increasing or decreasing the capacity or number of running service instances to match workload needs.
- How it is used
- Frames service capacity as an operational decision for matching workload load in distributed and Kubernetes applications.
- How deep it goes
- Demonstrates scaling services up and down and discusses replicas, but defers detailed autoscaler internals.
- Examples
- Uses a walking-skeleton application, local Kubernetes cluster, service replicas, and Knative scale-to-zero workloads.
- What you can do after
- Operate/configure Kubernetes services by scaling replicas, checking behavior, and considering autoscaling or availability needs.
Read how Platform Engineering on Kubernetes explains scalability →
Think Distributed Systems
Scalability is the ability of a system to maintain its intended behavior as demand or workload increases.
- How it is used
- Frames scalability as responsiveness under increasing load when evaluating distributed-system fitness.
- How deep it goes
- Defines responsiveness through service objectives, error rates, and error budgets, but stops short of implementation strategies.
- Examples
- Uses modern applications, increasing load, single-component limits, and interacting distributed components as concrete illustrations.
- What you can do after
- Evaluate scalability by defining acceptable responsiveness, service objectives, error rates, and error budgets under load.
Deep Learning with Python, Third Edition
The practice of increasing model size, data, or training compute to improve performance in machine learning.
- How it is used
- Frames scaling as a strategy for improving performance while testing whether larger systems overcome deep-learning limitations.
- How deep it goes
- Explains empirical scaling laws and practical caveats, then applies scaling operationally to feature preprocessing and attention scores.
- Examples
- Uses regression feature preprocessing, scarce-data model selection, GPT-3 and GPT-4 comparisons, and scaled dot-product attention code.
- What you can do after
- Evaluate whether scaling improves useful capability, choose smaller models for scarce data, and implement stable attention-score scaling.
Read how Deep Learning with Python, Third Edition explains scalability →
Akka in Action, Second Edition
Scalability is the ability of a system to adapt to changes in resource demand without negatively affecting performance.
- How it is used
- Frames concurrency and distributed deployment as ways to maintain performance while demand and resource requirements grow.
- How deep it goes
- Defines scalability and contrasts scaling up with scaling out, but stops short of implementation procedures or performance measurements.
- Examples
- Uses additional CPUs, additional cluster servers, cloud platforms, Docker, Kubernetes, and demand-versus-resources figures.
- What you can do after
- Evaluate whether changing demand calls for more CPU capacity or additional servers, while considering cost and performance.
Read how Akka in Action, Second Edition explains scalability →
Architecting an Apache Iceberg Lakehouse
Scalability is the ability of a system to handle increasing data volume or demand while maintaining acceptable operation through manageable expansion.
- How it is used
- Guides storage-layer and catalog-layer choices against rising data volume, demand, metadata traffic, cost, and operational effort.
- How deep it goes
- Defines workload questions and evaluation dimensions, but stops short of implementation procedures or scaling algorithms.
- Examples
- Uses seasonal workloads, real-time analytics, schema changes, multi-tenant catalogs, thousands of tables, and high-concurrency metadata requests.
- What you can do after
- Evaluate storage and catalog candidates against workload growth, latency, provisioning, availability, interoperability, and labor requirements.
Read how Architecting an Apache Iceberg Lakehouse explains scalability →
Grokking Concurrency
Scalability is a system characteristic describing its ability to increase performance by adding processing resources.
- How it is used
- Frames scalability as a way to address growing problem size and improve system performance by adding resources.
- How deep it goes
- Defines vertical and horizontal scaling, compares their trade-offs, and stops short of implementation details.
- Examples
- Uses more powerful processors, top-tier cloud instances, and interconnected computing clusters to illustrate scaling choices.
- What you can do after
- Understand the distinction between scaling up and scaling out, including their limits and concurrency requirements.
Effective Conversational AI
Scalability is the ability of a system to maintain acceptable operation as data volume, query volume, or workload increases.
- How it is used
- Balances personalized virtual-assistant experiences with responsiveness, operational efficiency, and cost across a large user base.
- How deep it goes
- Explains scalability through personalization, query complexity, indexing, storage, latency, QPS, and recall metrics.
- Examples
- Uses travel recommendations, vegan recipe preferences, customer support retrieval, news aggregation, and technical troubleshooting.
- What you can do after
- Evaluate assistant scalability by comparing workload handling, indexing speed, storage, latency, QPS, query complexity, and recall.
Apache Kafka in Action
Scalability is the ability of a system to maintain or improve its service as workload or resource capacity increases.
- How it is used
- Supports reliable real-time processing by distributing Kafka data, connector workers, tasks, and enterprise workloads.
- How deep it goes
- Explains worker rebalancing, task allocation, cluster separation, and production pitfalls, but does not quantify capacity or performance limits.
- Examples
- Uses an online-shopping platform, twelve database tables, distributed connector workers, and separated retailer clusters.
- What you can do after
- Operate/configure Kafka Connect by adding workers, setting tasks.max, and selecting production-suitable connectors.
Related concepts
- Knativeused by · Knative is an open-source, Kubernetes-based platform for deploying and operating serverless and event-driven workloads.
- horizontal scalingincludes · Horizontal scaling is adjusting system capacity by adding or removing computing nodes or equivalent instances that share or distribute the workload.
- vertical scalingcontrasts with · Vertical scaling is increasing or decreasing a system’s capacity by changing the resources available to an existing machine, node, or instance.
- nonfunctional requirementkind of · A nonfunctional requirement specifies a quality attribute, constraint, or operating condition that a system must satisfy beyond the particular functions it performs.
- dataset splitaddressed by · A dataset split is the division of a dataset into distinct subsets assigned different purposes, such as training, validation, or testing.
- Kubernetesused by · Kubernetes is an open-source platform for orchestrating and automating the deployment, scaling, and management of containerized workloads across computing clusters.
These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.