liveBook · Manning

Concurrency

Concurrency is the number of requests, users, processes, or tasks that a system or service instance handles simultaneously.

How 2 Manning books explain it

Quantization and Fast Inference

Concurrency is the number of requests or users being served simultaneously by a model instance.

How it is used
Use concurrency to compare batching gains, per-user throughput, and CPU-versus-GPU serving economics.
How deep it goes
Trace batching and memory effects through throughput and queue measurements, while noting when a workload never reaches saturation.
Examples
Compare L4 batching at concurrency 8 with an FP16-versus-AWQ sweep that never filled its request queue.
What you can do after
Evaluate serving tradeoffs by comparing aggregate and per-user throughput with memory headroom and queue behavior.

Read how Quantization and Fast Inference explains concurrency →

Starting Data Analytics with Generative AI and Python

Concurrency is the capacity of a system to handle multiple queries, processes, or tasks at the same time.

How it is used
Evaluates whether data-analysis systems can serve multiple users or tasks while processing continues.
How deep it goes
Defines concurrency and frames load-baseline questions, but stops short of implementation techniques or capacity-planning procedures.
Examples
Compares simultaneous queries or processes with latency, throughput, and response-time measurements.
What you can do after
Evaluate whether a data-analysis system can support current or anticipated simultaneous demand without degraded performance.

Read how Starting Data Analytics with Generative AI and Python explains concurrency →

Also covered in

Related concepts

Questions about concurrency

What is concurrency?

Concurrency is the number of requests, users, processes, or tasks that a system or service instance handles simultaneously.

Which Manning books explain concurrency?

Quantization and Fast Inference and Starting Data Analytics with Generative AI and Python.

What is the difference between concurrency and parallelism?

Concurrency is the number of requests, users, processes, or tasks that a system or service instance handles simultaneously. Parallelism is the execution of multiple independent tasks or computational operations at the same time rather than sequentially.

What kinds of concurrency are there?

Kinds of concurrency include internal concurrency.

These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.