liveBookmanning.com

What you will learn in Latency

Reduce delay in software systems

by Pekka Enberg

Read it on liveBook Get the book at manning.com

Chapter by chapter

Concepts explained in this book

• meanExecution latency is the elapsed time required for a computation or instruction sequence to complete.
• dataset splitA partition is a distinct portion of a dataset assigned to a node for storage or processing.
• parallelismParallelism is the simultaneous execution of tasks across multiple processors or other execution units.
• concurrencyConcurrency is the structuring of multiple computations so that their execution can overlap in time, including through interleaving on one execution resource.
• lockingA mutex lock is an exclusive synchronization primitive that permits only one concurrent execution context to access a protected resource at a time.
• asynchronous programmingAsynchronous processing is a technique in which an operation can proceed or return control before all of its work has completed, allowing other work to continue concurrently.
• event loopAn event loop is a control-flow pattern that waits for events, detects activity from event sources, and dispatches events to handlers.
• prefetchThe proactive retrieval of anticipated data before it is explicitly requested.
• cachingCaching is a technique that stores temporary copies of data closer to where it is accessed to reduce data-retrieval delay.
• hit rateCache hit ratio is the proportion of cache lookup requests whose requested data is found in the cache.
• eventual consistencyEventual consistency is a distributed-system guarantee that replicas converge to the same value when updates stop, although reads may temporarily differ or return stale data.
• performancePerformance is the observable efficiency or responsiveness of a software system.
• optimistic updateAn optimistic update is a client-side update applied before confirmation that the corresponding operation will succeed or be accepted by the authoritative system.
• cache replacement policyA cache replacement policy is a rule for selecting which cached element to evict when storage capacity is exhausted.
• wait-free progress conditionA wait-free data structure is a concurrent data structure whose every operation completes in a finite number of steps regardless of other threads' operations.
• resource contentionResource contention is competition among concurrent operations or workloads for a limited shared resource, causing some to wait.
• key hash partitioningKey hash partitioning assigns records to partitions according to a hash of their partitioning-key values.
• memory barrierA CPU memory barrier is an instruction or mechanism that constrains hardware-level reordering of memory operations.
• memory allocationDynamic memory allocation is reserving and releasing storage during program execution from a runtime-managed memory region.
• replicationA replication protocol is a protocol that coordinates the creation, updating, and consistency of multiple copies of data or state.
• synchronizationA synchronization primitive is an execution mechanism that coordinates concurrent operations and controls access to shared resources.
• range partitioningKey range partitioning assigns records to partitions according to contiguous ranges of partitioning-key values.
• benchmarkA benchmark is a repeatable performance measurement procedure applied to a defined workload or operation.
• Little's lawLittle's law is a queuing-theory theorem stating that the average number of items in a system equals the average arrival or throughput rate multiplied by the average time an item spends in the system.
• multiplexingI/O multiplexing is a technique for monitoring or coordinating multiple input/output operations through a shared event-notification mechanism.
• linear time complexityLinear time is an algorithmic growth class in which required operations increase proportionally to input size.
• partition imbalancePartition imbalance is the uneven distribution of data-processing work across partitions, causing some partitions to be overloaded while others are underutilized.
• queueA FIFO queue is an ordered data structure in which the element added earliest is removed first.
• tail latencyTail latency is the high-latency portion of a latency distribution, typically represented by its upper extreme or percentiles.
• latency distributionA latency distribution is a statistical representation of latency values across multiple comparable operations or observations.
• speculative executionSpeculative execution is a processor technique that executes predicted future instructions before their dependencies or control-flow conditions are confirmed.
• partitioning keyA partitioning key is the data field or value used to determine which partition stores a record.
• cardinalityCardinality is the number of distinct values in a dataset or column.
• data skewData skew is an uneven distribution of records or work across partitions.
• cache missA cache miss occurs when requested data or instructions are absent from the current cache level and must be fetched from slower storage.
• atomicityAtomicity is the transaction property that requires all of a transaction's operations to take effect or none of them to take effect.
• context switchingContext switching is the process of saving one execution context and loading another so multiple processes or threads can share a CPU core.
• virtual memoryVirtual memory is an operating-system memory-management abstraction that presents processes with virtual address spaces mapped to physical memory and secondary storage.
• latency varianceLatency variance is the variation in measured latency across comparable operations or systems.
• MapA lookup table is a data structure that stores precomputed values indexed for rapid retrieval.
• memoizationMemoization is a technique that caches the results of computations so repeated calls with the same inputs can return stored results.
• user-based partitioningUser-based partitioning is horizontal partitioning that assigns data associated with each user to a separate dataset or partition.
• heapA heap is a dynamically sized memory region used for storage whose allocation changes during program execution.
• serializationSerialization is the conversion of in-memory data structures or objects into a representation suitable for storage or transmission.
• acquire semanticsAn acquire memory barrier prevents subsequent memory operations from being reordered before a synchronization operation and observes effects published by a releasing operation.
• release barrierA release memory barrier prevents preceding memory operations from being reordered after a synchronization operation and makes their effects available to an acquiring observer.
• latency compoundingSerial compounding is the accumulation of latency when components execute sequentially and each component delays the next.
• last level cacheA last-level cache is the highest-level and typically largest cache in a processor's cache hierarchy, located immediately before main memory.
• co-locationColocation is the placement of communicating systems or services in the same physical facility or nearby location to reduce communication distance.
• pollingPolling is the repeated checking of a device or status condition by software instead of waiting for an interrupt.
• last-mile latencyLast-mile latency is the delay introduced by the final network segment connecting an access network or edge location to an end user or device.
• simultaneous multithreadingHyper-Threading is a processor technology that allows one physical core to execute multiple hardware threads concurrently.
• hotspotA hot partition is a partition that receives disproportionately large data volume or workload compared with other partitions.
• queuing latencyQueuing latency is the delay caused by a request waiting in a queue before processing.
• socket interfaceA socket API is an operating-system programming interface through which applications communicate over network sockets.
• synchronous callSynchronous processing is a processing model in which operations are handled sequentially and completion or errors are observed before subsequent processing proceeds.
• pattern-based prefetchingHistory-based prefetching is a predictive data-retrieval technique that anticipates future accesses from recorded past user actions.
• schedulerThe selection and allocation of runnable threads to processors for execution.
• data parallelData parallelism is the execution of identical operations simultaneously on separate portions of a dataset.
• non-blocking I/OA non-blocking operation allows execution to continue without waiting for the operation to complete.
• threadA thread is a schedulable sequence of instructions that represents a unit of execution within a process.
• interruptA hardware interrupt is an interrupt signal generated by hardware to request operating-system processing.
• network stackA network stack is the software and hardware subsystem that processes network packets across protocol layers between a physical network interface and applications.
• kernel bypassKernel bypass is a networking technique in which application or userspace code accesses network hardware without routing packet processing through the operating-system kernel.
• deserializationDeserialization is the conversion of a stored or transmitted representation back into in-memory data structures or objects.
• cache-aside cachingCache-aside caching is a pattern in which an application reads from a cache first and loads missing data from a backing store before inserting it into the cache.
• address spaceA process address space is the range of virtual memory addresses available to a running process.
• memory pageA page is a fixed-size block of virtual memory that the operating system maps independently to physical memory or storage.
• cpu cache hierarchyA CPU cache hierarchy is an ordered set of processor caches with differing capacities and access latencies that reduce accesses to slower memory.
• garbage collectionA garbage collection algorithm is a procedure for identifying and reclaiming memory that a program no longer uses.
• lockless techniqueLock-free synchronization is a concurrency technique that guarantees system-wide progress by ensuring that some thread completes an operation within a finite number of steps, without guaranteeing progress for every thread.
• cache hitA cache hit occurs when requested data is found in the cache and can be returned without accessing the backing store.
• thread-per-coreThread-per-core is a concurrency design in which a dedicated execution thread is assigned to each processor core.
• CPU cacheA CPU cache is a small, fast memory that stores copies of data likely to be accessed by a CPU.
• memory topologyMemory topology is the arrangement and connectivity of processors and memory resources that determines their relative access relationships.
• consistency modelA consistency model is a set of guarantees defining the ordering and visibility of operations and data updates in a distributed system.
• packet processing latencyPacket processing latency is the delay between a network packet arriving at a network interface and becoming available to a userspace thread for processing.
• service timeService time is the time required to process a request.
• primary-backup architectureA primary-replica architecture is a replication arrangement in which one primary accepts writes and broadcasts updates to one or more replicas.
• Criterion.rsCriterion is a Rust benchmarking library for statistically evaluating the performance of code.
• flame graphFlameGraph profiling is a visualization-based profiling technique that represents hierarchical sampled execution costs as nested horizontal bars.
• profilingCPU profiling is the measurement and attribution of processor execution time across program call paths.
• geographical latencyGeographical latency is communication delay caused by the physical distance between communicating locations.
• wait-free synchronizationAn algorithm that guarantees each participating thread completes its operation within a finite number of steps regardless of concurrent interference.
• flow steeringFlow steering is the directing of incoming network flows to selected queues or processors based on packet or request attributes.
• non-uniform memory accessNon-uniform memory access is a computer architecture in which memory-access latency differs according to the relationship between a processor and the memory location.
• work eliminationA technique that avoids unnecessary computation, data movement, or other processing to reduce execution effort and delay.
• stackA stack is a memory region that supports function execution by storing call state, registers, return information, and local data.
• RESTA REST API is an HTTP-based application programming interface that exposes resources and operations according to REST architectural constraints.
• SQLiteSQLite is an embedded relational database engine that stores and processes database contents within an application process.
• linearizabilityLinearizability is a consistency property in which each operation appears to take effect atomically at a single point between its invocation and completion, preserving real-time order.
• precalculationThe calculation of results before they are needed so later operations can retrieve or reuse them without repeating expensive computation.
• backing storeA backing store is the authoritative storage layer from which a cache obtains values that it does not contain or whose cached copies have expired.
• Amdahl’s LawAmdahl's law is a performance model that estimates the maximum speedup from parallelizing a fixed-size computation according to its parallelizable fraction and number of execution units.
• isolationTransaction isolation is the property governing how concurrently executing transactions are protected from one another's intermediate or conflicting effects.
• freshnessData freshness is the degree to which data reflects the latest available state of its underlying source.
• logical clockA logical clock is a protocol-maintained ordering mechanism that represents event or system-state progression without measuring physical time.
• CDNA content delivery network is a geographically distributed network of servers that caches and delivers content from locations near users.
• cache-coherent systemCache coherence is the property that multiple caches maintain mutually consistent views of shared data.
• cache keyA cache key is an identifier derived from a request or function input that selects a corresponding cached result.
• binary search treeA binary search tree is an ordered tree data structure in which values in each subtree follow a binary-search ordering relative to its node.
• constant timeConstant time is an algorithmic growth class in which required operations remain independent of input size.
• linear searchLinear search is a search algorithm that examines elements sequentially until it finds a target or reaches the end of a collection.
• logarithmic time complexityLogarithmic time is an algorithmic growth class in which required operations increase proportionally to the logarithm of input size.
• space complexitySpace complexity is the asymptotic growth of memory or storage required by an algorithm as input size increases.
• throughputThroughput is the rate at which a system processes or transfers units of work or data over time.
• replicated state machineState machine replication is a replication technique in which nodes execute the same ordered sequence of commands to maintain identical system state.
• idempotencyIdempotency is the property that repeating an operation produces the same externally relevant result as performing it once.
• consistencyA consistency guarantee is a specified condition governing the visibility and ordering of data updates across replicated copies.
• cache miss ratioCache miss ratio is the proportion of cache accesses that do not find the requested data in the cache.
• compare-and-exchange operationA compare-and-swap instruction atomically compares a memory value with an expected value and conditionally replaces it with a new value.
• parallel compoundingParallel compounding is the accumulation of latency when independent components execute concurrently and completion depends on the slowest component.
• propagation latencyPropagation latency is the time required for a bit or signal to travel across a physical distance or communication path.
• quorumA quorum is a majority subset of nodes whose agreement is sufficient for a distributed operation to complete.
• quorum compoundingQuorum compounding is a parallel execution pattern in which completion requires responses from a majority rather than all participating components.
• transmission latencyTransmission latency is the time required for an entire packet to travel between communication links, including transmission-related error handling and retransmission.
• vertical partitioningVertical partitioning divides a table into partitions containing different subsets of columns, typically retaining a linking key.
• embedded databaseAn embedded database is a database engine that runs in the same process or memory space as the application using it.
• PostgreSQLPostgreSQL is an open-source relational database management system that operates as a database server.
• wrkwrk is an HTTP benchmarking tool that generates concurrent requests and reports server performance measurements.
• L1 cacheAn L1 cache is the smallest and fastest cache level located closest to a CPU core.
• critical regionA critical section is a region of execution that accesses shared state and requires synchronization to prevent conflicting concurrent access.
• percentileA latency percentile is a threshold below which a specified percentage of measured requests have completed.
• replicaA replica is a server or other system member that maintains a copy of data or state replicated from a primary.
• power-saving optimizationHardware power saving comprises hardware or operating-system mechanisms that reduce energy consumption by limiting or adapting component activity.
• backpressureBackpressure is a flow-control technique that causes producers to reduce or pause transmission when consumers cannot process incoming data at the same rate.
• key-value storeA key-value store is a data-storage system that associates values with keys and supports operations such as retrieving and updating a value by its key.
• edge computingEdge computing is a computing approach that places computation and data processing near the users or devices that generate or consume the data.
• cache trashingCPU cache thrashing is repeated eviction and reloading of useful cache data caused by competing or changing working sets.
• interrupt handlingInterrupt handling is the processing performed when a hardware or software interrupt causes the CPU to suspend current execution and invoke an interrupt handler.
• predictive techniqueA predictive technique performs an operation before its need is confirmed by forecasting future access, demand, or execution behavior.
• page faultA page fault is a processor-generated exception raised when an accessed virtual memory page is not currently mapped to usable physical memory.
• multi-version concurrency controlMultiversion concurrency control, or MVCC, is a database concurrency-control technique that maintains multiple versions of records so transactions can read consistent snapshots while concurrent writes proceed.
• spsc bounded queueAn SPSC bounded queue is a fixed-capacity FIFO data structure with exactly one producer and one consumer.
• expiration dateTime-to-live is a caching mechanism that limits how long an entry remains valid before it is treated as expired and removed or refreshed.
• cache coherence protocolA cache coherence protocol is a set of rules that coordinates caches so copies of shared data are updated or invalidated consistently.
• cache lineA cache line is a fixed-size block of memory stored and managed as a unit in a CPU cache.
• MESIMESI is a cache coherence protocol whose cache-line states are Modified, Exclusive, Shared, and Invalid.
• resilienceFault tolerance is the ability of a system to continue providing acceptable service despite failures of components or dependencies.
• fine-grained lockingFine-grained locking is a synchronization technique that protects smaller portions of shared work or state with more numerous locks.
• processor affinityProcessor affinity is a scheduling constraint that binds a thread or process to one or more specified processors or CPU cores.
• Actix WebActix is a Rust framework for building actor-based applications.
• GPUA GPU is a processor designed with many parallel execution resources for graphics and general-purpose data-parallel computation.
• event-driven architectureEvent-driven concurrency coordinates work by responding to events that signal changes or operation completions rather than blocking execution while waiting.
• out-of-order executionOut-of-order execution is a CPU execution technique that dynamically executes instructions when their operands and execution resources are available rather than strictly following program order.
• big-endianA byte order that stores the most significant byte of a multi-byte value first and the least significant byte last.
• endiannessByte order is the ordering of the bytes that represent a multi-byte value in memory or in a serialized representation.
• FlatBuffersFlatBuffers is a serialization library that enables access to serialized structured data without first unpacking it into an intermediate object representation.
• little-endianA byte order that stores the least significant byte of a multi-byte value first and the most significant byte last.
• protocol bufferProtocol Buffers is a cross-platform binary serialization format commonly used for encoding messages in remote procedure calls.
• obstruction-freeObstruction-free is a progress guarantee that an operation completes if it eventually executes alone without interference.
• progress conditionA progress condition is a guarantee describing how concurrent operations advance despite interference, blocking, or scheduling choices.
• starvation-freeStarvation-free is a progress guarantee that every continuously pending operation eventually completes.
• retryA retry mechanism automatically attempts an operation again after a failed attempt.
• read-through cachingRead-through caching is a caching strategy in which the cache automatically retrieves missing values from a backing store and returns them to the requester.
• routingRequest routing is the strategy of forwarding incoming requests to the component responsible for processing them.
• semantic prefetchingSemantic prefetching is a predictive data-retrieval technique that uses application meaning, state, or user-specific information to anticipate needed data.
• memory orderingMemory ordering is the set of rules governing the observable order in which memory operations become effective across concurrent execution contexts.

These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.