liveBookmanning.com

What you will learn in Python Concurrency with asyncio

by Matthew Fowler

Read it on liveBook Get the book at manning.com

Chapter by chapter

Concepts explained in this book

• event loopAn event loop is a control mechanism that repeatedly selects runnable work, executes it, and responds when suspended work becomes ready.
• asyncio.waitasyncio process.wait is a coroutine method that waits for the represented subprocess to terminate and returns its status code.
• async/awaitAsync/await syntax is Python language syntax for declaring asynchronous functions and suspending them until awaitable operations complete.
• asyncioA Python library for asynchronous programming that executes coroutines using concurrency mechanisms such as an event loop.
• taskAn asyncio task is a future-like unit of work that runs a coroutine and eventually contains its result or exception.
• coroutineA coroutine function is a Python function whose invocation produces a coroutine that can suspend and resume during asynchronous execution.
• concurrencyA way of structuring work so that multiple tasks make progress during overlapping periods, including by switching between them while each is incomplete.
• multithreadingMultithreading is a concurrency technique in which multiple threads execute within a process and share its memory.
• asyncpgasyncpg is a Python asynchronous PostgreSQL client library that provides coroutine-based database operations.
• asyncio.gatherasyncio.gather is an asyncio function that concurrently schedules awaitables and completes after all of them finish, optionally returning exceptions as results.
• race conditionA race condition is a concurrency bug in which the result or observable behavior depends on the timing or ordering of concurrent operations.
• stream readerA stream reader is an object that provides asynchronous operations for receiving data from a network stream.
• blockingA socket operating mode in which an application waits for an I/O operation to complete before continuing.
• queue.joinqueue.join is an asynchronous queue method that waits until all enqueued items have been processed and marked complete.
• async withasync with is Python syntax for asynchronously entering and exiting a context manager around a block of code.
• futureA future is an object representing the eventual result or exception of an asynchronous operation that may not yet be complete.
• non-blocking socketA socket operating mode in which I/O calls return without waiting for the operation to complete.
• processAn asyncio process is an object representing a subprocess and providing asynchronous waiting plus process-control methods.
• multiprocessingMultiprocessing is a concurrency technique in which multiple processes perform work concurrently, typically with a parent process managing child processes.
• timeoutA timeout is a configured duration after which an incomplete operation is treated as having exceeded its allowed execution time.
• cooked modeCooked mode is a terminal input mode in which the terminal buffers, preprocesses, and echoes user input before delivering it to an application.
• single-threaded execution modelA single-threaded concurrency model is a concurrency design in which one thread interleaves progress among multiple tasks.
• queueA queue is a data structure that holds items for processing in an ordered sequence, typically first in first out.
• subprocessA subprocess is a process launched by another process, typically to execute an external program.
• queue.putqueue.put is an awaitable queue method that waits asynchronously until capacity is available and then inserts an item.
• connection poolingConnection pooling is the practice of maintaining reusable network connections so multiple operations can share them.
• echo serverAn echo server is a network server that sends received client data back to the client that transmitted it.
• tkinterTkinter is Python's standard interface to the Tcl/Tk toolkit for building graphical user interfaces.
• thread poolA thread pool executor is an execution service that runs submitted callables using a reusable pool of worker threads.
• ASGIASGI is the Asynchronous Server Gateway Interface standard for asynchronous communication between Python web servers, frameworks, and applications.
• global interpreter lockThe global interpreter lock is a mutual-exclusion mechanism in CPython that permits only one thread at a time to execute Python bytecode within a process.
• asyncio.create_subprocess_execasyncio.create_subprocess_exec is an asyncio function that asynchronously starts a subprocess by executing a program directly without invoking a shell.
• Process.communicateProcess.communicate is a coroutine method that waits for a subprocess to finish while concurrently consuming its standard output and standard error.
• parallelismParallelism is the concurrent execution of multiple computational operations, often across multiple processing resources.
• I/O-bound workBlocking I/O work is work whose progress can wait for input or output operations that do not complete immediately.
• semaphoreA semaphore is a synchronization primitive that maintains a counter and permits a bounded number of execution contexts to access a resource concurrently.
• daemon threadA daemon thread is a background thread that does not keep a Python application running when no non-daemon threads remain.
• WSGIWSGI is the Web Server Gateway Interface standard for communication between Python web servers and web frameworks or applications.
• queue.put_nowaitqueue.put_nowait is a nonblocking queue method that immediately inserts an item or raises a full-queue exception.
• threading moduleThe threading module is Python’s standard-library module for creating and managing threads.
• stream APIAn asyncio stream is a high-level abstraction for creating and managing network connections and reading or writing sequential data.
• protocol factoryA protocol factory is a callable that creates and returns a protocol instance for a connection.
• stream writerA stream writer is an object that provides operations for sending data through a network stream.
• backend-for-frontend patternA backend-for-frontend service is a service dedicated to making backend calls and aggregating responses for a particular type of frontend client.
• conditionA condition is a synchronization primitive that combines a lock for exclusive access with event-based waiting for a state or notification to become true.
• run_in_executorrun_in_executor is an asyncio event-loop method that schedules a synchronous callable to run in an executor and returns an awaitable for its result.
• asynchronous generatorAn asynchronous generator is a Python generator that yields values through an asynchronous iteration protocol and can suspend while awaiting asynchronous operations.
• asyncio.runasyncio.run is a high-level function that executes an asynchronous entry-point coroutine in a newly managed event loop and performs shutdown cleanup.
• StarletteStarlette is a lightweight Python ASGI web framework and toolkit for building asynchronous web applications.
• asynchronous queueAn asynchronous queue is a queue whose insertion and retrieval operations can suspend asynchronously while waiting for capacity or available items.
• critical regionA critical section is a region of code that accesses shared state and must not be executed concurrently by multiple execution contexts.
• StreamReader.readStreamReader.read is a coroutine method that waits for and returns a specified amount of data from a stream.
• HTTP stress testAn HTTP stress test is a test that generates many requests to a URL or service to evaluate its behavior under sustained or heavy load.
• non-atomic operationA non-atomic operation is an operation composed of multiple steps that can be interleaved with steps from another concurrent operation.
• task cancellationTask cancellation is the cooperative termination of an asynchronous task by requesting that it stop at a cancellation point.
• asyncio.as_completedasyncio.as_completed is an asyncio function that yields awaitables producing results in the order their underlying operations complete.
• CustomFutureCustomFuture is a book-local future class that can be subclassed to associate asynchronous completion with a coroutine.
• standard inputStandard input is the conventional input stream through which a process receives data from its execution environment or an attached source.
• asyncio.subprocess.PIPEasyncio.subprocess.PIPE is a sentinel that redirects a subprocess stream to an asyncio-managed pipe.
• afterafter is a Tkinter widget method that schedules a callback to run after a specified time interval in the GUI event loop.
• asyncio.run_coroutine_threadsafeasyncio.run_coroutine_threadsafe is an asyncio function that submits a coroutine to an event loop from another thread and returns a concurrent.futures future.
• call_soon_threadsafecall_soon_threadsafe is an asyncio event-loop method that schedules a regular callback from another thread for execution on the loop thread.
• progress barA progress bar is a graphical control that indicates the completion state of an ongoing operation.
• queue moduleThe queue module is Python's standard-library module for thread-safe queues that coordinate communication between threads.
• thread-safe queueA thread-safe queue is a queue whose operations can be safely invoked by multiple threads concurrently.
• deadlockA deadlock is a state in which concurrent execution cannot progress because each participant waits indefinitely for a resource held by another participant or for an unreleased resource.
• protocolasyncio.Protocol is the base protocol class for implementing event-driven network communication callbacks in asyncio.
• async viewAn async view is a web request handler implemented as an asynchronous callable that can suspend while awaiting operations.
• asyncio.Queueasyncio.Queue is a Python asyncio class implementing an asynchronous first-in-first-out queue with optional capacity limits.
• bounded queueA bounded queue is a queue configured with a finite maximum capacity for stored items.
• checkout_customercheckout_customer is a book-local coroutine function that processes one supermarket customer and marks the customer's queue item complete.
• producerA producer is a component that creates work items and makes them available for downstream processing.
• queue.getqueue.get is an awaitable queue method that waits asynchronously until an item is available and then removes and returns it.
• queue.get_nowaitqueue.get_nowait is a nonblocking queue method that immediately removes and returns an available item or raises an empty-queue exception.
• queue.task_donequeue.task_done is a queue method that records completion of one item previously retrieved from the queue.
• aiohttpaiohttp is a Python asynchronous HTTP client and server library built on asyncio.
• cbreak modeCbreak mode is a terminal input mode that delivers keystrokes individually while retaining selected terminal handling such as interpretation of interrupt keys.
• dequeA deque is a double-ended queue that supports insertion and removal at both ends.
• command-line SQL clientA command-line SQL client is a terminal application that accepts SQL queries interactively and reports their results.
• MessageStoreMessageStore is a book-local class that stores a bounded deque of output messages and invokes an asynchronous callback when items are appended.
• read coroutineThe read coroutine is a book-local coroutine that reads terminal input one character at a time, echoes it, handles deletion, and returns a completed input line.
• raw modeRaw mode is a terminal input mode in which keystrokes are delivered directly to an application without terminal buffering, preprocessing, or echoing.
• selectorsselectors is a Python standard-library module that provides efficient, portable I/O multiplexing over file objects and sockets.
• async forasync for is Python syntax that asynchronously retrieves successive values from an asynchronous iterable until iteration is complete.
• mainloopmainloop is a Tkinter method that starts processing GUI events until the application exits or the loop is stopped.
• socket.recvsocket.recv is a socket method that reads up to a specified number of bytes from the socket.
• circuit breaker patternA circuit breaker pattern is a fault-tolerance pattern that stops sending requests to a failing or slow service and resumes requests after a configured reset condition.
• awaitableAn awaitable is a Python object that can be supplied to await and produces a result when its asynchronous computation completes.
• asyncio.wait_forasyncio.wait_for is an asyncio function that awaits a coroutine or task with a specified time limit and cancels it when the limit is exceeded.
• sock_acceptsock_accept is an asyncio event-loop coroutine method that asynchronously accepts an incoming connection from a supplied non-blocking socket and returns the connection and client address.
• shutdown logicShutdown logic is application behavior that coordinates stopping work and releasing resources when an application is terminated.
• thread-per-connection modelThe thread-per-connection model assigns a dedicated thread to each connected client or connection.
• pollingPolling is a technique that repeatedly checks a resource to determine whether it is ready for processing.
• reentrant lockA reentrant lock is a synchronization lock that its owning thread can acquire multiple times, requiring a matching release for each acquisition.
• asyncio.create_subprocess_shellasyncio.create_subprocess_shell is an asyncio function that asynchronously starts a subprocess by executing a command through a system shell.
• priority queueA priority queue is a data structure that retrieves items according to assigned priority rather than insertion order.
• socket.acceptsocket.accept is a socket method that accepts an incoming connection and returns a connected socket together with the client's address.
• connect_read_pipeconnect_read_pipe is an asyncio event-loop coroutine that connects a protocol to a file-like pipe for reading.
• StreamReaderProtocolStreamReaderProtocol is an asyncio protocol implementation that forwards received stream data to a StreamReader.
• multiprocessing moduleThe multiprocessing module is Python’s standard-library module for creating and managing processes.
• done setA done set is the set returned by asyncio.wait containing awaitables that have completed or raised an exception.
• pending setA pending set is the set returned by asyncio.wait containing awaitables that have not completed when the wait returns.
• database result setA database result set is the collection of rows produced by executing a database query.
• asyncio.create_taskasyncio.create_task is an asyncio function that schedules a coroutine for concurrent execution and returns a task representing it.
• event notification systemAn operating-system mechanism that tracks asynchronous I/O resources and reports when they are ready for processing.
• WebSocketWebSocket is a network communication standard that provides a persistent, bidirectional connection between a client and server.
• thread safetyThread safety is the property of behaving correctly when accessed or modified concurrently by multiple threads.
• MapReduceMapReduce is a distributed data-processing pattern that applies a mapping function to input partitions and combines the resulting intermediate values with a reduction function.
• CancelledErrorCancelledError is the asyncio exception raised in a task when cancellation is delivered while the task is being awaited or resumed.
• command protocolA command protocol is an application-level convention for encoding commands and data exchanged between communicating programs.
• DjangoDjango is a batteries-included Python web application framework that provides components for building database-backed web sites and services.
• partition generatorA partition generator is a generator that yields successive chunks of a larger sequence or collection.
• closed stateThe closed state is the circuit-breaker state in which requests are sent normally to the protected service.
• open stateThe open state is the circuit-breaker state in which calls to a failing service are bypassed and fail immediately.
• requestsRequests is a Python library for making HTTP requests synchronously.
• asyncio.PriorityQueueasyncio.PriorityQueue is an asyncio queue class that retrieves enqueued items in priority order.
• heapsortHeapsort is a comparison-sorting algorithm that organizes elements using a heap.
• select statementselect is a selector operation that waits for monitored resources to become ready or for a timeout and returns the resulting events.
• acquireacquire is an asyncpg coroutine that obtains an available database connection from a connection pool and waits if necessary.
• create_poolcreate_pool is an asyncpg coroutine that creates and configures a pool of database connections.
• transportAn asyncio transport is a low-level abstraction that manages the transmission and reception of data over an underlying communication resource.
• ReadTransportReadTransport is an asyncio transport abstraction for receiving data from a readable operating-system resource.
• serve_foreverserve_forever is an asynchronous server method that continuously accepts and serves incoming client connections until the server is stopped.
• TimeoutErrorTimeoutError is a Python exception indicating that an operation exceeded its permitted time limit.
• cursorA database cursor is an object that traverses and retrieves rows from a database query result, potentially incrementally.
• asyncio.new_event_loopasyncio.new_event_loop is a Python API function that creates and returns a new event loop.
• I/OAn I/O operation transfers data between a program and an external resource such as a file, device, or network endpoint.
• selectorA selector is an I/O multiplexing abstraction that monitors multiple resources and reports which are ready for processing.
• coroutine objectA runtime object representing a coroutine computation that has been created but not yet executed to completion.
• microservice architectureMicroservice architecture is a system design style that structures an application as a set of independently deployable services communicating over defined interfaces.
• synchronizationSynchronization is the coordination of concurrent operations to control their ordering and access to shared state.
• concurrency bugA concurrency bug is a software defect caused by interactions among overlapping or concurrently executing tasks.
• database connectionA database connection is an active communication channel or session through which software sends operations to and receives results from a database system.
• context variableA context variable is state whose value is local to an asynchronous execution context and can be accessed without passing it explicitly through function arguments.
• sock_recvsock_recv is an asyncio event-loop coroutine method that asynchronously receives bytes from a supplied socket when they are available.
• hashlibhashlib is Python’s standard-library module providing common cryptographic hash algorithms and related constructors.
• BaseSelectorBaseSelector is the abstract base class defining the common interface for Python selector implementations.
• DefaultSelectorDefaultSelector is a selectors implementation that automatically selects the most efficient available operating-system selector.
• selector registrationSelector registration is the operation of associating a resource with a selector and specifying the events to monitor for it.
• connection_madeconnection_made is an asyncio protocol callback invoked after the underlying connection has been established.
• create_connectioncreate_connection is an asyncio event-loop coroutine that establishes a socket connection and returns its transport and protocol.
• data_receiveddata_received is an asyncio protocol callback invoked when bytes arrive from the transport.
• eof_receivedeof_received is an asyncio protocol callback invoked when the transport receives an end-of-file indication.
• TelnetTelnet is an application protocol and command-line client that provides bidirectional text communication over a TCP connection.
• Process.stdoutProcess.stdout is the Process attribute that exposes a subprocess's redirected standard-output stream reader.
• generatorA generator is a Python iterator object that produces values lazily by suspending and resuming execution around yield points.
• retryRetry is a fault-tolerance technique that repeats a failed or timed-out operation to recover from a potentially transient failure.
• asyncio process.terminateasyncio process.terminate is a non-blocking method that sends SIGTERM to the represented subprocess.
• endpointAn endpoint is a network-accessible address or interface through which a service exposes a resource or operation and handles requests and responses.
• websocket endpointA WebSocket endpoint is a server-side interface that accepts and manages a WebSocket connection.
• NumPyNumPy is a Python library for numerical computing with multidimensional arrays, matrices, and vectorized mathematical operations.
• socket.sendallsocket.sendall is a socket method that transmits all bytes in a given message before returning or raising an error.
• condition.wait_forCondition.wait_for is a coroutine method that waits until a supplied predicate returns true while managing the condition lock.
• notify_allCondition.notify_all is a method that wakes all tasks currently waiting on a condition.
• waitCondition.wait is a coroutine method that releases an acquired condition lock, waits for notification, and reacquires the lock before returning.
• preemptionPreemptive multitasking is a scheduling technique in which the system forcibly pauses running tasks according to its scheduling policy.
• connection socketA connection socket is a socket endpoint representing an established communication channel with a client.
• setsockoptsetsockopt is a socket API operation that changes an option on a socket.
• socket.bindsocket.bind is the socket operation that associates a socket with a local network address.
• socket.listensocket.listen changes a socket into a listening endpoint that can receive incoming connection requests.
• socket moduleThe Python socket module is a standard-library module that provides interfaces for creating and manipulating network sockets.
• socket.socketsocket.socket is the Python socket-module constructor that creates a socket object with specified address-family and socket-type parameters.
• TCPTCP is a reliable connection-oriented transport protocol that delivers an ordered byte stream between applications and retransmits lost data.
• CPythonThe standard reference implementation of the Python programming language, written primarily in C.
• reference countingA memory-management technique that tracks the number of active references to each object and reclaims an object when that count reaches zero.
• traffic-shaping algorithmTraffic shaping is the control of network traffic volume or rate to conform to specified limits.
• taketake is a book-local asynchronous generator that yields a requested maximum number of items from another asynchronous generator.
• separation of concernsSeparation of concerns is a design technique that gives distinct responsibilities to separate components or computations.

These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.