Scheduler
A scheduler is a system component that selects, orders, dispatches, or assigns execution resources to tasks according to timing, dependency, placement, or other scheduling criteria.
Data Pipelines with Apache Airflow, Second Edition
The scheduler is an Apache Airflow component that determines when eligible workflow tasks should be scheduled for execution.
- How it is used
- Determines when DAG task instances are eligible and queues them for execution.
- How deep it goes
- Explains periodic eligibility checks, scheduled and queued states, executor handoff, logging, and multi-scheduler locking without detailing the full scheduling loop.
- Examples
- Uses a task-run command, an `airflow scheduler` command, task-state flow, and scheduler log files.
- What you can do after
- Operate/configure it by running additional schedulers where database locking permits, and diagnose/debug scheduling through task states and logs.
Read how Data Pipelines with Apache Airflow, Second Edition explains scheduler →
Grokking Concurrency
A scheduler is a runtime or operating-system component that selects and orders tasks for execution and may control their access to processing resources.
- How it is used
- Selects ready tasks for processor time and assigns work to available workers in a concurrent application.
- How deep it goes
- Explains scheduling goals and preemptive versus cooperative control, then implements a central scheduler; stops short of OS scheduler internals.
- Examples
- Illustrates alternating turns for three tasks and a word-count scheduler that distributes files before reducing results.
- What you can do after
- Build it from scratch as a Python central scheduler that assigns map tasks, tracks progress, and initiates reduction.
Kubernetes in Action, Second Edition
The Kubernetes scheduler is the control-plane component that selects a worker node for each unscheduled application instance.
- How it is used
- Places each new application instance on a worker node between controller creation and Kubelet startup.
- How deep it goes
- Defines scheduling and traces its place in deployment, without explaining how the scheduler chooses among nodes.
- Examples
- Follows a kubectl-created Deployment through Pod assignment and Scheduled, Pulling, Created, and Started events.
- What you can do after
- Diagnose/debug with it by distinguishing a Pod’s node assignment from the Kubelet’s subsequent container startup.
Read how Kubernetes in Action, Second Edition explains scheduler →
Fast Python: High performance techniques for large datasets
- How it is used
- Distributes evaluation of lazy Dask task graphs across computing resources.
- How deep it goes
- Explains scheduler alternatives and the distributed scheduler’s components and setup, without detailing its task-placement algorithm.
- Examples
- Uses a CSV-backed Dask data frame to inspect its default scheduler and command-line examples to launch a local distributed deployment.
- What you can do after
- Operate/configure a local dask.distributed scheduler and workers, inspect their dashboards, and select sequential execution for debugging.
Read how Fast Python: High performance techniques for large datasets explains scheduler →
Elixir in Action, Third Edition
A scheduler is a runtime component that selects runnable processes and allocates execution time to them.
- How it is used
- Coordinates concurrent BEAM processes so available CPU cores can execute server work in parallel.
- How deep it goes
- Explains scheduler threads, time slots, preemption, core defaults, and graceful load behavior.
- Examples
- Uses a quad-core VM and architectural figures showing scheduler threads executing many lightweight processes.
- What you can do after
- Evaluate/understand how preemption and core-aware scheduling affect responsiveness and graceful degradation under load.
Read how Elixir in Action, Third Edition explains scheduler →
Build and Orchestrator in Go (From Scratch)
A scheduler is a component that assigns tasks to machines or nodes according to placement decisions.
- How it is used
- Assigns job tasks to worker machines as part of an orchestrator’s deployment and management flow.
- How deep it goes
- Explains feasibility, scoring, and picking, then defines a Go interface for interchangeable scheduling implementations.
- Examples
- Uses round-robin placement, E-PVM scoring, resource-starved workers, container images, and multi-worker Linux setups.
- What you can do after
- Build it from scratch by implementing candidate selection, scoring, and picking behind a Go Scheduler interface.
Read how Build and Orchestrator in Go (From Scratch) explains scheduler →
Spring in Action, Sixth Edition
A Project Reactor scheduler provides an execution context and worker resources for running reactive tasks.
- How it is used
- Controls the concurrency model used when reactive subscriptions and stream transformations execute.
- How deep it goes
- Defines five scheduler modes and demonstrates parallel execution, but stops short of scheduler internals.
- Examples
- Demonstrates player-name mapping, buffered fruit processing, parallel workers, and interleaved execution logs.
- What you can do after
- Apply it via Schedulers and subscribeOn() to distribute reactive work across selected worker models.
Read how Spring in Action, Sixth Edition explains scheduler →
100 Go Mistakes and How to Avoid Them
A scheduler is a runtime or operating-system component that selects executable threads or goroutines and assigns them processing resources.
- How it is used
- Explains how Go distributes goroutines across OS threads and CPU cores when reasoning about concurrency performance.
- How deep it goes
- Traces goroutine states, queues, work stealing, and workload effects without specifying deterministic thread or core assignment.
- Examples
- Uses fixed worker pools, shared channels, GOMAXPROCS, runnable-state diagrams, and a simplified runtime.schedule selection listing.
- What you can do after
- Diagnose concurrency behavior by separating goroutine readiness from actual execution and by relating worker-pool sizing to CPU-bound workloads.
Read how 100 Go Mistakes and How to Avoid Them explains scheduler →
Latency
The selection and allocation of runnable threads to processors for execution.
- How it is used
- Explains how runnable threads share CPUs and why scheduling delays matter to low-latency applications.
- How deep it goes
- Explains time-multiplexing, context-switch costs and practical controls, but does not specify a real OS scheduling algorithm.
- Examples
- Uses a three-thread, four-millisecond CPU timeline and a network-request latency scenario.
- What you can do after
- Diagnose/debug with it by considering scheduling delay and lock convoying, then evaluate thread count, affinity or priority.
Designing Deep Learning Systems: A guide for software engineers
A workflow-system component that detects submitted workflows and dispatches their tasks according to dependency and ordering constraints.
- How it is used
- Coordinates when workflows run and dispatches their dependency-ordered tasks to execution workers.
- How deep it goes
- Explains scheduling, DAG ordering, dispatch, and surrounding components, but does not detail scheduling algorithms or implementation internals.
- Examples
- Uses a monthly model-training workflow whose parsing, augmentation, dataset-building, and training tasks form a dependency chain.
- What you can do after
- Evaluate/understand how a scheduler places workflows and tasks within a production orchestration architecture.
Read how Designing Deep Learning Systems: A guide for software engineers explains scheduler →
ASP.NET Core in Action, 3e
A scheduler is the Quartz.NET runtime component that tracks triggers, creates jobs through a job factory, and executes them.
- How it is used
- Coordinates recurring or calendar-based background jobs and supports distributed execution across clustered application instances.
- How deep it goes
- Explains scheduler components and hosted-service installation, stopping before configuring concrete jobs and triggers.
- Examples
- Uses a generic-host configuration listing and an exchange-rate downloader converted into a Quartz.NET job.
- What you can do after
- Configure Quartz.NET as a hosted background service in a .NET application using dependency injection.
Code Like a Pro in Rust
A scheduler is a runtime component that determines when asynchronous tasks execute and switches between them.
- How it is used
- Coordinates asynchronous task execution while requiring code to yield control at appropriate points.
- How deep it goes
- Explains task switching, blocking, and yield points but defers detailed runtime internals.
- Examples
- Uses synchronous and Tokio timer programs to contrast blocking sleep with an awaited asynchronous timer.
- What you can do after
- Understand how yielding and blocking choices affect asynchronous task switching.
Related concepts
- BEAMused by · BEAM is the virtual machine and runtime environment that executes Erlang-family programs and applications.
- Apache Airflowpart of · Apache Airflow is an open-source platform for authoring, scheduling, and monitoring programmatically defined workflows, commonly represented as directed acyclic graphs.
- preemptionused by · Preemption is the interruption or displacement of active execution or resource allocation so another eligible activity can proceed.
- agent orchestrationpart of · Orchestration is the coordination and sequencing of agents, services, tasks, or components to carry out a larger workflow or objective.
- threaduses · A thread is an independently schedulable sequence or context of execution within a program or process.
- computation graphused by · A computation graph is a directed graph whose nodes represent computational operations or values and whose edges represent data dependencies between them.
These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.