Scheduler
A scheduler is a software component that selects, orders, or dispatches tasks for execution and may assign them execution resources according to defined policies or constraints.
Data Pipelines with Apache Airflow, Second Edition
The scheduler is an Apache Airflow component that determines when eligible workflow tasks should be scheduled for execution.
- How it is used
- Determines when DAG task instances are eligible and queues them for execution.
- How deep it goes
- Explains periodic eligibility checks, scheduled and queued states, executor handoff, logging, and multi-scheduler locking without detailing the full scheduling loop.
- Examples
- Uses a task-run command, an `airflow scheduler` command, task-state flow, and scheduler log files.
- What you can do after
- Operate/configure it by running additional schedulers where database locking permits, and diagnose/debug scheduling through task states and logs.
Read how Data Pipelines with Apache Airflow, Second Edition explains scheduler →
Grokking Concurrency
A scheduler is a runtime or operating-system component that selects and orders tasks for execution and may control their access to processing resources.
- How it is used
- Explains how runtimes allocate processing resources among ready tasks and coordinate worker assignments.
- How deep it goes
- Explains scheduling goals, preemption and cooperative yielding, then implements a state-based central scheduler without detailing an operating-system scheduling algorithm.
- Examples
- Uses time-sliced tasks, blocked I/O, and a word-count server that distributes files to map workers before reducing results.
- What you can do after
- Evaluate/understand scheduling trade-offs and avoid relying on a previously observed task execution order.
Kubernetes in Action, Second Edition
The Kubernetes scheduler is the control-plane component that selects a worker node for each unscheduled application instance.
- How it is used
- Places each new application instance on a worker node between controller creation and Kubelet startup.
- How deep it goes
- Defines scheduling and traces its place in deployment, without explaining how the scheduler chooses among nodes.
- Examples
- Follows a kubectl-created Deployment through Pod assignment and Scheduled, Pulling, Created, and Started events.
- What you can do after
- Diagnose/debug with it by distinguishing a Pod’s node assignment from the Kubelet’s subsequent container startup.
Read how Kubernetes in Action, Second Edition explains scheduler →
Fast Python: High performance techniques for large datasets
- How it is used
- Uses schedulers to execute lazy Dask task graphs on local or distributed resources.
- How deep it goes
- Explains scheduler choices and distributed components, then demonstrates local startup and inspection without detailing scheduling algorithms.
- Examples
- Demonstrates default-scheduler inspection for a CSV-backed data frame and a local distributed scheduler with workers and a client.
- What you can do after
- Operate and configure a local Dask distributed setup and inspect its workers through the dashboard.
Read how Fast Python: High performance techniques for large datasets explains scheduler →
Robotics for Programmers
A scheduler is software that determines which tasks execute and when they execute.
- How it is used
- Coordinates embedded real-time tasks so computations become available on time and deadlines can be met.
- How deep it goes
- Explains priorities and preemption, while deferring detailed scheduler implementation and timing analysis.
- Examples
- Uses a higher-priority task interrupting a lower-priority task as its concrete scheduling example.
- What you can do after
- Evaluate how task priorities and preemption support temporal correctness in embedded real-time systems.
Elixir in Action, Third Edition
A scheduler is a runtime component that selects runnable processes and allocates execution time to them.
- How it is used
- Coordinates concurrent BEAM processes so available CPU cores can execute server work in parallel.
- How deep it goes
- Explains scheduler threads, time slots, preemption, core defaults, and graceful load behavior.
- Examples
- Uses a quad-core VM and architectural figures showing scheduler threads executing many lightweight processes.
- What you can do after
- Evaluate/understand how preemption and core-aware scheduling affect responsiveness and graceful degradation under load.
Read how Elixir in Action, Third Edition explains scheduler →
Build and Orchestrator in Go (From Scratch)
A scheduler is a component that assigns tasks to machines or nodes according to placement decisions.
- How it is used
- Assigns job tasks to worker machines as part of an orchestrator’s deployment and management flow.
- How deep it goes
- Explains feasibility, scoring, and picking, then defines a Go interface for interchangeable scheduling implementations.
- Examples
- Uses round-robin placement, E-PVM scoring, resource-starved workers, container images, and multi-worker Linux setups.
- What you can do after
- Build it from scratch by implementing candidate selection, scoring, and picking behind a Go Scheduler interface.
Read how Build and Orchestrator in Go (From Scratch) explains scheduler →
Spring in Action, Sixth Edition
A Project Reactor scheduler provides an execution context and worker resources for running reactive tasks.
- How it is used
- Controls the concurrency model used when reactive subscriptions and stream transformations execute.
- How deep it goes
- Defines five scheduler modes and demonstrates parallel execution, but stops short of scheduler internals.
- Examples
- Demonstrates player-name mapping, buffered fruit processing, parallel workers, and interleaved execution logs.
- What you can do after
- Apply it via Schedulers and subscribeOn() to distribute reactive work across selected worker models.
Read how Spring in Action, Sixth Edition explains scheduler →
100 Go Mistakes and How to Avoid Them
A scheduler is a runtime or operating-system component that selects executable threads or goroutines and assigns them processing resources.
- How it is used
- Explains how Go distributes goroutines across OS threads and CPU cores when reasoning about concurrency performance.
- How deep it goes
- Traces goroutine states, queues, work stealing, and workload effects without specifying deterministic thread or core assignment.
- Examples
- Uses fixed worker pools, shared channels, GOMAXPROCS, runnable-state diagrams, and a simplified runtime.schedule selection listing.
- What you can do after
- Diagnose concurrency behavior by separating goroutine readiness from actual execution and by relating worker-pool sizing to CPU-bound workloads.
Read how 100 Go Mistakes and How to Avoid Them explains scheduler →
Latency
The selection and allocation of runnable threads to processors for execution.
- How it is used
- Explains how execution is allocated among threads and why allocation matters for low-latency applications.
- How deep it goes
- Explains time multiplexing and context-switch costs, using fixed intervals as a simplified model without detailing real scheduler algorithms.
- Examples
- Illustrates three threads sharing one CPU in 4-millisecond intervals and a coroutine yielding during asynchronous I/O.
- What you can do after
- Evaluate/understand scheduling delay and apply thread-count, affinity, or priority adjustments to address it.
Designing Deep Learning Systems: A guide for software engineers
A workflow-system component that detects submitted workflows and dispatches their tasks according to dependency and ordering constraints.
- How it is used
- Coordinates when workflows run and dispatches their dependency-ordered tasks to execution workers.
- How deep it goes
- Explains scheduling, DAG ordering, dispatch, and surrounding components, but does not detail scheduling algorithms or implementation internals.
- Examples
- Uses a monthly model-training workflow whose parsing, augmentation, dataset-building, and training tasks form a dependency chain.
- What you can do after
- Evaluate/understand how a scheduler places workflows and tasks within a production orchestration architecture.
Read how Designing Deep Learning Systems: A guide for software engineers explains scheduler →
ASP.NET Core in Action, 3e
A scheduler is the Quartz.NET runtime component that tracks triggers, creates jobs through a job factory, and executes them.
- How it is used
- Coordinates recurring or calendar-based background jobs and supports distributed execution across clustered application instances.
- How deep it goes
- Explains scheduler components and hosted-service installation, stopping before configuring concrete jobs and triggers.
- Examples
- Uses a generic-host configuration listing and an exchange-rate downloader converted into a Quartz.NET job.
- What you can do after
- Configure Quartz.NET as a hosted background service in a .NET application using dependency injection.
Code Like a Pro in Rust
A scheduler is a runtime component that determines when asynchronous tasks execute and switches between them.
- How it is used
- Coordinates asynchronous task execution while requiring code to yield control at appropriate points.
- How deep it goes
- Explains task switching, blocking, and yield points but defers detailed runtime internals.
- Examples
- Uses synchronous and Tokio timer programs to contrast blocking sleep with an awaited asynchronous timer.
- What you can do after
- Understand how yielding and blocking choices affect asynchronous task switching.
Also covered in
- Learn Concurrent Programming with Go
- Build a Frontend Web Framework (From Scratch)
- Kubernetes for Developers
- Distributed Machine Learning Patterns
- Go by Example
- Learn Go with Pocket-Sized Projects
- Rust in Action: Systems programming concepts and techniques
What to know first
- CPUA central processing unit (CPU) is a computer processor that executes program instructions and performs general-purpose computational operations.
- workerA worker is a person, process, thread, machine, or other execution resource that performs assigned work as part of a larger system or activity.
- GoGo is a statically typed, compiled, general-purpose programming language with garbage collection and built-in support for concurrency.
- lockingLocking is a synchronization technique that controls concurrent access to a shared resource or protected operation.
Related concepts
- Apache Airflowpart of · Apache Airflow is an open-source platform for authoring, scheduling, and monitoring programmatically defined workflows, commonly represented as directed acyclic graphs.
- BEAMused by · BEAM is the virtual machine and runtime environment that executes Erlang-family programs and applications.
- microtask queueuses · A microtask queue is an event-loop queue that holds pending microtasks for processing after the current synchronous execution and before subsequent task processing.
- threaduses · A thread is a distinct sequence of execution with its own control flow and state, managed or scheduled by a processor, runtime, or operating system.
- agent orchestrationpart of · Agent orchestration is the coordination of agents, their tasks, tools, and interactions to accomplish a larger objective or workflow.
- preemptionused by · Preemption is the interruption or displacement of active execution or resource allocation so another eligible activity can proceed.
Questions about scheduler
What is scheduler?
A scheduler is a software component that selects, orders, or dispatches tasks for execution and may assign them execution resources according to defined policies or constraints.
Which Manning books explain scheduler?
Data Pipelines with Apache Airflow, Second Edition, Grokking Concurrency, Kubernetes in Action, Second Edition, Fast Python: High performance techniques for large datasets, Robotics for Programmers, Elixir in Action, Third Edition, Build and Orchestrator in Go (From Scratch), Spring in Action, Sixth Edition and 5 more.
What should I know before learning scheduler?
Manning books that teach scheduler usually introduce CPU, worker, Go and locking first.
What is scheduler part of?
Scheduler is part of Apache Airflow. Apache Airflow is an open-source platform for authoring, scheduling, and monitoring programmatically defined workflows, commonly represented as directed acyclic graphs.
These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.












