Chapter 9: Concurrency

Introduction

The chapter introduces the difference between concurrency and parallelism, explains why concurrency helps applications stay responsive, and previews Go’s tools for concurrent work.

• Concurrency versus parallelism: Distinguish managing progress on multiple tasks from performing multiple tasks at the same instant, using the chores example if helpful.
  Why it matters: The distinction clarifies what it means for a Go program to be concurrent.
• Responsive concurrent applications: Explain why managing multiple tasks concurrently can keep an application responsive instead of making it finish each task before starting the next.
  Why it matters: Responsiveness is a practical reason to consider concurrency.
• Go concurrency tools: Describe goroutines as a way to run functions concurrently and channels as a way for them to communicate safely, while the Go runtime handles scheduling and execution details.
  Why it matters: These are the basic tools the chapter builds on for concurrent programs.

Key terms: concurrency, parallelism, responsive, goroutines, channels, Go runtime, scheduling

9.1 When to use concurrency

This section uses an RSS reader to show when concurrency helps, how the go keyword starts concurrent work, and why that work needs reliable synchronization.

• Reasons to use concurrency: Explain when concurrency is worth its added complexity: independent tasks, responsiveness during long-running work, making progress while waiting for I/O, or using multiple CPU cores for CPU-bound work. State why a simple solution is a good starting point.
  Why it matters: Choosing concurrency for a specific benefit helps avoid complexity that makes bugs harder to investigate.
• Synchronous RSS fetch bottleneck: Describe how the initial RSS reader fetches a feed from a URL, waits for the network response, then parses it and prints its title. Explain why a slow response blocks further progress in that synchronous version.
  Why it matters: The blocking fetch provides a concrete reason to try concurrent execution.
• Goroutine launch options: Describe how placing the go keyword before a function call starts a goroutine that can run independently of the main flow. Explain that the feed-fetching code can be launched through an anonymous function or a named printFeedTitle(url string) function, and that launching one per URL lets multiple requests proceed without waiting for earlier ones.
  Why it matters: The RSS example shows the basic step that turns separate blocking fetches into concurrent work.
• Main lifetime and synchronization: Explain why launching the feed fetch in a goroutine initially prints nothing: main returns and the program exits before the goroutine finishes. Distinguish the example’s time.Sleep workaround from reliable synchronization that waits for the work to complete.
  Why it matters: Starting concurrent work is not enough if the program exits before that work finishes.

Key terms: concurrency, independent tasks, blocking I/O, parallelism, RSS feed, synchronous operation, gofeed, blocked, go keyword, goroutine

9.2 Sync package

Introduces the sync package as a way to coordinate goroutines and protect shared resources.

• Sync package purposes: Explain that when multiple goroutines need coordination, the standard library’s sync package can help coordinate task completion and synchronize access to shared resources.
  Why it matters: These two needs explain why the chapter introduces sync tools.

Key terms: goroutines, sync package, shared resources, synchronize, coordinate the completion of tasks

9.2.1 Wait groups

This section shows how a sync.WaitGroup reliably waits for concurrent feed-fetching goroutines to finish, demonstrates two ways to track them, and notes that a WaitGroup does not collect errors.

• WaitGroup completion tracking: Explain that a WaitGroup tracks active goroutines with a counter and blocks until the counter reaches zero, so the main function does not exit before every feed has been fetched and printed.
  Why it matters: Waiting for completion is more reliable than guessing how long concurrent work will take.
• Goroutine registration approaches: Describe the two approaches shown: using the newer WaitGroup.Go method, or incrementing the counter before launching each goroutine and deferring wg.Done() inside it. Explain why the Add-and-Done approach may appear in legacy code.
  Why it matters: Recognizing both approaches helps you follow newer examples and older Go codebases.
• WaitGroup error limitation: State that a WaitGroup waits for goroutines to finish but does not collect their errors, leaving the problem of reporting a failure while still waiting for the other goroutines.
  Why it matters: Completion and error reporting are separate concerns in concurrent code.

Key terms: WaitGroup, counter, goroutines, synchronization primitive, WaitGroup.Go, Add, defer wg.Done(), anonymous functions, error handling

9.2.2 Error Groups

This section shows how errgroup waits for concurrent tasks and reports their errors.

• Error group synchronization and reporting: Explain how to use an errgroup.Group instead of a WaitGroup: launch error-returning functions with the group's Go method, then call Wait to block until they all finish and receive the first non-nil error, if any.
  Why it matters: An error group keeps a program from exiting before concurrent work finishes while making task failures available to handle.

Key terms: errgroup.Group, WaitGroup, Go, Wait, non-nil error

9.2.3 Mutexes

This section explains how mutexes protect a shared article store from races while allowing concurrent reads when no write is in progress.

• Shared data race prevention: Explain that uncoordinated access to shared data by multiple goroutines can cause race conditions, and that a mutex protects a critical section by controlling access to it.
  Why it matters: This is why the shared article store needs synchronization.
• Read and write lock choices: Distinguish exclusive Lock/Unlock from RLock/RUnlock: changes require a write lock, while read-only operations can share read access when no writer holds the lock.
  Why it matters: Choosing the right lock keeps updates safe without unnecessarily serializing readers.
• Concurrent article store use: Describe how the ArticleStore encapsulates shared articles: feed goroutines add fetched articles through AddArticles, and the application retrieves recent articles through GetRecent after the goroutines finish. State that the store is designed to keep each article once in a date-sorted list and protect its internal data during concurrent use.
  Why it matters: The example shows how to put locking around shared data rather than leave coordination to every caller.

Key terms: race condition, mutex, critical section, mutual exclusion, RWMutex, Lock, Unlock, RLock, RUnlock, ArticleStore

9.3 Adding some context

The section adds contexts to feed fetching so downstream HTTP operations can time out and related fetches can be canceled together.

• Timeout for feed fetching: Explain how a context with a 60-second timeout prevents a feed fetch from waiting indefinitely when the RSS server connects but sends no data.
  Why it matters: A connection can succeed while the operation still hangs, so downstream work needs a way to stop.
• Context flow through operations: Describe contexts as temporary values that carry information such as a timeout or metadata to downstream functions, and explain why they are passed through the system rather than embedded in structs.
  Why it matters: Passing a context along lets downstream work act on the same request-specific information.
• Error group context cancellation: Explain how deriving a context when initializing the error group lets an error from one fetch cancel other ongoing operations for an all-or-nothing set of fetches.
  Why it matters: There is no need to keep fetching the remaining feeds when the group must fail as a whole.

Key terms: context, timeout, cancel function, context package, Context interface, metadata, downstream, temporary values, derived context, error group

9.4 Channels

This section introduces channels as a way for goroutines to pass values and coordinate work, then applies them to a feed-fetching application with buffering, error handling, and select.

• Channel communication between goroutines: Explain that Go channels are typed conduits, similar to queues, through which goroutines send and receive values with the <- operator. Describe how passing values helps coordinate concurrent work without requiring explicit locking for that communication.
  Why it matters: Channels let concurrent parts of a program communicate as work becomes available.
• Channel values and consumer lifetime: Describe how channels created with make can be returned from functions, passed as arguments, or stored in structs. Trace the article example: a producer sends fetched articles to a channel, while a saving goroutine receives them as they arrive and its range loop ends after the channel is closed and drained.
  Why it matters: Passing channels as values separates the code that produces articles from the code that saves them.
• Directional channel type safety: Distinguish read-only, write-only, and bidirectional channel types. Explain that restricting a function to the direction it needs, such as receiving articles in the saving function, prevents accidental sends or receives in the wrong part of the application.
  Why it matters: Channel direction makes each component’s role in the data flow clearer and harder to misuse.
• Buffered channel capacity and blocking: Explain that channel capacity controls how many values can wait in the channel before another send blocks. Contrast the section’s capacity-one article channel, where a second send waits for the first value to be received, with a larger buffer that lets the producer queue more articles independently of the consumer’s immediate pace.
  Why it matters: Buffer size affects when producers must wait for consumers.
• Application channels and error flow: Describe how the App struct brings together the memory store, feed-fetching work, and channels for background ingestion. Explain that a separate errChan carries ingestion errors for handling or logging elsewhere, while an HTTP handler serves recent saved articles as JSON so the application’s results can be checked.
  Why it matters: The example shows how separate flows for articles and errors fit into an application that can expose its results.
• Select across channel events: Explain that select waits on multiple channel operations and proceeds with one that is ready, allowing an event loop to respond to work, errors, or cancellation without waiting exclusively on one channel. Mention that a default case permits nonblocking behavior.
  Why it matters: Select lets one goroutine respond to whichever of several communication events is ready.

Key terms: communicating sequential processes (CSP), channels, goroutines, <- operator, make, channel, producer, consumer, range, close

9.5 Concurrency patterns

This section introduces worker pools and fan-out/fan-in as patterns for improving throughput while keeping concurrent work coordinated.

• Worker pool task processing: Explain how a worker pool uses a fixed number of goroutines to take tasks from a shared channel, processing many independent jobs without allowing parallelism to grow without bound.
  Why it matters: A worker pool helps an application handle bursts of work without exhausting resources.
• Fan-out and fan-in roles: Distinguish fan-out, which launches goroutines to do work in parallel, from fan-in, which collects their results and coordinates aggregation, errors, or combined data.
  Why it matters: Separating parallel work from result collection helps improve responsiveness without losing control of the outcome.

Key terms: worker pool, worker goroutines, shared queue, channel, parallelism, fan-out, fan-in, goroutines, results

9.5.1 Workers

This section builds a background feed-ingestion worker pool that uses a shared channel to distribute work among concurrent workers.

• Background feed worker: Describe how the application treats a channel as a queue of incoming feed URLs and extends App into a worker that ingests feeds in the background.
  Why it matters: A single worker establishes the pattern that the pool expands.
• Shared worker pool queue: Explain how a WorkerPool creates a configurable number of workers that share one buffered channel of feed URLs, so the next available worker can take queued work.
  Why it matters: The shared queue lets multiple workers process feeds concurrently without assigning each URL to a particular worker.
• Application worker pool integration: Trace how App incorporates the WorkerPool: its run loop submits incoming feed URLs to the pool, whose background worker goroutines perform ingestion. Explain that separating submission from execution improves throughput and API responsiveness.
  Why it matters: This connects the worker-pool design to the application’s feed-ingestion flow.

Key terms: worker, channel, queue, App struct, WorkerPool, buffered channel, shared queue, pool size, run loop, worker goroutines

9.5.2 Fan-out and fan-in

This section uses fan-out/fan-in to refresh multiple feeds concurrently and return their combined articles and errors in one response.

• Parallel feed fetches: Explain how fan-out launches a separate goroutine for each feed URL so the feeds can be fetched in parallel, with each goroutine sending its result or error through a shared channel.
  Why it matters: Fan-out lets the handler attempt every feed fetch without waiting for one to finish before starting another.
• Complete result aggregation: Trace fan-in from receiving results through the shared channel to using a WaitGroup to detect when all fetches finish and then closing the channel. Explain that ranging over the channel collects articles and errors as they arrive, regardless of completion order, and finishes before the response is sent.
  Why it matters: Coordinating channel closure with completion lets the handler collect every available outcome before replying.
• Synchronous refresh endpoint: Describe the endpoint that triggers fetching new feed results, saves articles, and replies to the caller with aggregated articles and any errors. State that the example changes fetchFeeds to return data and publish messages on the channel, then adds the new handler to the API.
  Why it matters: The endpoint shows how a concurrent refresh can still provide the caller with a complete reply.
• Concurrent code complexity: Explain that adding modules and concurrency patterns made the example harder to reason about, and that focused modules, clear communication, and careful channel and synchronization management help control risks such as races, deadlocks, and resource leaks.
  Why it matters: Concurrency patterns are useful only if the resulting code remains safe and maintainable.

Key terms: fan-out, goroutine, results channel, fan-in, WaitGroup, for-range loop, fetchFeeds, handler, articles, errors

9.6 Helpful concurrency tips

This introduction frames Go concurrency as useful but potentially risky and previews guidance on safe practices and when to use it.

• Concurrency benefits and risks: Explain that Go concurrency can improve performance and responsiveness, but easy-to-launch goroutines and channels do not eliminate risks such as race conditions, deadlocks, and resource leaks. Describe how the following sections address best practices, good use cases, and cases where concurrency is not the right choice.
  Why it matters: Recognizing both the benefits and risks helps you decide when and how to use concurrency safely.

Key terms: goroutines, channels, race conditions, deadlocks, resource leaks, best practices

9.6.1 Best practices for Go concurrency

This section gives practical guidelines for writing clear, safe, and manageable concurrent Go code.

• Simple and safe data sharing: Explain why concurrent code should stay straightforward and how channels and synchronization primitives help avoid races when goroutines exchange or share data.
  Why it matters: Simple code and deliberate synchronization make concurrency easier to debug and safer to change.
• Cancellation and clean channel closure: Describe using context.Context for cancellation, deadlines, and request-scoped values, and closing channels when they are no longer needed so goroutines can exit cleanly.
  Why it matters: Managing cancellation and channel closure helps prevent work from continuing unnecessarily or leaking resources.
• Bounded and tracked goroutines: Explain why goroutine counts should be limited and how worker pools or errgroup.SetLimit can bound concurrency, while WaitGroup or errgroup tracks goroutines instead of relying on a bare go statement.
  Why it matters: Bounding and tracking goroutines keeps workloads within system capacity and makes concurrent work manageable.
• Concurrent errors and race testing: Describe explicitly capturing and propagating errors from concurrent operations, and using go test -race during development to catch race conditions.
  Why it matters: Errors must reach code that can handle them, while race testing can expose unsafe access before production.

Key terms: channels, shared memory, sync.Mutex, sync.RWMutex, race conditions, context.Context, cancellation, deadlines, request-scoped values, close channels

9.6.2 When to use concurrency in Go

This section identifies situations where independent or parallel work can make Go applications faster or more responsive.

• Independent work as a criterion: Explain that concurrency is useful when tasks can run independently or in parallel, improving performance or responsiveness by letting work overlap.
  Why it matters: This criterion helps you decide whether concurrency fits a particular workload.
• Responsive requests and background work: Describe how concurrent handling of network requests and I/O operations can improve throughput or reduce waiting, while background goroutines can keep tasks such as emails or scheduled jobs from blocking the main flow. Relate the same responsiveness benefit to user interfaces.
  Why it matters: These examples show how concurrency keeps an application responsive while other work proceeds.
• Pipelines pools and parallel workloads: Describe how concurrent pipeline stages can pass data through channels, worker pools can distribute queued jobs while controlling parallelism, and independent streams, devices, or CPU-bound computations can be handled concurrently.
  Why it matters: These examples show different ways to organize independent work, not just launch unrelated tasks.

Key terms: concurrency, independently, in parallel, responsiveness, goroutine, I/O-bound operations, background tasks, throughput, responsive user interfaces, data pipelines

9.6.3 When not to use concurrency in Go

This section explains when Go concurrency is a poor fit and recommends starting with a sequential solution before adding it where needed.

• Cases favoring sequential execution: Describe why simple or single-task programs, strictly ordered operations, limited concurrency experience, and tasks too small to justify coordination overhead may be better handled sequentially.
  Why it matters: Recognizing these cases helps you avoid complexity or bugs without a useful payoff.
• Sequential-first concurrency decision: Explain the recommendation to begin with a straightforward sequential solution and introduce concurrency only when needed, balancing Go’s ability to handle multiple tasks against race conditions and added complexity.
  Why it matters: Starting simply helps keep code maintainable while leaving room to use concurrency when it provides a real benefit.

Key terms: sequential approach, synchronization, race conditions, goroutines, channels, sequential solution

9.7 Summary

Summarizes Go’s concurrency tools, common patterns, and when concurrency is worth using.