Caching
Caching is the practice of storing reusable data or computation results so later access can avoid repeating retrieval or computation.
Building Web APIs with ASP.NET Core
Caching is the temporary storage of reusable responses or data so subsequent requests can be served without repeating the original retrieval or computation.
- How it is used
- Reduces repeated response generation and database retrieval for recurring web API requests.
- How deep it goes
- Works through response directives, middleware setup, and a keyed in-memory query cache; introduces distributed caching as the multi-server alternative.
- Examples
- Uses board-game GET responses, an EF Core query cached by request parameters, browser header inspection, and breakpoint checks.
- What you can do after
- Operate/configure it with ASP.NET Core response-caching middleware and IMemoryCache, then verify hits and misses on a board-game endpoint.
Read how Building Web APIs with ASP.NET Core explains caching →
Learn Go with Pocket-Sized Projects
A cache is a data store that retains reusable results or data to reduce access time or computational cost.
- How it is used
- Uses cached values to avoid repeated costly retrievals when an earlier result remains acceptable.
- How deep it goes
- Builds a generic map cache, then elaborates concurrency, expiration, and capacity decisions; presents a dated-file cache as a proposal.
- Examples
- Uses repeated currency-conversion requests, generic key-value operations, and a capacity test where updating a key changes which entry is evicted.
- What you can do after
- Build it from scratch as a Go key-value cache with lookup, updates, deletion, expiration, concurrency protection, and bounded size.
Read how Learn Go with Pocket-Sized Projects explains caching →
Advanced Algorithms and Data Structures
Caching is the technique of storing retrieved data or computed results in faster storage for possible reuse.
- How it is used
- Avoids repeated fetching and computation, particularly in web applications where cached results can reduce latency and downstream load.
- How deep it goes
- Develops caching from remembered results through capacity and replacement choices, then considers deployment and concurrency rather than treating it as one container.
- Examples
- Uses a social-network sentiment service, cached database work, and application-scale memory demands to show where reuse helps and capacity matters.
- What you can do after
- Evaluate/understand when to cache reusable results and how capacity, eviction, placement, and shared access affect the design.
Read how Advanced Algorithms and Data Structures explains caching →
Pro ASP.NET Core 7, Tenth Edition
Caching is the technique of storing generated data or content so it can be reused without being regenerated for every request.
- How it is used
- Reuses expensive data values, entire responses, or rendered fragments instead of generating them for every request.
- How deep it goes
- Works through service registration, middleware placement, endpoint policies, expiry controls, and browser checks; stops short of implementing persistent output-cache storage.
- Examples
- Uses a timestamped sum endpoint, a 30-second controller policy, and cached Razor timestamps to make reuse observable.
- What you can do after
- Operate/configure it with ASP.NET Core cache services, middleware, endpoint policies, or tag helpers, then verify reuse and expiry.
Read how Pro ASP.NET Core 7, Tenth Edition explains caching →
Amazon Web Services in Action, Third Edition: An in-depth guide to AWS
A caching layer is a software layer that stores frequently accessed data in faster temporary storage to reduce access time and load on a primary data store.
- How it is used
- Places an in-memory layer between applications and primary data stores to serve frequent reads with less latency and database load.
- How deep it goes
- Explains freshness, eviction, a Redis CLI exercise, monitoring metrics, and scaling choices without detailing application-side cache integration.
- Examples
- Uses a game leaderboard, expiring gamer session, Redis high-score commands, and a Discourse deployment backed by PostgreSQL.
- What you can do after
- Operate/configure it with ElastiCache and Redis commands, then diagnose cache pressure through CloudWatch metrics and scaling options.
Read how Amazon Web Services in Action, Third Edition: An in-depth guide to AWS explains caching →
Latency
Caching is a technique that stores temporary copies of data closer to where it is accessed to reduce data-retrieval delay.
- How it is used
- Uses temporary nearby copies to reduce data-access latency when backing-store retrieval is slower or full replication is impractical.
- How deep it goes
- Works through hit and miss behavior, strategies, freshness, replacement, and an implemented cache-aside API with latency comparison.
- Examples
- Uses user profiles, sessions, product catalogs, and a PostgreSQL-backed REST API with a Moka cache.
- What you can do after
- Apply it via Moka to cache PostgreSQL query results in a REST API, then compare latency with and without the cache.
Java Persistence with Spring Data and Hibernate epub
Caching is the retention of data in a faster-access storage layer so subsequent reads can reuse it.
- How it is used
- Supports entity access and change tracking during a unit of work while making cache size a practical memory concern.
- How deep it goes
- Explains snapshots, memory growth, eviction and read-only controls; does not detail a separate application-tier cache implementation.
- Examples
- Shows EntityManager detach and clear operations, Hibernate read-only settings, and an entity changed without a database update at flush.
- What you can do after
- Operate/configure it to limit persistence-context memory use and prevent dirty checking for selected read-only entities.
Read how Java Persistence with Spring Data and Hibernate epub explains caching →
Generative AI in Action
Caching is the practice of storing reusable data or results in faster-access storage so they can be returned without repeating the original computation or request.
- How it is used
- Uses caching to avoid repeated LLM calls and improve latency and cost for frequently requested or rarely changing information.
- How deep it goes
- Explains cache-aside flow, keys, TTL, hit-rate monitoring, and failure handling, then illustrates configuration without a complete production implementation.
- Examples
- Demonstrates a RedisVL semantic cache for Azure OpenAI responses and reports cache hits, misses, and comparative response times.
- What you can do after
- Apply it via RedisVL to cache selected LLM responses while planning invalidation, monitoring, and cache-failure behavior.
Build AI-Enhanced Web Apps
The technique of storing computed or retrieved data temporarily so that later requests can return it without repeating the original work.
- How it is used
- Reduces latency and repeated model work while allowing changing API responses to bypass caching.
- How deep it goes
- Shows Next.js route and fetch settings and a Redis feedback key, but does not specify cache invalidation or Redis cache implementation.
- Examples
- Uses a dynamic chat API route, per-fetch revalidation options, and Redis-cached interview feedback.
- What you can do after
- Apply it via Next.js route and fetch settings, or use Redis to avoid regenerating session feedback.
React Hooks in Action: With Suspense and Concurrent Mode
A cache is temporary storage of previously retrieved data kept for faster access and synchronization with its source.
- How it is used
- Improves responsive React data-fetching experiences by reusing server data and synchronizing newer results in the background.
- How deep it goes
- Explains key-based retrieval, background refresh, and direct access, while deferring detailed stale-data timing.
- Examples
- Demonstrates shared bookables data, parameterized bookings queries, and an edit form initialized from cached bookables.
- What you can do after
- Apply it via React Query to share fetched data, reduce loading states, define query keys, and initialize components from cached results.
Read how React Hooks in Action: With Suspense and Concurrent Mode explains caching →
The Art of Code
A cache is a storage layer that retains recently or frequently used results for faster retrieval and reduced dependency load.
- How it is used
- Reduces remote calls and external-dependency load while supporting efficient, sustainable application behavior.
- How deep it goes
- Explains cache tiers, expiration choices, and memory-retention tradeoffs without detailing eviction algorithms or implementation internals.
- Examples
- Illustrates browser or intermediary HTTP reuse and a Java map that loads a user profile only on a cache miss.
- What you can do after
- Apply it by selecting an appropriate cache tier, setting expiration rules, and limiting unnecessary retained data.
ASP.NET Core in Action, 3e
A cache is local storage of previously retrieved data that allows subsequent requests to reuse the stored response without contacting the origin server.
- How it is used
- Uses caching to reduce browser requests, server work, or repeated calls to third-party services.
- How deep it goes
- Explains response-versus-output behavior and implements scheduled background updates, while deferring detailed invalidation configuration.
- Examples
- Uses HTTP caching headers, server-stored generated responses, and a scheduled exchange-rate cache as concrete examples.
- What you can do after
- Operate/configure ASP.NET Core caching and build a BackgroundService that refreshes shared data for fast API responses.
Clojure: The Essential Reference
A cache is a storage structure that retains previously computed results for retrieval by associated keys or repeated access.
- How it is used
- Uses caching to avoid repeated function computations and to retain realized sequence results.
- How deep it goes
- Explains argument-keyed memoization through its contract and examples, then details how lazy realization affects sequence memory.
- Examples
- Shows repeated addition calls, cached spelling-distance rankings, and a lazy sequence viewing an updated Java list.
- What you can do after
- Apply it via `memoize` to reuse results, and evaluate when retained lazy-sequence elements increase memory use.
Read how Clojure: The Essential Reference explains caching →
The Design of Web APIs, Second Edition
Caching is the temporary storage of data or responses so that subsequent requests can be served without repeating the original retrieval or computation.
- How it is used
- Improves API efficiency by reusing responses and retrieving representations only when they have changed.
- How deep it goes
- Explains policy choices, Cache-Control directives, cache keys, ETags, conditional requests, and 304 responses.
- Examples
- Uses banking account data, ten-minute freshness, Cache-Control, ETag, If-None-Match, and 304 Not Modified.
- What you can do after
- Operate/configure HTTP caches with explicit policies and conditional requests that limit stale data and unnecessary transfers.
Read how The Design of Web APIs, Second Edition explains caching →
Learn Docker in a Month of Lunches, Second Edition
A cache is a temporary storage area containing previously retrieved data so that repeated requests can be served more quickly.
- How it is used
- Improves reverse-proxy performance and reliability by serving repeated responses without contacting application containers.
- How deep it goes
- Explains cache hits, freshness periods, stale fallbacks, and nginx directives, but does not detail cache storage internals.
- Examples
- Uses an image gallery, API responses, curl headers, nginx configuration, and a pi-calculation application.
- What you can do after
- Operate/configure nginx as a caching reverse proxy and verify misses, hits, freshness, and resilience with curl.
Read how Learn Docker in a Month of Lunches, Second Edition explains caching →
React in Depth
A cache is a stored copy of retrieved or computed data that can be returned without obtaining or calculating it again.
- How it is used
- Uses cached server results to reduce request delays and keep remote-dependent React views responsive.
- How deep it goes
- Works through query invalidation, mutation-driven updates, and partial initial data in hooks; leaves freshness policy details unspecified.
- Examples
- Uses customer deletion, an updated detail record, and cached list fields that seed a detail view.
- What you can do after
- Apply it via TanStack Query to invalidate affected results, update cached queries after mutations, and prefill detail views.
Machine Learning System Design
A storage mechanism that keeps copies of data or results so repeated requests can be served without recomputation or retrieval from the original source.
- How it is used
- Reduces inference latency, server load, and compute cost by reusing results for repeated or similar requests.
- How deep it goes
- Explains key-value, TTL, runtime updates, and fuzzy matching, but does not develop cache invalidation or consistency mechanics.
- Examples
- Uses an in-memory get-or-compute cache, semantic or regex-based fuzzy keys, and a PhotoStock ElastiCache design.
- What you can do after
- Apply it via AWS ElastiCache, Redis, or Memcached to reduce repeated runtime inference work.
Functional Programming in JavaScript
An intermediary repository that stores previously computed or retrieved values for later reuse.
- How it is used
- Speeds applications by reusing results of expensive or repetitive operations.
- How deep it goes
- Explains cache keys, hits, misses, wrappers, and performance effects, but does not detail cache eviction or invalidation.
- Examples
- Uses cached student lookup, memoized ROT13, composed student display, and recursive factorial examples.
- What you can do after
- Apply memoization to reduce repeated computation in JavaScript functions.
Read how Functional Programming in JavaScript explains caching →
C# Concurrency: Asynchronous and multithreaded programming
A cache is a storage layer that retains previously obtained values so they can be returned more quickly on later requests.
- How it is used
- Speeds repeated long-running operations by returning stored results and coordinating access to shared cached state.
- How deep it goes
- Explains lock placement, asynchronous races, compiler reordering, and CPU cache memory, but does not develop eviction or expiration policies.
- Examples
- Uses a C# Dictionary cache around an asynchronous HttpClient request and contrasts whole-method locking with split lock boundaries.
- What you can do after
- Apply it via C# Dictionary, lock, and async/await while recognizing duplicate misses and unsafe unsynchronized memory access.
Read how C# Concurrency: Asynchronous and multithreaded programming explains caching →
Building Reliable AI Systems
Storing results so they can be reused instead of recomputed or retrieved again.
- How it is used
- Reduces redundant LLM calls, latency, and cost when users repeat or rephrase queries.
- How deep it goes
- Works through exact matching and an embedding-based cache, including similarity lookup, fallback, and storage.
- Examples
- Uses store-hours and refund-policy questions alongside Python cache lookups and query-handling code.
- What you can do after
- Apply it via FAISS or GPTCache to reuse answers and call the LLM on cache misses.
Architecting an Apache Iceberg Lakehouse
Caching is the technique of storing reusable data or computation results in a faster-access location to reduce subsequent processing or retrieval cost.
- How it is used
- Reduces repeated remote retrieval and processing across lakehouse queries, dashboards, machine-learning pipelines, and AI workloads.
- How deep it goes
- Explains caching through engine behavior, workload patterns, and tradeoffs but stops short of cache implementation details or invalidation policies.
- Examples
- Uses Dremio reflections, interactive dashboards, GPU training pipelines, regulatory reports, and AI-agent queries as workload examples.
- What you can do after
- Apply it via Dremio and lakehouse query engines to reduce redundant scans and improve latency for governed consumption workloads.
Read how Architecting an Apache Iceberg Lakehouse explains caching →
API Design Patterns
A cache is a stored copy of data retained for faster or more convenient access than retrieving the original data again.
- How it is used
- Uses cached copies to reduce extra API calls and deduplicate repeated requests.
- How deep it goes
- Explains references, embedded copies, response replay, storage costs, expiration, invalidation, and collision safeguards.
- Examples
- Demonstrates book-author resources, repeated non-idempotent requests, large batch responses, and pagination result caches.
- What you can do after
- Apply it via request identifiers, response caches, embedded resources, and explicit freshness policies.
AI Agents in Action, Second Edition
- How it is used
- Uses caching to reduce repeated agent-tool work, latency, and production costs.
- How deep it goes
- Explains idempotency, gives an input-keyed cache example, and flags invalidation; it does not detail cache infrastructure.
- Examples
- Contrasts weather lookups with email sending and demonstrates cached tool results and repeated research queries.
- What you can do after
- Apply caching to idempotent tool calls and repeated queries, and clear caches to limit stale results.
Read how AI Agents in Action, Second Edition explains caching →
Modern C, Third Edition
A cache is storage that retains previously computed values so they can be reused instead of recomputed.
- How it is used
- Uses retained results to eliminate repeated recursive computation and improve algorithmic performance.
- How deep it goes
- Explains cache lookup, storage, initialization, and linear-time behavior, but stops short of concrete performance measurements.
- Examples
- Demonstrates caching through recursive Fibonacci computation using an array of previously computed values.
- What you can do after
- Build it from scratch by adding an array cache to recursive Fibonacci and reasoning about its linear execution time.
Data Pipelines with Apache Airflow, Second Edition
Caching is the technique of storing computed or retrieved objects so later requests can reuse them without repeating the original work.
- How it is used
- Supports efficient Airflow reruns and avoids repeated metastore lookups when hooks reuse sessions or workflows reuse intermediate datasets.
- How deep it goes
- Shows instance-level implementation and pipeline retention trade-offs, but does not specify a general cache invalidation strategy.
- Examples
- Uses a cached API session and base URL, plus cloud-stored fetch and preprocess outputs.
- What you can do after
- Apply it via Airflow hooks and shared filesystems to reduce repeated work and rerun selected tasks after failures.
Read how Data Pipelines with Apache Airflow, Second Edition explains caching →
Modern C
A cache is a storage structure containing previously computed values for reuse.
- How it is used
- Uses cached values to eliminate repeated work in recursive Fibonacci computation.
- How deep it goes
- Explains array initialization, lookup, assignment, storage tradeoffs, and complexity improvement, while deferring implementation details and measurements.
- Examples
- Demonstrates caching through fibCacheRec, fibCache, repeated fib(2) calls, and a variable-length array.
- What you can do after
- Build it from scratch by adding an initialized array cache to recursive Fibonacci computation.
Build Python Web Apps with Streamlit
Caching is the storage of computed results or loaded data so they can be reused without repeating the underlying work.
- How it is used
- Improves Streamlit dashboard performance by reusing loaded data during interactions, reruns, and page refreshes.
- How deep it goes
- Explains cached function results, decorators, refresh behavior, TTL, and loading feedback, but does not detail cache internals.
- Examples
- Uses a Streamlit load_data function that reads a sales CSV into a pandas DataFrame and displays spinner feedback.
- What you can do after
- Apply caching via Streamlit's st.cache_data decorator to reduce repeated data-loading work.
Read how Build Python Web Apps with Streamlit explains caching →
Authorization in Action
A cache is a temporary data store that retains copies of information for faster retrieval or reduced dependence on an origin system.
- How it is used
- Balances authorization latency and upstream dependence against the assurance requirements of signal freshness.
- How deep it goes
- Explains cache placement, invalidation, blast radius, TTL trade-offs, and signal-specific durations without detailing cache algorithms.
- Examples
- Uses on-call status with a five-minute TTL and employment status with a 24-hour TTL.
- What you can do after
- Operate/configure authorization caches by selecting placement, invalidation behavior, and TTLs for different signal risks.
GitHub Actions in Action: Continuous integration and delivery for DevOps
Caching is the practice of storing reusable data or outputs so they can be retrieved without repeating their computation or acquisition.
- How it is used
- Speeds GitHub Actions workflows by reusing dependencies and other frequently used files on fresh job machines.
- How deep it goes
- Explains cache actions, paths, keys, lockfile hashing, setup actions, and Docker cache configuration.
- Examples
- Uses npm setup-node caching, a NuGet package-path key, Docker cache settings, and shared artifact cache keys.
- What you can do after
- Apply it via GitHub Actions setup actions or actions/cache with paths and dependency-sensitive keys.
Read how GitHub Actions in Action: Continuous integration and delivery for DevOps explains caching →
Machine Learning Platform Engineering
A caching strategy is a method for storing and reusing computational results to avoid repeating expensive processing.
- How it is used
- Reduces repeated LLM processing, token consumption, latency, and operational cost in RAG and prompt-management systems.
- How deep it goes
- Distinguishes context, semantic, and client-side caching, explains trade-offs, and demonstrates vector-similarity implementation.
- Examples
- Uses documentation sections, FAQ systems, rephrased API-key questions, RedisVL vector queries, and Langfuse-managed prompts.
- What you can do after
- Apply caching with RedisVL vector similarity or Langfuse client-side prompt caching while accounting for reuse, freshness, and storage costs.
Read how Machine Learning Platform Engineering explains caching →
Snowflake Data Engineering
Data caching is the retention of previously accessed data in faster storage or memory so subsequent operations can retrieve it more efficiently.
- How it is used
- Improves Snowflake query performance by reusing query results, metadata, or warehouse-resident table data.
- How deep it goes
- Defines caching and maps three cache types to Snowflake architecture layers, focusing on behavior rather than cache-management internals.
- Examples
- Uses repeated queries, metadata-only queries, and subsequent queries against the same virtual warehouse as caching examples.
- What you can do after
- Evaluate/understand Snowflake query-performance behavior by distinguishing cache types and locating caching in the query profile.
The Quick Python Book, Fourth Edition
A cache is stored data containing previously computed results so that later requests can reuse them without repeating the computation.
- How it is used
- Speeds repeated calls to expensive functions by reusing results for previously seen argument combinations.
- How deep it goes
- Demonstrates in-memory lookup and disk persistence, but leaves concurrency handling unresolved.
- Examples
- Uses the sole calculation, repeated argument triples, a spreadsheet-cell exercise, and a pickled cache file.
- What you can do after
- Apply it via Python dictionaries and pickle to avoid repeated calculations within and across program sessions.
Read how The Quick Python Book, Fourth Edition explains caching →
Deep Learning with Python, Third Edition
- How it is used
- Accelerates repeated work during autoregressive generation and reduces repeated dataset reads during TPU training.
- How deep it goes
- Explains why past attention keys and values remain reusable and gives dataset.cache() guidance, but does not show a key/value-cache implementation.
- Examples
- Uses repeated mini-GPT next-token prediction and a small GCS-resident dataset cached with dataset.cache().
- What you can do after
- Apply caching with dataset.cache() to keep a small dataset in virtual-machine memory and read it from GCS once.
Read how Deep Learning with Python, Third Edition explains caching →
Deep Learning with PyTorch, Second Edition
The practice of storing preprocessed data temporarily so it can be reused efficiently later instead of recomputing it.
- How it is used
- Speeds data loading when parsing, disk access, conversion, or cropping would otherwise repeat for every sample.
- How deep it goes
- Explains memory and disk layers, memoization, invalidation, and augmentation ordering, but defers the full rationale for the setup.
- Examples
- Uses cached CT objects and extracted candidate arrays, then places caching before affine image augmentation.
- What you can do after
- Operate/configure memory and disk caches around CT-loading functions while preserving correct augmentation and invalidating stale results.
Read how Deep Learning with PyTorch, Second Edition explains caching →
LLMs in Production
Caching is the technique of storing reusable computation results or data so later requests can be served without repeating the original work.
- How it is used
- Improves LLM-service latency by avoiding inference for repeated, normalized requests.
- How deep it goes
- Recommends caching conditionally but stops short of specifying cache keys, invalidation, storage, or implementation.
- Examples
- Uses repeated user requests with easily normalized inputs as the motivating example.
- What you can do after
- Apply caching when request repetition and normalization make avoided inference likely to improve service speed.
Natural Language Processing in Action, Second Edition
Caching is the practice of storing computed results so repeated requests can be served without recomputing them.
- How it is used
- Speeds Streamlit question-answering applications by reusing loaded models and avoiding repeated expensive initialization.
- How deep it goes
- Explains input hashing, pickle-backed outputs, and compatibility limits, then demonstrates resource decorators without covering cache invalidation.
- Examples
- Uses cached FAISS document-store loading and cached embedding-retriever and reader construction in a Streamlit QA app.
- What you can do after
- Operate/configure Streamlit applications with st.cache_resource around expensive model-loading functions.
Read how Natural Language Processing in Action, Second Edition explains caching →
Financial AI in Practice
A cache is a storage layer that retains previously computed or retrieved results for faster reuse.
- How it is used
- Serves stored answers to repeated financial AI queries before slower, more expensive processing.
- How deep it goes
- Defines cached-response serving and its latency and cost benefits, but gives no cache implementation mechanics.
- Examples
- Uses mortgage-rate FAQs and popular stock summaries as examples of repeat queries that can receive cached responses.
- What you can do after
- Apply caching to repeated financial queries by serving stored responses instead of rerunning expensive LLM pipelines.
Building LLM Applications with DSPy
Caching is the practice of storing a language model provider's response so repeated identical requests can be served without querying the provider.
- How it is used
- Uses cached language-model responses to avoid repeated provider queries for identical inputs.
- How deep it goes
- Describes response reuse and LM-level and global controls without detailing cache internals.
- Examples
- Shows DSPy generating a poem twice from the same question after global caching is disabled.
- What you can do after
- Apply caching controls in DSPy by setting cache options on an LM and globally.
Read how Building LLM Applications with DSPy explains caching →
Related concepts
- DSPyused by · DSPy is a Python framework for declaratively programming, composing, evaluating, and optimizing language-model applications and pipelines.
- memoizationused by · Memoization is an optimization technique that stores computation results with their inputs so equivalent later evaluations can reuse them without recomputation.
- meanaddresses · A mean is a measure of central tendency, calculated as the arithmetic average of values or, for a random variable, as the probability-weighted average of its possible values.
These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.