Caching
Caching is the temporary storage of data or computed results for reuse, reducing the work or delay of retrieving or computing them again.
Building Web APIs with ASP.NET Core
Caching is the temporary storage of reusable responses or data so subsequent requests can be served without repeating the original retrieval or computation.
- How it is used
- Reduces repeated response generation and database retrieval for recurring web API requests.
- How deep it goes
- Works through response directives, middleware setup, and a keyed in-memory query cache; introduces distributed caching as the multi-server alternative.
- Examples
- Uses board-game GET responses, an EF Core query cached by request parameters, browser header inspection, and breakpoint checks.
- What you can do after
- Operate/configure it with ASP.NET Core response-caching middleware and IMemoryCache, then verify hits and misses on a board-game endpoint.
Read how Building Web APIs with ASP.NET Core explains caching →
Learn Go with Pocket-Sized Projects
A cache is a data store that retains reusable results or data to reduce access time or computational cost.
- How it is used
- Uses caching to avoid repeated expensive retrievals or computations when previously obtained values remain acceptable.
- How deep it goes
- Builds a generic cache from basic map operations through concurrency protection, expiration, and bounded-capacity eviction.
- Examples
- Uses exchange-rate files and a capacity-three cache whose oldest entry is evicted after an update.
- What you can do after
- Build a generic cache from scratch with storage operations, concurrency protection, expiration, and a capacity limit.
Read how Learn Go with Pocket-Sized Projects explains caching →
Advanced Algorithms and Data Structures
Caching is the technique of storing retrieved data or computed results in faster storage for possible reuse.
- How it is used
- Avoids repeated fetching and computation, particularly in web applications where cached results can reduce latency and downstream load.
- How deep it goes
- Develops caching from remembered results through capacity and replacement choices, then considers deployment and concurrency rather than treating it as one container.
- Examples
- Uses a social-network sentiment service, cached database work, and application-scale memory demands to show where reuse helps and capacity matters.
- What you can do after
- Evaluate/understand when to cache reusable results and how capacity, eviction, placement, and shared access affect the design.
Read how Advanced Algorithms and Data Structures explains caching →
Pro ASP.NET Core 7, Tenth Edition
Caching is the technique of storing generated data or content so it can be reused without being regenerated for every request.
- How it is used
- Reuses expensive data values, entire responses, or rendered fragments instead of generating them for every request.
- How deep it goes
- Works through service registration, middleware placement, endpoint policies, expiry controls, and browser checks; stops short of implementing persistent output-cache storage.
- Examples
- Uses a timestamped sum endpoint, a 30-second controller policy, and cached Razor timestamps to make reuse observable.
- What you can do after
- Operate/configure it with ASP.NET Core cache services, middleware, endpoint policies, or tag helpers, then verify reuse and expiry.
Read how Pro ASP.NET Core 7, Tenth Edition explains caching →
Amazon Web Services in Action, Third Edition: An in-depth guide to AWS
A caching layer is a software layer that stores frequently accessed data in faster temporary storage to reduce access time and load on a primary data store.
- How it is used
- Places an in-memory layer between applications and primary data stores to serve frequent reads with less latency and database load.
- How deep it goes
- Explains freshness, eviction, a Redis CLI exercise, monitoring metrics, and scaling choices without detailing application-side cache integration.
- Examples
- Uses a game leaderboard, expiring gamer session, Redis high-score commands, and a Discourse deployment backed by PostgreSQL.
- What you can do after
- Operate/configure it with ElastiCache and Redis commands, then diagnose cache pressure through CloudWatch metrics and scaling options.
Read how Amazon Web Services in Action, Third Edition: An in-depth guide to AWS explains caching →
Latency
Caching is a technique that stores temporary copies of data closer to where it is accessed to reduce data-retrieval delay.
- How it is used
- Uses caching to reduce data-access latency when key–value access is sufficient or changing the backing system is impractical.
- How deep it goes
- Explains cache strategies, freshness, replacement, and trade-offs, then implements and benchmarks an in-application cache.
- Examples
- Uses user profiles, session state, a product catalog, and a cached PostgreSQL query result as examples.
- What you can do after
- Apply it via Moka to a Rust REST API and compare latency against PostgreSQL without caching.
Vibe Engineering
Caching is the storage of previously computed or retrieved results so that later equivalent requests can be served with less work or cost.
- How it is used
- Uses caching to reduce repeated word-lookup work and share build artifacts across agents.
- How deep it goes
- Compares eager and lazy loading, eviction, memory, startup, and latency; evaluates alternatives with benchmarks and load tests.
- Examples
- Compares dictionary word-existence caches and illustrates agents sharing a compiled dependency through a remote cache.
- What you can do after
- Evaluate caching alternatives using workload assumptions, memory costs, startup time, and latency measurements.
Java Persistence with Spring Data and Hibernate epub
Caching is the retention of data in a faster-access storage layer so subsequent reads can reuse it.
- How it is used
- Uses persistence-context caching to retain managed entities and support application-tier performance.
- How deep it goes
- Explains snapshot memory costs, eviction and read-only controls, with cautions about collections and transaction semantics.
- Examples
- Shows an Item made read-only through Hibernate’s Session and flushed without a database update.
- What you can do after
- Operate/configure persistence-context caching by limiting query results, detaching entities, clearing the context, and setting read-only modes.
Read how Java Persistence with Spring Data and Hibernate epub explains caching →
Acing the Frontend Interview
Database caching uses database-internal cached data or query results to reduce repeated storage access and accelerate reads.
- How it is used
- Frames caching as a system-design technique for reducing latency, backend load, and repeated storage access across multiple layers.
- How deep it goes
- Explains cache layers and trade-offs, then points toward TTLs, eviction, invalidation, and distributed-cache concerns without deeply implementing database internals.
- Examples
- Uses browser and CDN assets, API responses, sessions, product listings, computed thumbnails, Redis, Memcached, TTLs, and LRU eviction.
- What you can do after
- Evaluate caching designs by selecting a layer and articulating performance, consistency, expiration, invalidation, and scalability trade-offs.
Generative AI in Action
Caching is the practice of storing reusable data or results in faster-access storage so they can be returned without repeating the original computation or request.
- How it is used
- Reduces repeated LLM API calls, latency, token use, and computational load for frequently requested information.
- How deep it goes
- Explains cache-aside flow, freshness, monitoring, and fallback, then illustrates a RedisVL semantic cache configuration.
- Examples
- Illustrates cached and uncached capital-city questions with RedisVL, including sample hit, miss, and timing output.
- What you can do after
- Apply it via RedisVL and Redis to reuse LLM responses while considering freshness and operational reliability.
Build AI-Enhanced Web Apps
The technique of storing computed or retrieved data temporarily so that later requests can return it without repeating the original work.
- How it is used
- Uses caching to reduce latency and repeated computation in AI web applications.
- How deep it goes
- Explains Next.js route and fetch configuration and revalidation behavior, but does not implement a cache from scratch.
- Examples
- Uses session-specific interview feedback cached in Redis and a dynamic AI chat route.
- What you can do after
- Apply it via Next.js fetch options and route-handler configuration to select caching or server reevaluation.
React Hooks in Action: With Suspense and Concurrent Mode
A cache is temporary storage of previously retrieved data kept for faster access and synchronization with its source.
- How it is used
- Improves responsive React data-fetching experiences by reusing server data and synchronizing newer results in the background.
- How deep it goes
- Explains key-based retrieval, background refresh, and direct access, while deferring detailed stale-data timing.
- Examples
- Demonstrates shared bookables data, parameterized bookings queries, and an edit form initialized from cached bookables.
- What you can do after
- Apply it via React Query to share fetched data, reduce loading states, define query keys, and initialize components from cached results.
Read how React Hooks in Action: With Suspense and Concurrent Mode explains caching →
The Art of Code
A cache is a storage layer that retains recently or frequently used results for faster retrieval and reduced dependency load.
- How it is used
- Uses caching to reduce remote calls and dependency load while supporting performance during partial outages.
- How deep it goes
- Explains cache tiers, expiration, and memory tradeoffs, and shows a lookup implementation without detailing eviction mechanisms.
- Examples
- Shows a Java profile cache that loads missing entries from a database and describes HTTP response caching.
- What you can do after
- Evaluate cache placement and expiration against data-change frequency, remote-call reduction, and memory use.
ASP.NET Core in Action, 3e
A cache is local storage of previously retrieved data that allows subsequent requests to reuse the stored response without contacting the origin server.
- How it is used
- Uses caching to reduce browser requests, server work, or repeated calls to third-party services.
- How deep it goes
- Explains response-versus-output behavior and implements scheduled background updates, while deferring detailed invalidation configuration.
- Examples
- Uses HTTP caching headers, server-stored generated responses, and a scheduled exchange-rate cache as concrete examples.
- What you can do after
- Operate/configure ASP.NET Core caching and build a BackgroundService that refreshes shared data for fast API responses.
Fast by Default
A cache is a storage layer containing reusable copies of data or computation results to avoid repeated retrieval or calculation.
- How it is used
- Reduces repeated data retrieval and computation within dashboards, applications, and performance-critical rendering paths.
- How deep it goes
- Explains bounded storage, expiration, invalidation, distribution, consistency trade-offs, and graceful failure, but not cache implementation code.
- Examples
- Uses dashboard filter computations, Redis Cluster and Memcached, and pre-resized mobile thumbnails to illustrate caching choices.
- What you can do after
- Operate/configure caches with size limits, expiration, invalidation, consistency boundaries, and failure-degradation checks.
Clojure: The Essential Reference
A cache is a storage structure that retains previously computed results for retrieval by associated keys or repeated access.
- How it is used
- Uses caching to avoid repeated function computations and to retain realized sequence results.
- How deep it goes
- Explains argument-keyed memoization through its contract and examples, then details how lazy realization affects sequence memory.
- Examples
- Shows repeated addition calls, cached spelling-distance rankings, and a lazy sequence viewing an updated Java list.
- What you can do after
- Apply it via `memoize` to reuse results, and evaluate when retained lazy-sequence elements increase memory use.
Read how Clojure: The Essential Reference explains caching →
The Design of Web APIs, Second Edition
Caching is the temporary storage of data or responses so that subsequent requests can be served without repeating the original retrieval or computation.
- How it is used
- Reduces repeat API calls and downloads when consumers revisit data, while requiring attention to outdated responses.
- How deep it goes
- Works through cache directives, representation keys, and conditional validation, but leaves entity-tag generation to implementation choice.
- Examples
- Uses a repeatedly opened bank account screen, a ten-minute cache, HTTP headers, and an unchanged PDF statement.
- What you can do after
- Operate/configure it with Cache-Control and Vary, then validate stored responses using Etag, If-None-Match, and 304 Not Modified.
Read how The Design of Web APIs, Second Edition explains caching →
Software Engineering for Data Scientists
A cache is a storage mechanism that temporarily retains computed or retrieved values for faster reuse.
- How it is used
- Speeds repeated machine-learning prediction requests by reusing prior function-call results.
- How deep it goes
- Defines memoization, demonstrates functools.lru_cache, and explains LRU eviction and cache-size limits.
- Examples
- Uses repeated random-forest artist predictions and a timed Python lru_cache implementation.
- What you can do after
- Apply it via Python's functools.lru_cache to cache repeated model-prediction calls.
Read how Software Engineering for Data Scientists explains caching →
Learn Docker in a Month of Lunches, Second Edition
A cache is a temporary storage area containing previously retrieved data so that repeated requests can be served more quickly.
- How it is used
- Improves reverse-proxy performance and reliability by serving repeated responses without contacting application containers.
- How deep it goes
- Explains cache hits, freshness periods, stale fallbacks, and nginx directives, but does not detail cache storage internals.
- Examples
- Uses an image gallery, API responses, curl headers, nginx configuration, and a pi-calculation application.
- What you can do after
- Operate/configure nginx as a caching reverse proxy and verify misses, hits, freshness, and resilience with curl.
Read how Learn Docker in a Month of Lunches, Second Edition explains caching →
React in Depth
A cache is a stored copy of retrieved or computed data that can be returned without obtaining or calculating it again.
- How it is used
- Uses cached server results to reduce request delays and keep remote-dependent React views responsive.
- How deep it goes
- Works through query invalidation, mutation-driven updates, and partial initial data in hooks; leaves freshness policy details unspecified.
- Examples
- Uses customer deletion, an updated detail record, and cached list fields that seed a detail view.
- What you can do after
- Apply it via TanStack Query to invalidate affected results, update cached queries after mutations, and prefill detail views.
Designing AI Systems
Response caching is the storage and reuse of complete model responses for requests that match according to a defined request-matching rule.
- How it is used
- Reduces model-service cost and latency by reusing complete responses for matching requests.
- How deep it goes
- Defines cache keys, configuration, eviction, and monitoring; describes semantic matching without implementing it.
- Examples
- Contrasts repeated documentation and password-reset questions with a creative-writing assistant that needs varied responses.
- What you can do after
- Operate and configure response caching using per-request settings, TTL, capacity, and hit-rate monitoring.
Machine Learning System Design
A storage mechanism that keeps copies of data or results so repeated requests can be served without recomputation or retrieval from the original source.
- How it is used
- Reduces inference latency, server load, and compute cost by reusing results for repeated or similar requests.
- How deep it goes
- Explains key-value, TTL, runtime updates, and fuzzy matching, but does not develop cache invalidation or consistency mechanics.
- Examples
- Uses an in-memory get-or-compute cache, semantic or regex-based fuzzy keys, and a PhotoStock ElastiCache design.
- What you can do after
- Apply it via AWS ElastiCache, Redis, or Memcached to reduce repeated runtime inference work.
Functional Programming in JavaScript
An intermediary repository that stores previously computed or retrieved values for later reuse.
- How it is used
- Speeds applications by reusing results of expensive or repetitive operations.
- How deep it goes
- Explains cache keys, hits, misses, wrappers, and performance effects, but does not detail cache eviction or invalidation.
- Examples
- Uses cached student lookup, memoized ROT13, composed student display, and recursive factorial examples.
- What you can do after
- Apply memoization to reduce repeated computation in JavaScript functions.
Read how Functional Programming in JavaScript explains caching →
C# Concurrency: Asynchronous and multithreaded programming
A cache is a storage layer that retains previously obtained values so they can be returned more quickly on later requests.
- How it is used
- Speeds repeated long-running operations by returning stored results and coordinating access to shared cached state.
- How deep it goes
- Explains lock placement, asynchronous races, compiler reordering, and CPU cache memory, but does not develop eviction or expiration policies.
- Examples
- Uses a C# Dictionary cache around an asynchronous HttpClient request and contrasts whole-method locking with split lock boundaries.
- What you can do after
- Apply it via C# Dictionary, lock, and async/await while recognizing duplicate misses and unsafe unsynchronized memory access.
Read how C# Concurrency: Asynchronous and multithreaded programming explains caching →
Building Reliable AI Systems
Storing results so they can be reused instead of recomputed or retrieved again.
- How it is used
- Reduces repeated LLM calls, response time, and cost in query-response systems.
- How deep it goes
- Contrasts exact and semantic matching and sketches lookup, thresholding, fallback, and storage.
- Examples
- Uses store-hours and refund-policy questions to illustrate repeated and similarly phrased queries.
- What you can do after
- Apply caching via FAISS or GPTCache to reuse responses for matching queries.
Architecting an Apache Iceberg Lakehouse
Caching is the technique of storing reusable data or computation results in a faster-access location to reduce subsequent processing or retrieval cost.
- How it is used
- Reduces repeated remote retrieval and processing across lakehouse queries, dashboards, machine-learning pipelines, and AI workloads.
- How deep it goes
- Explains caching through engine behavior, workload patterns, and tradeoffs but stops short of cache implementation details or invalidation policies.
- Examples
- Uses Dremio reflections, interactive dashboards, GPU training pipelines, regulatory reports, and AI-agent queries as workload examples.
- What you can do after
- Apply it via Dremio and lakehouse query engines to reduce redundant scans and improve latency for governed consumption workloads.
Read how Architecting an Apache Iceberg Lakehouse explains caching →
API Design Patterns
A cache is a stored copy of data retained for faster or more convenient access than retrieving the original data again.
- How it is used
- Uses cached copies to reduce extra API calls and deduplicate repeated requests.
- How deep it goes
- Explains references, embedded copies, response replay, storage costs, expiration, invalidation, and collision safeguards.
- Examples
- Demonstrates book-author resources, repeated non-idempotent requests, large batch responses, and pagination result caches.
- What you can do after
- Apply it via request identifiers, response caches, embedded resources, and explicit freshness policies.
Grokking Software Architecture
Caching is the technique of temporarily storing reusable data or results to reduce latency or repeated computation.
- How it is used
- Reduces response time and repeated database work before adding capacity through more servers.
- How deep it goes
- Defines cache hits and misses and walks through a database-backed example, while noting complexity and consistency costs.
- Examples
- Uses a dashboard summary with three database queries and contrasts local with shared caching.
- What you can do after
- Apply a cache-first pattern to reuse results and reduce repeated computation or database queries.
AI Agents in Action, Second Edition
- How it is used
- Reduces repeated tool work and research-query retrieval to improve agent latency, performance, and cost.
- How deep it goes
- Defines idempotency, demonstrates input-key caching, and flags stale-cache invalidation without detailing production storage.
- Examples
- Contrasts weather lookup with email sending, demonstrates input-derived keys, and identifies prompt, response, and embedding caches.
- What you can do after
- Apply it by making tool calls idempotent and caching results under a key for faster reuse and safer replay.
Read how AI Agents in Action, Second Edition explains caching →
Modern C, Third Edition
A cache is storage that retains previously computed values so they can be reused instead of recomputed.
- How it is used
- Uses retained results to eliminate repeated recursive computation and improve algorithmic performance.
- How deep it goes
- Explains cache lookup, storage, initialization, and linear-time behavior, but stops short of concrete performance measurements.
- Examples
- Demonstrates caching through recursive Fibonacci computation using an array of previously computed values.
- What you can do after
- Build it from scratch by adding an array cache to recursive Fibonacci and reasoning about its linear execution time.
Data Pipelines with Apache Airflow, Second Edition
Caching is the technique of storing computed or retrieved objects so later requests can reuse them without repeating the original work.
- How it is used
- Supports efficient Airflow reruns and avoids repeated metastore lookups when hooks reuse sessions or workflows reuse intermediate datasets.
- How deep it goes
- Shows instance-level implementation and pipeline retention trade-offs, but does not specify a general cache invalidation strategy.
- Examples
- Uses a cached API session and base URL, plus cloud-stored fetch and preprocess outputs.
- What you can do after
- Apply it via Airflow hooks and shared filesystems to reduce repeated work and rerun selected tasks after failures.
Read how Data Pipelines with Apache Airflow, Second Edition explains caching →
Modern C
A cache is a storage structure containing previously computed values for reuse.
- How it is used
- Uses cached values to eliminate repeated work in recursive Fibonacci computation.
- How deep it goes
- Explains array initialization, lookup, assignment, storage tradeoffs, and complexity improvement, while deferring implementation details and measurements.
- Examples
- Demonstrates caching through fibCacheRec, fibCache, repeated fib(2) calls, and a variable-length array.
- What you can do after
- Build it from scratch by adding an initialized array cache to recursive Fibonacci computation.
Build Python Web Apps with Streamlit
Caching is the storage of computed results or loaded data so they can be reused without repeating the underlying work.
- How it is used
- Improves Streamlit dashboard performance by reusing loaded data during interactions, reruns, and page refreshes.
- How deep it goes
- Explains cached function results, decorators, refresh behavior, TTL, and loading feedback, but does not detail cache internals.
- Examples
- Uses a Streamlit load_data function that reads a sales CSV into a pandas DataFrame and displays spinner feedback.
- What you can do after
- Apply caching via Streamlit's st.cache_data decorator to reduce repeated data-loading work.
Read how Build Python Web Apps with Streamlit explains caching →
Authorization in Action
A cache is a temporary data store that retains copies of information for faster retrieval or reduced dependence on an origin system.
- How it is used
- Balances authorization latency and upstream dependence against the assurance requirements of signal freshness.
- How deep it goes
- Explains cache placement, invalidation, blast radius, TTL trade-offs, and signal-specific durations without detailing cache algorithms.
- Examples
- Uses on-call status with a five-minute TTL and employment status with a 24-hour TTL.
- What you can do after
- Operate/configure authorization caches by selecting placement, invalidation behavior, and TTLs for different signal risks.
GitHub Actions in Action: Continuous integration and delivery for DevOps
Caching is the practice of storing reusable data or outputs so they can be retrieved without repeating their computation or acquisition.
- How it is used
- Speeds GitHub Actions workflows by reusing dependencies and other frequently used files on fresh job machines.
- How deep it goes
- Explains cache actions, paths, keys, lockfile hashing, setup actions, and Docker cache configuration.
- Examples
- Uses npm setup-node caching, a NuGet package-path key, Docker cache settings, and shared artifact cache keys.
- What you can do after
- Apply it via GitHub Actions setup actions or actions/cache with paths and dependency-sensitive keys.
Read how GitHub Actions in Action: Continuous integration and delivery for DevOps explains caching →
Spring Microservices in Action
A cache is a store of previously retrieved data that can be reused without querying its source again.
- How it is used
- Avoids repeated organization-service calls during licensing lookups and reduces Eureka registry lookups during service discovery.
- How deep it goes
- Implements Redis lookup, miss-triggered REST retrieval, cache population, and exception fallback without specifying broader cache policies.
- Examples
- Demonstrates caching organization records in Redis and caching service locations locally with Ribbon.
- What you can do after
- Apply it via Spring Redis repositories to reuse organization records and fall back to REST when Redis is unavailable.
Machine Learning Platform Engineering
A caching strategy is a method for storing and reusing computational results to avoid repeating expensive processing.
- How it is used
- Reduces repeated LLM processing, token consumption, latency, and operational cost in RAG and prompt-management systems.
- How deep it goes
- Distinguishes context, semantic, and client-side caching, explains trade-offs, and demonstrates vector-similarity implementation.
- Examples
- Uses documentation sections, FAQ systems, rephrased API-key questions, RedisVL vector queries, and Langfuse-managed prompts.
- What you can do after
- Apply caching with RedisVL vector similarity or Langfuse client-side prompt caching while accounting for reuse, freshness, and storage costs.
Read how Machine Learning Platform Engineering explains caching →
Snowflake Data Engineering
Data caching is the retention of previously accessed data in faster storage or memory so subsequent operations can retrieve it more efficiently.
- How it is used
- Improves Snowflake query performance by reusing query results, metadata, or warehouse-resident table data.
- How deep it goes
- Defines caching and maps three cache types to Snowflake architecture layers, focusing on behavior rather than cache-management internals.
- Examples
- Uses repeated queries, metadata-only queries, and subsequent queries against the same virtual warehouse as caching examples.
- What you can do after
- Evaluate/understand Snowflake query-performance behavior by distinguishing cache types and locating caching in the query profile.
The Quick Python Book, Fourth Edition
A cache is stored data containing previously computed results so that later requests can reuse them without repeating the computation.
- How it is used
- Speeds repeated calls to expensive functions by reusing results for previously seen argument combinations.
- How deep it goes
- Demonstrates in-memory lookup and disk persistence, but leaves concurrency handling unresolved.
- Examples
- Uses the sole calculation, repeated argument triples, a spreadsheet-cell exercise, and a pickled cache file.
- What you can do after
- Apply it via Python dictionaries and pickle to avoid repeated calculations within and across program sessions.
Read how The Quick Python Book, Fourth Edition explains caching →
Deep Learning with Python, Third Edition
- How it is used
- Uses caching to reduce repeated model work during autoregressive generation and repeated reads of training data from Google Cloud Storage.
- How deep it goes
- Explains why past per-layer key and value vectors can be reused and gives a single dataset.cache() instruction.
- Examples
- Uses a mini-GPT generation loop and caching a dataset to avoid repeated reads from Google Cloud Storage.
- What you can do after
- Apply it via key/value reuse during Transformer generation or tf.data.Dataset.cache() to reduce repeated work.
Read how Deep Learning with Python, Third Edition explains caching →
Microservices Patterns, Second Edition
A caching pattern stores previously obtained data so that subsequent requests can be served without accessing the original provider.
- How it is used
- Reduce load and latency in synchronous service calls, and preserve useful API-composition responses when providers fail.
- How deep it goes
- Defines cached-response use and provider-proxy fallback, noting stale data and meaningful cache keys without explaining cache construction.
- Examples
- Illustrates cached delivery information in an order response when the Delivery Service is unavailable.
- What you can do after
- Apply caching by storing provider responses in a proxy and querying them when the provider is unavailable.
Read how Microservices Patterns, Second Edition explains caching →
Deep Learning with PyTorch, Second Edition
The practice of storing preprocessed data temporarily so it can be reused efficiently later instead of recomputing it.
- How it is used
- Speeds data loading when parsing, disk access, conversion, or cropping would otherwise repeat for every sample.
- How deep it goes
- Explains memory and disk layers, memoization, invalidation, and augmentation ordering, but defers the full rationale for the setup.
- Examples
- Uses cached CT objects and extracted candidate arrays, then places caching before affine image augmentation.
- What you can do after
- Operate/configure memory and disk caches around CT-loading functions while preserving correct augmentation and invalidating stale results.
Read how Deep Learning with PyTorch, Second Edition explains caching →
LLMs in Production
Caching is the technique of storing reusable computation results or data so later requests can be served without repeating the original work.
- How it is used
- Improves LLM-service latency by avoiding inference for repeated, normalized requests.
- How deep it goes
- Recommends caching conditionally but stops short of specifying cache keys, invalidation, storage, or implementation.
- Examples
- Uses repeated user requests with easily normalized inputs as the motivating example.
- What you can do after
- Apply caching when request repetition and normalization make avoided inference likely to improve service speed.
Retrieval Augmented Generation, The Foundational Ideas
Caching stores reusable computation results or retrieved information to avoid repeating expensive operations.
- How it is used
- Uses caching to avoid repeated claim extraction and verification during RAG evaluation.
- How deep it goes
- Explains result reuse and structured persistence, but does not detail eviction, invalidation, or cache implementation.
- Examples
- Uses repeated context–answer pairs and cached evaluation results stored in databases or versioned JSON.
- What you can do after
- Operate/configure caching to skip redundant extraction steps and retain evaluation results for trend tracking.
Read how Retrieval Augmented Generation, The Foundational Ideas explains caching →
AI Model Evaluation
The storage of previously computed or retrieved data so subsequent requests can be served more quickly.
- How it is used
- Improves user-facing AI performance by reducing visible latency in recommendations, search, chat, and other interactive features.
- How deep it goes
- Connects caching to real and perceived latency but stops short of explaining cache invalidation or storage internals.
- Examples
- Uses cached recommendation rows, hidden backend work, TTFB optimization, and cached asynchronous AI content as examples.
- What you can do after
- Apply caching to reduce visible response time and diagnose whether latency originates in inference or the surrounding system.
Natural Language Processing in Action, Second Edition
Caching is the practice of storing computed results so repeated requests can be served without recomputing them.
- How it is used
- Speeds Streamlit question-answering applications by reusing loaded models and avoiding repeated expensive initialization.
- How deep it goes
- Explains input hashing, pickle-backed outputs, and compatibility limits, then demonstrates resource decorators without covering cache invalidation.
- Examples
- Uses cached FAISS document-store loading and cached embedding-retriever and reader construction in a Streamlit QA app.
- What you can do after
- Operate/configure Streamlit applications with st.cache_resource around expensive model-loading functions.
Read how Natural Language Processing in Action, Second Edition explains caching →
Financial AI in Practice
A cache is a storage layer that retains previously computed or retrieved results for faster reuse.
- How it is used
- Serves stored answers to repeated financial AI queries before slower, more expensive processing.
- How deep it goes
- Defines cached-response serving and its latency and cost benefits, but gives no cache implementation mechanics.
- Examples
- Uses mortgage-rate FAQs and popular stock summaries as examples of repeat queries that can receive cached responses.
- What you can do after
- Apply caching to repeated financial queries by serving stored responses instead of rerunning expensive LLM pipelines.
Building LLM Applications with DSPy
Caching is the practice of storing a language model provider's response so repeated identical requests can be served without querying the provider.
- How it is used
- Uses cached language-model responses to avoid repeated provider queries for identical inputs.
- How deep it goes
- Describes response reuse and LM-level and global controls without detailing cache internals.
- Examples
- Shows DSPy generating a poem twice from the same question after global caching is disabled.
- What you can do after
- Apply caching controls in DSPy by setting cache options on an LM and globally.
Read how Building LLM Applications with DSPy explains caching →
Context Engineering
The technique of storing information or computation results for reuse to reduce repeated processing or retrieval.
- How it is used
- Reduces redundant retrieval and latency by preloading reusable external context.
- How deep it goes
- Defines caching through cache-augmented generation and defers detailed trade-offs and selection criteria.
- What you can do after
- Understand how preloading reusable context differs from query-time retrieval and direct injection.
Also covered in
- Crack Any Codebase with AIIn early access (MEAP)
- Distributed Machine Learning Patterns
- Designing Deep Learning Systems: A guide for software engineers
- ScyllaDB in Action
- Logs and Telemetry
- Modern Angular
- Learn Concurrent Programming with Go
- Vanilla WebIn early access (MEAP)
- Fabulous Adventures in Data Structures and Algorithms
- Bootstrapping Microservices, Second Edition: With Docker, Kubernetes, GitHub Actions, and Terraform
- Road to Kubernetes
- Functional Programming in C#, Second Edition
What to know first
- meanThe mean is a measure of central location equal to the arithmetic average of numerical observations, or the probability-weighted average of a random variable’s possible values.
- language modelA language model is a computational model that represents patterns in linguistic sequences to assign probabilities, predict, generate, or evaluate language.
- embeddingAn embedding is a numerical vector representation of an item in a feature space, encoding selected properties or relationships for computational tasks such as comparison or retrieval.
- contractA contract is an agreed specification of the conditions, expectations, rights, or obligations that govern behavior or interactions between parties or software components.
Related concepts
- DSPyused by · DSPy is a Python framework for declaratively programming, composing, evaluating, and optimizing language-model applications and pipelines.
- eager cachingincludes · Eager caching populates a cache before its values are requested, typically during application initialization.
- intermediate cachingincludes · Intermediate caching is a caching technique in which a proxy or content-distribution service between a client and origin server stores and serves reusable responses.
Questions about caching
What is caching?
Caching is the temporary storage of data or computed results for reuse, reducing the work or delay of retrieving or computing them again.
Which Manning books explain caching?
Building Web APIs with ASP.NET Core, Learn Go with Pocket-Sized Projects, Advanced Algorithms and Data Structures, Pro ASP.NET Core 7, Tenth Edition, Amazon Web Services in Action, Third Edition: An in-depth guide to AWS, Latency, Vibe Engineering, Java Persistence with Spring Data and Hibernate epub and 41 more.
What should I know before learning caching?
Manning books that teach caching usually introduce mean, language model, embedding and contract first.
What is the difference between caching and sync.Pool?
Caching is the temporary storage of data or computed results for reuse, reducing the work or delay of retrieving or computing them again. sync.Pool is a Go standard-library type that temporarily retains reusable objects and permits concurrent retrieval and return of those objects.
What kinds of caching are there?
Kinds of caching include eager caching and intermediate caching.
What is caching also called?
Caching is also called cache.
These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.
















































