LRU
LRU is a cache replacement policy that evicts the item whose most recent access occurred furthest in the past.
Advanced Algorithms and Data Structures
An LRU cache is a cache that evicts the entry that has been unused for the longest time when it reaches capacity.
- How it is used
- Uses caching to avoid repeated computation while retaining fast lookup, insertion, retrieval, sizing, and eviction.
- How deep it goes
- Develops the hash-table and doubly-linked-list design through pseudocode, synchronized updates, and amortized constant-time analysis.
- Examples
- Uses company-name entries, an Acme cache miss, an EvilCorp update, and diagrams or pseudocode for insertion and eviction.
- What you can do after
- Build it from scratch with a capacity, hash table, doubly linked list, head, tail, insertion, lookup, update, and eviction operations.
Read how Advanced Algorithms and Data Structures explains LRU →
Designing Deep Learning Systems: A guide for software engineers
The least recently used algorithm evicts the item that has gone unused for the longest time when a cache reaches its capacity.
- How it is used
- Manages model swapping in multitenant prediction services when memory or GPU capacity cannot hold every required model.
- How deep it goes
- Defines the eviction role and pairs it with model partitioning, but defers detailed cache implementation to a prediction-service example.
- Examples
- Uses a multitenant prediction service and an intent-classification service to illustrate model caching and swapping.
- What you can do after
- Operate model caches that evict inactive models and free capacity for incoming prediction requests.
Read how Designing Deep Learning Systems: A guide for software engineers explains LRU →
Related concepts
- cachingconfigures · Caching is the practice of storing reusable data or computation results so later access can avoid repeating retrieval or computation.
- least frequently usedcontrasts with · Least frequently used is a cache-eviction policy that removes the entry with the lowest access frequency when capacity is exhausted.
- hash mapuses · A hash map is a key-value data structure that uses a hash function to locate entries and support efficient lookup, insertion, and removal by key.
- Redisuses · Redis is an open-source, in-memory data-structure store that provides network-accessible data operations and supports persistence.
- cache replacement policykind of · A cache replacement policy is a rule or algorithm for selecting which cached item to remove when capacity is insufficient or cached data must be discarded.
These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.