Caching temporarily stores a computed or retrieved result so it can be reused instead of computed again. It trades memory for computational efficiency: repeated work becomes faster, but stored results consume space. This is useful when the same function inputs occur repeatedly, such as users requesting predictions from a deployed machine-learning model. section 6.0 section 6.3 section 6.7
Memoization is caching function-call results. On a first call, the function computes a result and stores it; on a later call with the same inputs, it returns the stored result. This can reduce latency for repeated prediction lookups. section 6.3
Cache repeated model predictions
Python's functools.lru_cache can cache a prediction function by adding the @lru_cache decorator. A function can accept a tuple of artist features, format those features for a random-forest model, and return its prediction. Calling it once performs the prediction; calling it again with the same tuple retrieves the cached result. The illustrated timing falls from about 0.075 seconds to about 0.00012 seconds after the result is available. section 6.3 section 6.3
LRU means Least Recently Used. An LRU cache retains up to a chosen number of items and removes the least recently used result when a new result needs room. functools.lru_cache has no maximum size by default, so setting maxsize can limit how many results remain stored. section 6.3
Use caching when repeated computation is a recognizable bottleneck:
- Reuse model predictions when an application receives the same input values more than once. section 6.3
- Avoid repeating identical computations, alongside profiling, vectorization, multiprocessing, and suitable data structures for efficiency work. section 6.7
- Practice by implementing an LRU or another cache from scratch and testing it with model predictions. section 6.6
Listing 6.20 This code is available as part of cache_ml_model_call.py in the ch6 directory. To create the cache, we just need to add the lru_cache method as a decorator to the function that we want to cache. Next, we write the function we’re caching (in this case, a function that fetches model predictions). Lastly, we time the difference between running the model to get the prediction vs. fetching it from the cache (after it’s available).
from functools import lru_cache #1
@lru_cache #2
def get_model_predictions(model, inputs): #3
formatted_inputs = pd.DataFrame.\
from_dict({"followers": inputs[0],
"num_genres": inputs[1]},
orient = "index").\
transpose()
return model.predict(formatted_inputs)
inputs = (2, 4) #4
start = time.time()
get_model_predictions(forest_model, inputs) #5
end = time.time()
print("Model predictions runtime (w/o) cache: ", end - start)
start = time.time()
get_model_predictions(forest_model, inputs) #6
end = time.time()
print("Model predictions runtime (w/) cache: ", end - start)
Inspect a focused Python example that decorates a model-prediction function with lru_cache and times an uncached call against a cached lookup. section 6.3
















































