Caching stores a function’s results so Streamlit can reuse them instead of rerunning expensive work whenever the app loads or its input changes. It is especially useful for heavy datasets and slow-to-load models. section 10.3.12
A Streamlit cache works by:
- Hashing the function input so Streamlit can compare it with previous inputs. section 10.3.12
- Saving the function output in a pickle file. section 10.3.12
- Caching compatible models while recognizing that the FAISS document store cannot be cached or hashed in this setup. section 10.3.12
Listing 10.25 Loading the Reader and Retriever
>>> @st.cache_resource >>> def load_retriever(document_store): #1 >>> embedder = "sentence-transformers/multi-qa-mpnet-base-dot-v1" >>> return EmbeddingRetriever( >>> document_store=document_store, >>> embedding_model=embedder) >>> >>> @st.cache_resource >>> def load_reader(): ... return TransformersReader( ... model_name_or_path="deepset/roberta-base-squad2")
Decorate the retriever and reader loaders with st.cache_resource so model-loading work can be reused by the QA application. section 10.3.12
