Suppose a daily recipe pipeline must decide whether an uploaded recipe is new, changed, or already stored. Comparing every recipe word by word would be inefficient. A UUID gives the pipeline a fixed-length key to check instead. Here, Weaviate’s `generate_uuid5` creates a 32-character key from a string; the same input produces the same key. section 14.5 section 14.7
Preprocessing puts each recipe’s ingredients and instructions into separate `chunk` rows alongside `recipe_name`. It strips surrounding whitespace, then generates `recipe_uuid` from the name and `chunk_uuid` from each chunk’s text. This gives the pipeline one key for recipe identity and separate keys for its content. Text details matter: changing an ingredient can change a chunk UUID, while unchanged input retains its UUID. section 14.5
Listing 14.4 Creating UUIDs for the recipe name and chunks
import pandas as pd
from weaviate.util import generate_uuid5
def assign_uuids(df: pd.DataFrame) -> pd.DataFrame:
return (
df
.assign(
chunk = lambda df: df.chunk.astype(str).str.strip(), #1
recipe_name=lambda df: df.recipe_name.astype(str).str.strip(), #1
recipe_uuid=lambda df:[generate_uuid5(n) for n in df.recipe_name], #2
chunk_uuid=lambda df:[generate_uuid5(c) for c in df.chunk], #2
)
.reset_index(drop=True)
)
The `assign_uuids` function shows the two `generate_uuid5` calls and the whitespace stripping that precedes them. section 14.5
In the Weaviate collection, `recipe_uuid` and `chunk_uuid` are UUID-typed properties; `recipe_name` and `chunk` are text properties. The UUID fields are excluded from vectorization. The comparison task loads preprocessed rows from MinIO and checks them against existing database records. section 14.6.2 section 14.7
The comparison produces three operational outcomes:
- No existing recipe: mark its records `create`. section 14.7
- Existing recipe with different chunk UUIDs: mark it `update`. section 14.7
- Matching recipe name and chunk UUIDs: remove its records from the tracking DataFrame to avoid reprocessing. section 14.7
Content-derived keys make repeat uploads and changes practical to detect, but the comparison policy remains application-specific; adding tenancy, for example, would require reconsidering the update strategy. section 14.5 section 14.7
