A Zarr array is a persistent multidimensional array whose data and metadata are stored in independently addressable chunks. Zarr repositories organize arrays in a tree of groups, while array access follows a NumPy-like interface. section 8.4.1 section 8.4.4
A Zarr array is useful when data must remain structured, scalable, and processable without loading everything into memory:
- Stores multidimensional datasets such as a calls matrix alongside related one-dimensional position and allele arrays. section 8.4.1
- Exposes groups, arrays, shapes, dtypes, and storage information for traversing and inspecting a repository. section 8.4.1
- Reads and writes independently addressable chunks, enabling chunk-aligned processing and parallel writes. section 8.4.4
- Requires chunk sizes calibrated to algorithms, memory capacity, storage type, and filesystem behavior. section 8.4.3
Concatenating chromosome arrays
A concatenated array can combine chromosome-level calls into shape (3,976,554, 210), using uint8 values and chunks of (20,000, 210). Sequential copying initializes 199 chunks; parallel workers can process those chunks independently, avoiding concurrent writes to the same file. section 8.4.3 section 8.4.4
import zarr
genomes = zarr.open("db.zarr")
genomes.tree() #1
Inspect the repository tree to see how chromosome groups contain arrays and how array shape and dtype describe the stored datasets. section 8.4.1
