Data lake
A data lake is a repository that stores large volumes of data in raw or lightly processed form and varied formats for later processing and analysis.
Think Like a CTO
A data lake is a large-scale storage repository that retains data in its original or varied formats for later processing and analysis.
- How it is used
- Positions a data lake as a staging and storage option when varied incoming data lacks a permanent home or awaits transformation.
- How deep it goes
- Defines the storage role and compares it with other stores, but does not detail lake implementation.
- Examples
- Uses AWS S3 as a possible starting point for parking data before deciding how and where to transform it.
- What you can do after
- Evaluate/understand when varied files need staging rather than a structured database or an analytics-focused warehouse.
Architecting an Apache Iceberg Lakehouse
A data lake is a storage-centered data platform that retains data in flexible formats on relatively inexpensive storage for varied downstream processing.
- How it is used
- Stores diverse, fast-growing data for downstream analytics when warehouse storage is too rigid or expensive.
- How deep it goes
- Explains storage architecture, file formats, historical engines, and governance failures, but defers detailed table-format implementation.
- Examples
- Uses clickstream events, application logs, IoT readings, social media data, HDFS, Amazon S3, Parquet, ORC, and CSV.
- What you can do after
- Evaluate data-lake tradeoffs and understand why table formats support more reliable lakehouse analytics.
Read how Architecting an Apache Iceberg Lakehouse explains data lake →
Effective Data Analysis
A centralized repository that stores large volumes of raw data in structured, semi-structured, or unstructured form for later processing and analysis.
- How it is used
- Positions raw data before warehouse curation, supporting varied analytical needs across wide and high-volume sources.
- How deep it goes
- Explains storage format, schema flexibility, suitable workloads, and cleanup requirements, but stops short of implementation details.
- Examples
- Uses mixed source formats, wide and big data, and company records covering clicks, sign-ups, purchases, and installations.
- What you can do after
- Evaluate storage options and justify whether a data lake suits anticipated analytical needs.
Related concepts
- object storeuses · An object store is a storage system that stores and retrieves data as independently addressable objects containing content and associated metadata.
- columnar datauses · Columnar data is tabular data organized and stored by column rather than by complete row, enabling efficient access to selected fields and data compression.
- data lakehouseused by · A data lakehouse is a data architecture pattern combining the flexible storage of data lakes with the governance, reliability, and analytical capabilities of data warehouses.
These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.