Data engineer
A data engineer is a professional who designs, builds, operates, and maintains systems and pipelines for collecting, transforming, storing, governing, and delivering data.
Delivering Machine Learning Projects: From design to deployment
A data engineer is a specialist who acquires, moves, transforms, manages, and operationalizes data resources and their supporting infrastructure.
- How it is used
- Positions the data engineer as the specialist who makes difficult data resources usable during ML project delivery.
- How deep it goes
- Defines the role and illustrates setup assignments, but does not specify data-engineering procedures or implementations.
- Examples
- Uses a retail revenue-and-churn project in which an assigned data engineer begins infrastructure setup and data investigations.
- What you can do after
- Practice/perform it by assigning data-engineering work during setup and initiating access requests before data investigations.
Read how Delivering Machine Learning Projects: From design to deployment explains data engineer →
Designing Deep Learning Systems: A guide for software engineers
A specialist who builds and operates systems for collecting, transforming, enriching, labeling, and continuously ingesting data.
- How it is used
- Supports reliable collection, preparation, and delivery of training data within the machine-learning development cycle.
- How deep it goes
- Maps role responsibilities to a three-layer dataset service and a concrete ingestion-to-training workflow.
- Examples
- Uses activity logs, customer surveys, labeled datasets, ingestion APIs, and intent-classification training data.
- What you can do after
- Operate data ingestion and dataset-fetching workflows that supply prepared training data to model development.
Read how Designing Deep Learning Systems: A guide for software engineers explains data engineer →
The Art of AI Product Development
A data engineer is a practitioner who builds and operates systems and processes for collecting, transforming, governing, and providing access to data.
- How it is used
- Supports AI development by making usable, reliable, and appropriately accessible data available to stakeholders and downstream systems.
- How deep it goes
- Explains core responsibilities and rising importance for unstructured generative AI data, but does not detail implementation methods.
- What you can do after
- Operate data-processing systems, monitor data quality, and manage data access for key stakeholders.
Read how The Art of AI Product Development explains data engineer →
Related concepts
- pipelineuses · A pipeline is an ordered or coordinated sequence of connected processing stages or tasks that passes data, work, or results toward an outcome.
- data scientistcontrasts with · A data scientist is a professional who applies statistical, computational, analytical, and domain methods to analyze data, derive insights, and build data-driven models or systems.
- data engineeringimplements · Data engineering is the discipline of designing and operating systems and processes that collect, transform, integrate, store, and deliver data for computational use.
These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.