Data transformation
A data transformation is an operation that maps input data or representations to output by changing their values, structure, or form.
The Joy of JavaScript
A map is a function or method that applies a transformation to each element or value in a structure and returns a corresponding structure.
- How it is used
- Transforms array elements, contained values, and observable events as stages in composable data pipelines.
- How deep it goes
- Works through array pipelines, the map/compose correspondence, shared object contracts, and an observable operator implementation; leaves full ADT theory aside.
- Examples
- Uses block validation, wallet-balance calculation, mapped Success values, and a number-sequence observable.
- What you can do after
- Build it from scratch as an observable operator, or apply it to compose array and object transformations.
Read how The Joy of JavaScript explains data transformation →
Statistics Every Programmer Needs
map is a pandas Series method that applies a mapping or function to each element and returns the transformed values.
- How it is used
- Prepares categorical outcomes and predictors as numeric values for logistic regression, decision trees, and random forests.
- How deep it goes
- Demonstrates dictionary-based Series mapping and output checks, then distinguishes encoding from conditional and distribution-shaping transformations.
- Examples
- Uses raisin-variety labels mapped to 0 and 1, plus football venue and play-type labels mapped to numeric codes.
- What you can do after
- Apply it via pandas `map()` to encode categories, then inspect the resulting values and non-null counts before modeling.
Read how Statistics Every Programmer Needs explains data transformation →
Learn AI Data Engineering
Data transformation is the process of changing data's structure, representation, or values to meet downstream requirements.
- How it is used
- Prepares raw pipeline data for reliable storage, analysis, enrichment, and downstream consumption.
- How deep it goes
- Works through cleaning rules, nested-data flattening, structured AI outputs, and semantic labeling rather than transformation theory.
- Examples
- Uses messy retail orders, surf-shop records, nested library JSON, news articles, and baseball play-by-play.
- What you can do after
- Apply it via Python, pandas, and Chat Completions to clean, flatten, validate, categorize, and enrich records.
Read how Learn AI Data Engineering explains data transformation →
Learn Rust in a Month of Lunches
map is an iterator adaptor that applies a transformation to each item and returns a new iterator producing the transformed items.
- How it is used
- Uses map to transform iterator items or array elements into results for further use.
- How deep it goes
- Explains lazy evaluation, collection, ownership choices, and the map signature’s closure bound.
- Examples
- Demonstrates doubling vector values, converting array integers to strings, and classifying hours with an enum.
- What you can do after
- Apply it via Rust’s iterator and array map methods to transform values and collect iterator results.
Read how Learn Rust in a Month of Lunches explains data transformation →
Snowflake Data Engineering
Data transformation is the conversion of data from a raw source representation into information that satisfies defined requirements.
- How it is used
- Transforms ingested Snowflake data into consumer-ready information between ingestion and presentation.
- How deep it goes
- Progresses from SQL summaries to JSON flattening, stored procedures, validation, exception handling, and logging.
- Examples
- Uses bakery CSV orders, staging tables, COPY, MERGE, reporting summaries, scheduled tasks, and JSON files.
- What you can do after
- Build and automate Snowflake transformations using SQL, staging tables, MERGE, transformed tables, and scheduled tasks.
Read how Snowflake Data Engineering explains data transformation →
Platform Engineering on Kubernetes
A transformation function is a function that maps an incoming event into a different event representation, optionally enriching its data.
- How it is used
- Converts collected CloudEvents into CDEvents that metric calculations can consume.
- How deep it goes
- Works through a concrete event mapping and an asynchronous, scheduled function architecture, but does not show transformation code in the supplied evidence.
- Examples
- Maps a Knative API server Deployment event to a service-deployed CDEvent and uses a Kubernetes-based deployment-frequency example.
- What you can do after
- Operate/configure it by scheduling transformation functions and feeding their CDEvents into deployment-frequency calculations.
Read how Platform Engineering on Kubernetes explains data transformation →
Kotlin in Action, 2E
map is a Kotlin standard-library operation that transforms each element and collects the resulting values into a new collection.
- How it is used
- Transforms each collection element into a new form as part of functional collection manipulation.
- How deep it goes
- Defines element-wise transformation, contrasts filtering, and demonstrates lambdas, member references, chaining, indexed mapping, and map values.
- Examples
- Demonstrates squaring numbers, extracting Person names, filtering by age before mapping, indexed mapping, and uppercasing map values.
- What you can do after
- Apply it via the Kotlin standard library to replace manual element-by-element collection transformations.
Read how Kotlin in Action, 2E explains data transformation →
Data-Oriented Programming in Java
Data transformation is the process of producing successive data states by changing or converting an existing representation.
- How it is used
- Models domain behavior as data flowing through named input and output types.
- How deep it goes
- Develops transformation sequences and lifecycle states while deliberately deferring implementation details during design.
- Examples
- Illustrates a tossed ball's successive states and an invoice-to-late-fee pipeline with Draft and Open stages.
- What you can do after
- Apply it via named domain types to sketch behaviors and translate semantic values into legacy entities.
Read how Data-Oriented Programming in Java explains data transformation →
Machine Learning for Tabular Data: XGBoost, Deep Learning, and AI
A feature transformation maps input feature values to a different representation, often to simplify relationships for a model.
- How it is used
- Uses feature transformations to prepare classical machine-learning inputs and express nonlinearities or interactions more effectively.
- How deep it goes
- Explains transformation purposes and several strategies, then uses partial dependence to diagnose relationships without deriving transformation mathematics.
- Examples
- Uses linear-form transformations, binning, discretization, precomputed interactions, partial dependence plots, and noisy nonlinear-function model comparisons.
- What you can do after
- Diagnose nonlinearities with partial dependence plots and choose transformations that support model representation and split efficiency.
Deep Learning with Python, Third Edition
A transformation that maps data or representations within or between geometric spaces.
- How it is used
- Uses transformations to explain learned representations and to prepare datasets for image and text models.
- How deep it goes
- Explains geometric transformations through an analogy and demonstrates dataset mapping, without a formal geometric derivation.
- Examples
- Illustrates transformations with crumpled paper classes, reshaped vectors, and bag-of-words review encoding.
- What you can do after
- Apply it via TensorFlow Dataset.map to reshape elements or vectorize text, and evaluate learned transformations as task-specific mappings.
Read how Deep Learning with Python, Third Edition explains data transformation →
Effective Data Analysis
A data transformation is a mathematical or computational modification of data values intended to change their representation, scale, distribution, or modeled relationship.
- How it is used
- Uses transformations to improve statistical assumptions and represent relationships in t-tests and linear regression.
- How deep it goes
- Explains exploratory diagnosis, predictor and outcome transformations, and iterative model evaluation, but does not catalog transformation families.
- Examples
- Demonstrates squared and square-root wind speed, square-root rat sightings, weather outlier removal, and Boston–New York weather testing.
- What you can do after
- Diagnose model problems and apply transformations while reporting what the transformed results can and cannot mean to stakeholders.
Read how Effective Data Analysis explains data transformation →
Math and Architectures of Deep Learning
A mapping between geometric spaces, used here as the view of a machine learning model as mapping input points to output points.
- How it is used
- Uses geometric mappings to place machine-learning data in spaces better suited to classification or simplification.
- How deep it goes
- Defines mappings through vectors and matrices, then connects them to manifolds and neural-network layers without implementation details.
- Examples
- Uses rotation, three-dimensional projection, cylindrical unrolling, and successive neural-network layer mappings.
- What you can do after
- Evaluate machine-learning models as geometric mappings and understand why transformed representations can enable linear classification.
Read how Math and Architectures of Deep Learning explains data transformation →
F# in Action: A Revised Edition of Get Programming with F#
A data transformation is a computation that converts input data into a different value or representation.
- How it is used
- Frames calculations and collection processing as successive conversions of data, separate from final state changes.
- How deep it goes
- Works through expression-based branching and an imperative-versus-declarative collection query, while deferring details of individual pipeline stages.
- Examples
- Uses an age-based description and an away-win query that filters results, counts wins by team, and selects the maximum.
- What you can do after
- Apply it via F# List.filter, List.countBy, and List.maxBy to build a collection query without mutable summary loops.
Read how F# in Action: A Revised Edition of Get Programming with F# explains data transformation →
Kafka for Architects
A map operation transforms each input event into a corresponding event in another format or representation.
- How it is used
- Frames transformation as an architectural choice for preparing, integrating, and reshaping event data across systems.
- How deep it goes
- Explains stateless mappings, transformation locations, and trade-offs, while deferring complex transformations to streaming frameworks.
- Examples
- Uses type casting, filtering, renaming, format conversion, Customer 360 joins, event mapping, routing, and key changes.
- What you can do after
- Apply it via Kafka Connect or streaming frameworks to reshape, filter, route, join, or aggregate event data.
Read how Kafka for Architects explains data transformation →
Python How-To
- How it is used
- Transforms existing Python iterables into new values or containers for subsequent calculations.
- How deep it goes
- Explains map conversion, zip-based dictionary construction, list-comprehension alternatives, and map's status as a class.
- Examples
- Uses character strings, numeric strings, paired task identifiers and titles, and tuples with repeated values.
- What you can do after
- Apply map, list comprehensions, zip, and dict to convert iterable data into values or containers.
Idiomatic Rust
map is an iterator or result-like-type method that applies a function to each available value and returns the transformed result.
- How it is used
- Transforms iterator items or present Option and Result values.
- How deep it goes
- Defines map's closure signature and laziness, then demonstrates collection and Result handling without implementing the iterator.
- Examples
- Demonstrates converting integers to strings, parsing strings, and transforming ranked dog-breed entries.
- What you can do after
- Apply map to transform iterator items or present Option and Result values, then collect or chain the results.
Delivering Machine Learning Projects: From design to deployment
Data transformation is the conversion of data into a different structure, representation, or format.
- How it is used
- Uses transformations to prepare distributed, unreliable data for exploration, model training, evaluation, and reproducible delivery.
- How deep it goes
- Explains aggregation, rebalancing, formatting, filtering, augmentation, and checkpointing, but does not detail transformation algorithms.
- Examples
- Uses consolidated tables or streams, standardized unstructured files, filtered corrupt items, rebalanced data, and augmented training examples.
- What you can do after
- Builds maintainable transformations that produce current, checkpointed, consumable data for model development and evaluation.
100 C++ Mistakes and How to Avoid Them
Mapping is a data-processing operation that applies a function independently to each input element and produces one output element per input element.
- How it is used
- Uses mapping to transform each value in a range pipeline before passing results to later operations.
- How deep it goes
- Defines one-to-one mapping and illustrates it with absolute values, without detailing implementation internals.
- Examples
- Shows absolute-value transformation followed by even-number filtering and accumulation.
- What you can do after
- Apply it via std::views::transform to map values in a range pipeline.
Read how 100 C++ Mistakes and How to Avoid Them explains data transformation →
Deep Learning with R, Third Edition
A data transformation is an operation that changes the form or encoding of data while preserving information relevant to a task.
- How it is used
- Uses transformed representations to make task-solving rules simpler and less brittle.
- How deep it goes
- Defines several transformation types and frames their selection as hypothesis-space search, without deriving algorithms.
- Examples
- Uses handwritten digits, pixel histograms, closed-loop counts, coordinate changes, projections, translations, and nonlinear selections.
- What you can do after
- Evaluate transformations by how well their representations support simpler rules on development data.
Read how Deep Learning with R, Third Edition explains data transformation →
Grokking AI Applications
Data transformation is the modification, restructuring, cleaning, or reformatting of data to meet processing or storage requirements.
- How it is used
- Applies LLM reasoning to structured business data arriving from files, databases, and enterprise systems.
- How deep it goes
- Explains transformation through KNIME nodes that load, join, sort, aggregate, and reshape data for LLMs.
- Examples
- Uses CRM records, spreadsheets, databases, generated strings, and separated output fields as transformation examples.
- What you can do after
- Apply it via KNIME and Langflow to prepare structured inputs and integrate transformed outputs with downstream systems.
Read how Grokking AI Applications explains data transformation →
Advanced Algorithms and Data Structures
A map operation transforms each input record or partition into intermediate information relevant to a later computation.
- How it is used
- Uses map operations to extract intermediate information before distributed aggregation.
- How deep it goes
- Defines partition-level mapping and its handoff to reduction, while deferring detailed implementation beyond coin recognition and output handling.
- Examples
- Uses coin-recognition output containing unsorted denomination occurrences as the concrete mapped artifact.
- What you can do after
- Apply map operations to extract intermediate records that later reducers can count, sum, or filter.
Read how Advanced Algorithms and Data Structures explains data transformation →
Software Testing with Generative AI
The process of converting data from one structure, representation, or language into another while preserving its intended information.
- How it is used
- Accelerates testing by converting data, code, and notes into formats needed for exploratory testing, debugging, and automation.
- How deep it goes
- Demonstrates reusable prompts and field-completeness checks, but defers final validation because outputs may be inconsistent or invalid.
- Examples
- Uses language translation, JSON-to-SQL conversion, cross-language code conversion, and testing-note summarization as concrete transformations.
- What you can do after
- Apply it via reusable LLM prompts to generate or convert test data while checking structure, relationships, and preserved values.
Read how Software Testing with Generative AI explains data transformation →
Fast Python: High performance techniques for large datasets
- How it is used
- Guides selection of efficient Python solutions for data-processing problems.
- How deep it goes
- Frames architectural trade-offs and practical comparisons but does not detail a specific transformation implementation.
- What you can do after
- Evaluate computing architectures and choose performance strategies suited to available resources and goals.
Read how Fast Python: High performance techniques for large datasets explains data transformation →
Also covered in
- Data-Oriented Programming: Reduce software complexity
- Tiny Python Projects: Learn coding and testing with puzzles and games
- Grokking Simplicity
- Functional Programming in Scala, Second Edition
- Python Concurrency with asyncio
- Functional Programming in JavaScript
- High Performance Unity Game DevelopmentIn early access (MEAP)
- Python Workout: 50 ten-minute exercises
- Inside Deep Learning: Math, Algorithms, Models
- Python Workout, Second Edition
- AI Agents and Applications
- DuckDB in Action
What to know first
- functionA function is a callable unit of program code that performs a computation or operation, can accept inputs, and may produce a result or side effect.
- MapA map is an associative data structure that stores key-value associations and supports retrieving values by key.
- arrayAn array is an ordered collection of elements organized in one or more dimensions and accessible by position or index.
- functional programmingFunctional programming is a programming paradigm that expresses computation through the evaluation and composition of functions, typically favoring immutable data and minimizing mutable state and side effects.
Related concepts
- functionuses · A function is a callable unit of program code that performs a computation or operation, can accept inputs, and may produce a result or side effect.
- iteratorproduces · An iterator is an object or abstraction that provides successive values from a collection or other data source through a traversal interface.
- arrayuses · An array is an ordered collection of elements organized in one or more dimensions and accessible by position or index.
- pipelinepart of · A pipeline is an ordered sequence of connected processing stages through which data or work flows, with each stage passing or transforming its output for subsequent stages.
- comprehensionalternative to · A comprehension is a language construct that creates a collection by iterating over one or more sources, optionally filtering elements and transforming them.
- higher-order functionuses · A higher-order function is a function that accepts functions as arguments, returns functions as results, or both.
Questions about data transformation
What is data transformation?
A data transformation is an operation that maps input data or representations to output by changing their values, structure, or form.
Which Manning books explain data transformation?
The Joy of JavaScript, Statistics Every Programmer Needs, Learn AI Data Engineering, Learn Rust in a Month of Lunches, Snowflake Data Engineering, Platform Engineering on Kubernetes, Kotlin in Action, 2E, Data-Oriented Programming in Java and 15 more.
What should I know before learning data transformation?
Manning books that teach data transformation usually introduce function, Map, array and functional programming first.
What is the difference between data transformation and comprehension?
A data transformation is an operation that maps input data or representations to output by changing their values, structure, or form. A comprehension is a language construct that creates a collection by iterating over one or more sources, optionally filtering elements and transforming them.
What is data transformation part of?
Data transformation is part of pipeline. A pipeline is an ordered sequence of connected processing stages through which data or work flows, with each stage passing or transforming its output for subsequent stages.
What is data transformation also called?
Data transformation is also called transform.
These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.






















