Data representation
A data representation is a form or encoding that structures information for interpretation, processing, storage, or exchange.
Timeless Algorithms, The Foundational Ideas
A representation encodes informative properties of data in a form suitable for processing or interpretation.
- How it is used
- Uses learned representations to make high-dimensional data easier to process and to simplify downstream prediction.
- How deep it goes
- Explains layer transformations, representational width and depth, and the geometry of classification, while treating visual layer meanings as illustrative.
- Examples
- Illustrates image compression, pedestrian detection, and a six-observation transformation that makes classes linearly separable.
- What you can do after
- Evaluates how a representation can simplify prediction and why training-set separation may not generalize.
Read how Timeless Algorithms, The Foundational Ideas explains data representation →
Kafka for Architects
A data format is a defined representation and structure for encoding data exchanged between software components.
- How it is used
- Defines the representation producers and consumers agree on when exchanging Kafka events across teams.
- How deep it goes
- Explains schema and serialization roles, compares three formats, and traces wrong-format failures; stops short of a full implementation walkthrough.
- Examples
- Uses customer-data events, a Client schema, serializer settings, and a format comparison table.
- What you can do after
- Evaluate/understand format trade-offs, document an event structure, align clients, and plan for invalid messages.
Read how Kafka for Architects explains data representation →
API Design Patterns
A serialization format is a defined representation for converting structured data into a transmissible or storable form and reconstructing it later.
- How it is used
- Uses serialization formats to define predictable API values across programming languages and client-server exchanges.
- How deep it goes
- Explains type loss, encoding, normalization, and identifier representation, but stops short of a complete formal schema.
- Examples
- Uses ambiguous addition versus concatenation, UTF-8 representations of è, and Crockford’s Base32 identifiers.
- What you can do after
- Apply it by choosing documented API types, standardizing encodings, and accounting for lossy or ambiguous representations.
Deep Learning with PyTorch, Second Edition
The representation scheme used to encode and store data for interchange or loading.
- How it is used
- Uses numerical representations to connect real-world inputs, neural-network processing, and interpretable outputs.
- How deep it goes
- Explains intermediate features and tensor serialization, but does not show an interoperable storage method.
- Examples
- Uses handwritten-digit features, image-recognition features, and a flattened image classifier.
- What you can do after
- Apply it via PyTorch to save and reload tensors, while recognizing the format’s interoperability limitation.
Read how Deep Learning with PyTorch, Second Edition explains data representation →
Deep Learning with R, Third Edition
A data representation is a particular encoding or view of data that can make a task easier to solve.
- How it is used
- Uses representations to make input data more amenable to a task and to connect machine-learning outputs with expected results.
- How deep it goes
- Explains RGB and HSV and point classification, then traces abstraction and information loss across convnet layers.
- Examples
- Uses RGB and HSV images, black-and-white point classification, and cat-image activations across convnet layers.
- What you can do after
- Understand how changing an input representation can suit a task and how convnet features become more abstract with depth.
Read how Deep Learning with R, Third Edition explains data representation →
Apache Kafka in Action
A data format is a specification for representing and organizing data for storage, transmission, and interpretation.
- How it is used
- Guides how Kafka producers encode messages and consumers interpret them without Kafka imposing a format.
- How deep it goes
- Compares readability, size, consistency, and serialization, while deferring detailed treatment of schemas and schema management.
- Examples
- Contrasts JSON with Protobuf and Avro, then uses Debezium ingestion to show why raw database formats may need consolidation.
- What you can do after
- Evaluate/understand format trade-offs and decide when consistent schemas or consolidated data structures will help consumers.
Read how Apache Kafka in Action explains data representation →
Machine Learning System Design
A defined representation or encoding that specifies how data is structured and stored for exchange or processing.
- How it is used
- Turns data into inputs an ML model can use, including learned representations reused across applied tasks.
- How deep it goes
- Defines representations as model inputs and gives pretrained-embedding examples, while deferring detailed feature engineering.
- Examples
- Uses ImageNet image representations and CLIP or SentenceTransformers embeddings as examples.
- What you can do after
- Apply pretrained-model representations to train a shallow classifier or store embeddings as reusable features.
Read how Machine Learning System Design explains data representation →
Machines that Think
A structured encoding of information that can support reasoning or computation.
- How it is used
- Explains how machines encode language and other inputs for learning and prediction.
- How deep it goes
- Explains tokenization, relational embeddings, and learned layer-by-layer representations without detailing their implementation.
- Examples
- Uses edges, shapes, and objects to illustrate abstraction and letter counting in “strawberry” to illustrate representational brittleness.
- What you can do after
- Evaluate how a system’s representation can support generalization while making some tasks unexpectedly difficult.
Modern C
A representation is the concrete encoding used to express a value inside a program or in an external file.
- How it is used
- Separates reasoning about C values from concerns about their concrete encodings to support portable code.
- How deep it goes
- Explains architecture- and type-dependent encodings, then relates evaluation to observable output and compiler optimization.
- Examples
- Uses decimal, Roman, and textual number notation, plus a double calculation printed to the terminal.
- What you can do after
- Understand why portable C code should reason primarily about values rather than machine-specific representations.
Starting Data Analytics with Generative AI and Python
A defined representation and encoding scheme for storing or exchanging data.
- How it is used
- Supports reliable integration between generative AI and analytics solutions by matching formats for requests and responses.
- How deep it goes
- Recommends compatible formats and data transformation, but does not name formats or explain their technical details.
- What you can do after
- Configure integrations with compatible request and response formats, transforming data when needed to prevent loss or misinterpretation.
Read how Starting Data Analytics with Generative AI and Python explains data representation →
Rust in Action: Systems programming concepts and techniques
A file format is a defined encoding and structural convention for representing data in files.
- How it is used
- Uses file formats to persist and retrieve data, inspect stored bytes, and implement durable key-value storage.
- How deep it goes
- Descends from ordered byte sequences and design trade-offs to record headers, parsing, and checksum verification.
- Examples
- Compares Shakespeare text, EPUB, and MOBI byte patterns, then traces a Bitcask key-value record.
- What you can do after
- Build a custom on-disk format and parse Bitcask records by reading headers, bodies, and checksums.
Read how Rust in Action: Systems programming concepts and techniques explains data representation →
Also covered in
- Data-Oriented Programming in JavaIn early access (MEAP)
- Data-Oriented Programming: Reduce software complexity
- The Design of Web APIs, Second Edition
- How Large Language Models Work
- Geometry for Programmers
- Get Programming: Learn to code with Python
- The Creative Programmer
- Modern C, Third Edition
- Acing the CCNA Exam: Volume 2 Advanced Networking and Security
- Java Persistence with Spring Data and Hibernate epub
- Graph Neural Networks in Action
- LLMs in Production
What to know first
- JSONJSON is a language-independent text-based data-interchange format that represents structured values using objects, arrays, strings, numbers, booleans, and null.
- language modelA language model is a computational model that represents patterns in linguistic sequences to assign probabilities, predict, generate, or evaluate language.
- parameterA parameter is a value or named input that specifies or influences the behavior, computation, or output of a function, model, or system.
- contractA contract is an agreed specification of the conditions, expectations, rights, or obligations that govern behavior or interactions between parties or software components.
Related concepts
- neural network layerproduced by · A neural network layer is a computational component or stage that transforms inputs into outputs within a neural network.
- schemacontrasts with · A data schema is a formal specification of the structure, fields, types, relationships, and constraints of data.
- YAMLincludes · YAML is a human-readable data serialization standard for representing structured data, commonly using indentation to express mappings and sequences.
- JSONincludes · JSON is a language-independent text-based data-interchange format that represents structured values using objects, arrays, strings, numbers, booleans, and null.
- language modelproduced by · A language model is a computational model that represents patterns in linguistic sequences to assign probabilities, predict, generate, or evaluate language.
- neural networkproduced by · A neural network is a computational model composed of interconnected, parameterized units or transformations that learn to map inputs to outputs.
Questions about data representation
What is data representation?
A data representation is a form or encoding that structures information for interpretation, processing, storage, or exchange.
Which Manning books explain data representation?
Timeless Algorithms, The Foundational Ideas, Kafka for Architects, API Design Patterns, Deep Learning with PyTorch, Second Edition, Deep Learning with R, Third Edition, Apache Kafka in Action, Machine Learning System Design, Machines that Think and 3 more.
What should I know before learning data representation?
Manning books that teach data representation usually introduce JSON, language model, parameter and contract first.
What is the difference between data representation and schema?
A data representation is a form or encoding that structures information for interpretation, processing, storage, or exchange. A data schema is a formal specification of the structure, fields, types, relationships, and constraints of data.
What kinds of data representation are there?
Kinds of data representation include YAML and JSON.
What is data representation also called?
Data representation is also called data format.
These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.










