Dataset splitting divides data among threads so they can perform concurrent work. It is one way to structure multithreaded tasks, and those design choices affect performance and code clarity. section 8.0 section 8.1
Dataset split
Data-based work division assigns different portions of input data to different threads for processing.
How C++ Concurrency in Action, Second Edition covers it
- How deep it goes
- Introduces data division as the first among several techniques for dividing work between threads.
- What you can do after
- Evaluate data division as an option when choosing how threads should perform work.
- How it connects
- Links thread assignment with concurrent application design, performance, and code clarity.
Also explained in
-
Advanced Algorithms and Data StructuresSharding is the technique of partitioning data, users, or transactions across different machines or clusters.
-
LatencyA partition is a distinct portion of a dataset assigned to a node for storage or processing.
-
Machine Learning Platform EngineeringDataset splitting is the partitioning of a dataset into subsets for separate training, validation, testing, or other processing purposes.
-
Amazon Web Services in Action, Third Edition: An in-depth guide to AWSSharding is the technique of partitioning a key-space or dataset across multiple nodes so each node stores a distinct portion.
-
Apache Kafka in ActionSharding divides data across multiple subsystems or database nodes, typically by a shard key.
-
Microsoft Azure in ActionA partition is a subset of database data stored separately from other subsets for distribution and access management.
-
Natural Language Processing in Action, Second EditionA train-test split partitions examples into separate subsets for model fitting and evaluation.
-
Machine Learning System DesignA procedure or method for partitioning data into separate subsets for training, validation, testing, or comparative evaluation.
-
Kafka for ArchitectsPartitioning is the distribution of records across separate partitions so data processing and storage can be divided across resources.
-
Deep Learning with R, Third EditionSharding is the partitioning of a variable or data object into distinct chunks assigned to different devices.
-
Deep Learning with Python, Third EditionOne of the K disjoint subsets created when data is split for cross-validation.
-
Causal Inference for Data ScienceA train-test split partitions observations into a training subset for fitting a model and a test subset for evaluating its predictions.
-
Grokking ConcurrencyPartitioning is a parallel algorithm design technique that identifies independent computational tasks or data subsets for concurrent execution.
-
Cyber Threat HuntingA train-test split partitions a labeled data set into separate subsets for model training and performance evaluation.
-
Transformers in ActionSharding is the partitioning of model parameters, data, or computation across multiple devices or storage units.
-
Hugging Face in ActionA dataset split is a partition of a dataset designated for a particular stage or purpose in machine-learning workflows.
-
Graph Neural Networks in ActionA partition of a dataset into training and testing subsets for model fitting and evaluation.
-
Grokking Algorithms, Second EditionPartitioning is the process of dividing a sequence around a selected pivot into groups based on their comparison with the pivot.
-
Graph Algorithms for Data ScienceA dataset split partitions available data into separate subsets for purposes such as feature generation, training, validation, and testing.
Compare how 21 books explain dataset split →
Related concepts
- pivotuses · In partitioning, a pivot is a selected element whose value divides other elements into groups according to their relation to it.
- quicksortused by · Quicksort is a comparison-based sorting algorithm that partitions elements around a pivot and recursively sorts the resulting subranges.
- vertical partitioningincludes · Vertical partitioning is a technique that divides tabular data into separate portions containing different subsets of columns while preserving row relationships.
- Elasticsearch indexpart of · An Elasticsearch index is a named, searchable collection of documents organized for querying and distributed across storage shards and replicas.
This explainer was generated by Manning's knowledge graph from the text of C++ Concurrency in Action, Second Edition. The section links point to the parts of the book it draws on.