concept data parallelism in category .net
This is an excerpt from Manning's book Concurrency in .NET.
In the coming chapters, we’ll discuss alternative concurrent approaches, such as data parallelism, asynchronous, and the message-passing programming model. We’ll build libraries using the best tools that each of these programming languages can offer and compare those with other languages. We’ll also examine tools and libraries like the TPL and Reactive Extensions (Rx) that have been successfully designed, inspired, and implemented by adopting the functional paradigm to obtain composable abstraction.
4.1 What is data parallelism?
Data parallelism is a programming model that performs the same set of operations on a large quantity of data in parallel. This programming model is gaining traction because it quickly processes massive volumes of data in the face of a variety of big data problems. Parallelism can compute an algorithm without requiring reorganization of its structure, thereby progressively increasing scalability.
Data parallelism targets the distribution of a given data set into smaller partitions across multiple tasks, where each task performs the same instruction in parallel. For example, data parallelism could refer to an image-processing algorithm, where each image or pixel is updated in parallel by independent tasks. Conversely, task parallelism would compute in parallel a set of images, applying a different operation for each image. See figure 4.2. Figure 4.2 Data parallelism is the simultaneous execution of the same function across the elements of a data set. Task parallelism is the simultaneous execution of multiple different functions across the same or different data sets.
Is summary, task parallelism focuses on executing multiple functions (tasks), and aims to reduce the overall time of computation by running these tasks simultaneously. Data parallelism reduces the time it takes to process a data set by splitting the same algorithm computation among multiple CPUs to be performed in parallel.