12 Parallel workflow and agent programming with TPL Dataflow

 

This chapter covers

  • Using TPL Dataflow blocks
  • Constructing a highly concurrent workflow
  • Implementing a sophisticated Producer/Consumer pattern
  • Integrating Reactive Extensions with TPL Dataflow

Today’s global market requires that businesses and industries be agile enough to respond to a constant flow of changing data. These workflows are frequently large, and sometimes infinite or unknown in size. Often, the data requires complex processing, leading to high throughput demands and potentially immense computational loads. To cope with these requirements, the key is to use parallelism to exploit system resources and multiple cores.

But today’s .NET Framework’s concurrent programming models weren’t designed with dataflow in mind. When designing a reactive application, it’s fundamental to build and treat the system components as units of work. These units react to messages, which are propagated by other components in the chain of processing. These reactive models emphasize a push-based model for applications to work, rather than a pull-based model (see chapter 6). This push-based strategy ensures that the individual components are easy to test and link, and, most importantly, easy to understand.

This new focus on push-based constructions is changing how programmers design applications. A single task can quickly grow complex, and even simple-looking requirements can lead to complicated code.

12.1 The power of TPL Dataflow

 
 
 

12.2 Designed to compose: TPL Dataflow blocks

 
 

12.2.1 Using BufferBlock<TInput> as a FIFO buffer

 
 
 

12.2.2 Transforming data with TransformBlock<TInput, TOutput>

 
 
 

12.2.3 Completing the work with ActionBlock<TInput >

 
 
 

12.2.4 Linking dataflow blocks

 
 

12.3 Implementing a sophisticated Producer/Consumer with TDF

 
 

12.3.1 A multiple Producer/single Consumer pattern: TPL Dataflow

 
 

12.3.2 A single Producer/multiple Consumer pattern

 

12.4 Enabling an agent model in C# using TPL Dataflow

 
 

12.4.1 Agent fold-over state and messages: Aggregate

 
 

12.4.2 Agent interaction: a parallel word counter

 
 

12.5 A parallel workflow to compress and encrypt a large stream

 
 
 
 

12.5.1 Context: the problem of processing a large stream of data

 
 
 

12.5.2 Ensuring the order integrity of a stream of messages

 
 

12.5.3 Linking, propagating, and completing

 
 
 

12.5.4 Rules for building a TDF workflow

 

12.5.5 Meshing Reactive Extensions (Rx) and TDF

 
 

Summary

 
 
 
sitemap

Unable to load book!

The book could not be loaded.

(try again in a couple of minutes)

manning.com homepage
test yourself with a liveTest