Intel TBB: Pipeline, processing data in order

In the last article (TBB: Pipeline, the power of the software pipeline), we finally raised several questions. Let's take a look at how TBB: Pipeline solves them one by one.

 

 

Why can pipeline ensure the sequence of Data Execution? Since TBB executes tasks through multiple threads in the final analysis, why is the string read after reading two strings first processed by the next task? Is there something in pipeline similar to FIFO first-in-first-out queue?

 

 

I have previously questioned the performance of pipeline and even wanted to simulate a pipeline by myself using multithreading, but soon I found the difficulties in implementation. The sequence of Data Execution is one of them.

 

Assume that a thread represents a node in the pipeline. If a node is executed concurrently, more than two threads (A and B) are required ), is the sequence data processed by the previous node first sent to A or B? After processing is completed, should data in A or B be sent to the next node? Even if priority rules between A and B can be manually specified, there are still many unpredictable difficulties in actual operation due to the uncertainty of thread scheduling.

 

A notable feature of the pipeline is to ensure that each data flow through each node in the same order. Therefore, TBB: a primary task in pipeline is to ensure the order of the processed data while the node is executed concurrently without additional processing code. In addition, for nodes requiring serial processing, ensure that even the data in the front line is processed first, even if the data in the back line arrives first.

 

The central idea of pipeline is to use tokens to control the data processing sequence and pipeline depth. The pipeline: Run function specifies the maximum value of the token:

 

Void pipeline: Run (size_t max_number_of_live_tokens ){}

 

 

Each data entry to pipeline will be allocated a token in sequence, such as line1:

 

Task * stage_task: Execute (){

_ Tbb_assert (! My_at_start |! My_object, null );

If (my_at_start ){

If (my_filter-> is_serial ()){

If (my_object = (* my_filter) (my_object ))){

My_token = my_pipeline.token_counter ++; // line1

My_token_ready = true;

Itt_policy (sync_releasing, & my_pipeline.input_tokens );

If (-- my_pipeline.input_tokens> 0)

Spawn (* New (allocate_additional_child_of (* my_pipeline.end_counter) stage_task (my_pipeline ));

} Else {

My_pipeline.end_of_input = true; // line2

Return NULL;

}

...

}

 

If all the tokens in the current pipeline are used up, new data will not be processed until the data that has entered pipeline is processed and there are idle tokens (line2)

 

Taking text_filter in TBB as an example, the pipeline is myinputfilter-> mytransformfilter-> myoutputfiler, myinputfilter reads data from the disk, mytransformfilter converts data into uppercase letters, and myoutputfilter writes the converted data to the disk. Therefore, the myinputfilter node and myoutputfiler node must be executed in serial mode, while mytransformfilter can be executed concurrently. For a string of ordered data read by myinputfilter, the token is 1-> 2-> 3. How can we ensure that the converted data is written to the disk in the same order?

 

The secret lies in a class TBB: Internal: ordered_buffer in TBB. myoutputfilter uses this class to ensure that the data in its queue is executed in the order of tokens, regardless of the order in which the data enters the queue. In other words, even if token 2 is sent to myoutputfilter after being processed by a mytransformfilter node, As long as token 1 does not arrive and is not executed by myoutputfilter, data 2 will not be written to the disk before data 1. Each node that requires serial processing has an ordered_buffer member variable.

 

Let's take a look at the definition of ordered_buffer:

 

//! A buffer of Ordered items.

/** Each item is a task, inserted into a position in the buffer when writing ding to a token .*/

Class ordered_buffer {

Typedef token size_type;

 

//! Array of deferred tasks that cannot yet start executing.

/** Element is null if unused .*/

Task ** array; // array, which stores all tasks to be processed in sequence.

 

//! Size of Array

/** Always 0 or a power of 2 */

Size_type array_size; // array size

 

//! Lowest token that can start executing.

/** All prior token have already been seen .*/

Token low_token; // the token currently being processed,

 

//! Serializes updates.

Spin_mutex array_mutex; // The lock used to protect concurrent access to the array

};

 

Still in task * stage_task: Execute (){

...

If (ordered_buffer * input_buffer = my_filter-> input_buffer ){

// The next filter must execute tokens in order.

Stage_task & clone = * New (allocate_continuation () stage_task (my_pipeline, my_filter );

Clone. my_token = my_token; // token number

Clone. my_token_ready = my_token_ready;

Clone. my_object = my_object; // data

Next = input_buffer-> put_token (clone); // put the task into the queue

} Else {

/* A semi-hackish way to reexecute the same task object immediately without spawning.

Recycle_as_continuation marks the task for future execution,

And then 'eas' pointer is returned to bypass spawning .*/

Recycle_as_continuation ();

Next = this;

}

} Else {

...

}

 

For nodes that require serial processing, use the put_token function of ordered_buffer to put relevant data and task references into the queue. Put_token implementation is the key:

 

Template <typename stagetask>

Task * put_token (stagetask & putter ){

Task * result = & putter;

{

Spin_mutex: scoped_lock lock (array_mutex );

Token token = putter. next_token_number ();

If (Token! = Low_token ){

// Trying to put token that is beyond low_token.

// Need to wait until low_token catches up before dispatching.

Result = NULL;

_ Tbb_assert (tokendiff_t) (token-low_token)> 0, null );

If (token-low_token> = array_size)

Grow (token-low_token + 1 );

Itt_policy (sync_releasing, this );

Array [token & array_size-1] = & putter;

}

}

Return result;

}

The essence of this function is to first obtain the token to be processed, and then put the task to be executed into the "proper position" in the ordered_buffer task queue ", low_token points to the token number to be processed.

 

For example, low_token = 0. Currently, the token 0 must be processed, and the next token is 1. Therefore, the task is saved at array [1] and blocked. After the token 0 is processed, add 1 to low_token, and retrieve the task corresponding to token 1 from the array for processing.

 

In pipeline, a serial node is notified in this way to process a piece of data:

Or in task * stage_task: Execute (){

...

If (ordered_buffer * input_buffer = my_filter-> input_buffer)

Input_buffer-> note_done (my_token, * This );

...

}

 

Let's take a look at the implementation of note_done! If the newly completed token is the top-priority token (low_token), extract the next task to be executed and dispatch the task schedwn of TBB in the form of spawn:

 

 

//! Note that processing of a token is finished.

/** Fires up processing of the next token, if processing was deferred .*/

Void note_done (token, task & spawner ){

Task * wakee = NULL;

{

Spin_mutex: scoped_lock lock (array_mutex );

If (token = low_token ){

// Wake the next task

Task * & item = array [++ low_token & array_size-1];

Itt_policy (sync_acquired, this );

Wakee = item;

Item = NULL;

}

}

If (wakee ){

Spawner. spawn (* wakee );

}

}

 

 

 

 

Ordered_buffer is a very interesting implementation. Compared to the common use of FIFO queue to implement data transmission between threads, ordered_buffer is exquisite. We can take advantage of the ordered_buffer principle to further improve our code.

About Author:

Softarts has worked in Alcatel-Lucent and Nokia, and is engaged in telecommunications system software R & D. Research interests: C ++, multi-core computing, Linux

Reprinted please indicate the source

 

This article from the csdn blog, reproduced please indicate the source: http://blog.csdn.net/softarts/archive/2009/04/28/4134806.aspx