LINQ Parallel execution

First, Parallel LINQ

1. 1. Parallel Query

. NET4 contains a new class paralleienumerable in the System.Linq namespace, which can be decomposed across multiple threads by decomposing the work of the query. Although the Enmerable class defines an extension method for the Ienunerable<t> interface,

Most extension methods of the Paralleienumerable class are extensions of the Parallelquery<tsource> class. An important exception is the AsParallel () method, which extends the Ienumerable<tsource> interface and returns the Parallelquery<tsource> class, so the normal collection class can be queried in parallel.

Cases:

const int arraySize = 100000000;

var data = new Int[arraysize];

var r = new Random ();

for (int i = 0; i < arraySize; i++)

{

Data[i] = R.next (40);

}

You can now filter the data by using a LINQ query to get the sum of the filtered data. The query uses a WHERE clause to define a filter that only corresponds to an item with a value less than 20, and then calls the aggregate function sum () method. The only difference from the previous LINQ query is that this time the AsParallel () method is called.

var sum = (from x in data. AsParallel ()

where x < 20

Select x). Sum ();

As with the previous LINQ query, the compiler modifies the syntax to invoke the AsParallel (), Where (), Select (), and sum () methods. The AsParallel () method is defined with the ParallelEnumerable class to extend the Ienumerable<t> interface, so it is called on simple data. The AsParallel () method returns parallelquery<tsource>. Because of the type returned, the Where () method chosen by the compiler is parallelenumerable.where () instead of Enumerable.where (). In the following code, the Select () and sum () methods are also from the ParallelEnumerable class. Contrary to the implementation code of the enumerable class, for the ParallelEnumerable class, the query is partitioned so that multiple threads can process the query at the same time. An array can be divided into sections, where each part is processed by a different thread to filter the remaining items. Once you have finished working on the partition, you need to merge to get the sum of all the parts.

var sum=data. AsParallel (). Where (X=>X<20). Select (x=>x). Sum ();

Running this line of code starts the Task Manager, so you can see that all the CPUs on the system are busy. If you delete the AsParallel () method, it is not possible to use multiple CPUs. Of course, if you don't have multiple CPUs on your system, you won't see improvements in parallel versions.

1. 2. partition Device

The AsParallel () method extends not only the Ienumerable<t> interface, but also the partition class. It allows you to influence the partition you want to create.

The Partitioner class is defined with the System.Collection.Concurrent namespace and has different variants. The Create method accepts an array or an object that implements the Ilist<t> class. According to this, as well as the type of the parameter LoadBalance and some overloaded versions of the method, a different partitioner type is returned. for arrays,. Net4 contains dynamicpartitionerforarray<tsource> derived from orderablepartitioner<tsource> of the abstract base class Classes and Staticpartitionerforarray<tsource> classes.

var q1 = (from X in Partitioner.create (data). AsParallel ()

where x < 20

Select x). Sum ();

You can also call the Withexecutionmode () and Withdegreeofparallelism () methods to pass a default value of Parallelexecutionmode or a forceparallelism value. By default, parallel LINQ avoids the use of a high-overhead parallelism mechanism. For the Withdegreeofparallelism () method, you can pass an integer value to specify the maximum number of tasks to run in parallel.

Cases:

const int arraySize = 100000000;

var data = new Int[arraysize];

var r = new Random ();

for (int i = 0; i < arraySize; i++)

{

Data[i] = R.next (40);

}

Stopwatch watch = new Stopwatch ();

Watch. Start ();

A notation that does not add dynamic load balancing, the time required to execute 1300 milliseconds

var q1 = (from X in Partitioner.create (data). AsParallel ()

where x < 80

Select x). Sum ();

In the second notation, dynamic load balancing is added, and the time required to complete the execution is 660 milliseconds.

var q1 = (from X in Partitioner.create (data,true). AsParallel ()

where x < 80

Select x). Sum ();

Watch. Stop ();

Console.WriteLine (watch. Elapsedmilliseconds.tostring ());

1. 3. Cancellation

. NET provides a standard way to cancel long-running tasks, which also applies to parallel LINQ. To cancel a long query, you can add the WithCancellation () method to the query and pass a CancellationToken token as a parameter. CancellationToken tokens are created from the CancellationTokenSource class. The query runs in a separate thread, capturing an exception of type operationcancelexception in that thread. This exception is triggered if the query is canceled. In the main thread, the Cancel () method of the CancellationTokenSource class is called to cancel the task.

const int arraySize = 100000000;

var data = new Int[arraysize];

var r = new Random ();

for (int i = 0; i < arraySize; i++)

{

Data[i] = R.next (40);

}

var cts = new CancellationTokenSource ();

New Thread (() =

{

Try

{

var sum = (from x in data. AsParallel (). WithCancellation (CTS. Token)

where x < 80

Select x). Sum ();

Console.WriteLine ("Query finished, sum: {0}", sum);

}

catch (OperationCanceledException ex)

{

Console.WriteLine (ex. Message);

}

}). Start ();

Console.WriteLine ("Query Started");

Console.Write ("Cancel");

int input = Console.read ();

if (input = = ' Y ' | | input = = ' Y ')

{

cancel!

Cts. Cancel ();

}

LINQ Parallel execution