C # parallel programming-PLINQ protocol operations and Aggregate functions

C # parallel programming-PLINQ protocol operations and Aggregate functions
Summary

PLINQ can simplify the process of applying the same function to all members of a sequence or group. This process is called a protocol operation. A Sum () function is a standard operation. PLINQ provides an interface that can overload Aggregate. Here you can define your own function.

A protocol operation is an operation performed on each member. After the operation is completed, the operation results may need to be summarized to obtain a final result. This is the concept of aggregation.

Protocol operation

In this example, we need to calculate the average number of vertices that can be divided by 5 in the range of 1 to 50000000 divided by PI. It can be completed by using LINQ or PLINQ.

Sample Code:

using System;using System.Text;using System.Threading;using System.Threading.Tasks;using System.Diagnostics;using System.Linq;using System.IO;using System.Collections.Generic;namespace Sample6_2_plinq_calculate{    class Program    {        static int NUM_INTS = 50000000;        static IEnumerable
  
    GenerateInputeData()        {            return Enumerable.Range(1, NUM_INTS);        }        static ParallelQuery
   
     GenerateInputeData4Parallel()        {            return ParallelEnumerable.Range(1, NUM_INTS);        }        static void Main(string[] args)        {            var seqTarget = GenerateInputeData();            Console.WriteLine("============================================================");            Console.WriteLine("TEST NORMAL LINQ");            Console.WriteLine("============================================================");            var swatchpn = Stopwatch.StartNew();            var seqQuery = (from intNum in seqTarget                            where ((intNum % 5) == 0)                            select (intNum / Math.PI)).Average();            swatchpn.Stop();            Console.WriteLine("LINQ Result: " + seqQuery + "    LINQ Use Time: {0}", swatchpn.Elapsed);            var palTarget = GenerateInputeData4Parallel();            Console.WriteLine("\n\n");            Console.WriteLine("============================================================");            Console.WriteLine("TEST PARALLEL LINQ");            Console.WriteLine("============================================================");            var swatchp = Stopwatch.StartNew();            var palQuery = (from intNum in palTarget.AsParallel()                            where ((intNum % 5) == 0)                            select (intNum / Math.PI)).Average();            swatchp.Stop();            Console.WriteLine("PLINQ Result: " + palQuery + "    LINQ Use Time: {0}", swatchp.Elapsed);            Console.ReadLine();        }    }}
   
  

Test results:

Aggregate operations

The sample code calculates the standard deviation, skewness, and peaks of an array to illustrate the use of aggregation.
By the way, make up the mathematics:

Standard Deviation: a standard used to measure the degree to which data values deviate from the arithmetic mean. The smaller the standard deviation, the less the deviation from the average value, and vice versa. The size of the standard deviation can be measured by the rate relationship between the standard deviation and the average value.
The image formula is from Baidu Baike.
Skewness: The skewness coefficient is a number of features that describe the degree of deviation from the symmetry of the distribution. When the distribution is symmetric, The skewness coefficient is 0. When the skewness coefficient is greater than 0, that is, when the weight tail is on the right, the distribution is on the right. When the skewness coefficient is less than 0, that is, when the weight tail is on the left, the distribution is left biased.

Kurtosis: indicates whether the distribution is more towering or flat than the normal distribution. A positive value indicates a relatively towering distribution, and a negative value indicates a relatively flat peak. To put it simply, the peak degree is the steep degree to describe the distribution form. It can also be understood that, under the same standard deviation, the greater the kurtosis coefficient, the more extreme values are distributed, and the remaining values must be more concentrated around the crowd, the distribution must be steep.

For more information about parameters of the Aggregate function, see
Https://msdn.microsoft.com/en-us/zh-en/library/dd383667 (v = vs.110). aspx

A brief description of parameters:

Seed: The value initialized by The accumulators. Update accumulator function: calculates each value in the array. In PLINQ, the data source is partitioned and then computed in parallel, the result produced in this step is actually the computing result of each saved partition. Combine accumulator function: accumulates the computing results of each partition to obtain an aggregate array. Result selector: calculates the accumulated result to obtain the final result, that is, the return value.

The focus of the example is not a variety of numeric operations, but that Aggregate () can calculate each element of the data source and then summarize the results again, which can be completed in one step, saves the trouble of Writing separately. In addition, it performs data partitioning and parallel tasks. <喎?http: www.bkjia.com kf ware vc " target="_blank" class="keylink"> VcD4NCjxwPs/Cw + bKx7zGy + O1xLT6wuvKvsD9o7o8L3A + DQo8cHJlIGNsYXNzPQ = "brush: java;"> using System;using System.Text;using System.Threading;using System.Threading.Tasks;using System.Diagnostics;using System.Linq;using System.IO;using System.Collections.Generic;namespace Sample6_2_plink_aggregate{ class Program { static void Main(string[] args) { int[] inputInts = {0,3,4,8,15,22,34,57,68,32,30}; var mean = inputInts.AsParallel().Average(); var standarddeviation = inputInts.AsParallel().Aggregate( 0d, // seed // update accumulator function // An accumulator function to be invoked on each element in a partition (subTotal, thisNumber) => subTotal + Math.Pow((thisNumber - mean), 2), // combine accumulator function // An accumulator function to be invoked on the yielded accumulator result from each partition. (total, thisTask) => total + thisTask, // result selector // A function to transform the final accumulator value into the result value. (finalSum) => Math.Sqrt((finalSum / (inputInts.Count()-1))) ); var skewness = inputInts.AsParallel().Aggregate( 0d, // seed // update accumulator function // An accumulator function to be invoked on each element in a partition (subTotal, thisNumber) => subTotal + Math.Pow(((thisNumber - mean) / standarddeviation), 3), // combine accumulator function // An accumulator function to be invoked on the yielded accumulator result from each partition. (total, thisTask) => total + thisTask, // result selector // A function to transform the final accumulator value into the result value. (finalSum) => (finalSum * inputInts.Count()) / ((inputInts.Count()-1)*(inputInts.Count()-2)) ); var kurtosis = inputInts.AsParallel().Aggregate( 0d, // seed // update accumulator function // An accumulator function to be invoked on each element in a partition (subTotal, thisNumber) => subTotal + Math.Pow(((thisNumber - mean) / standarddeviation), 4), // combine accumulator function // An accumulator function to be invoked on the yielded accumulator result from each partition. (total, thisTask) => total + thisTask, // result selector // A function to transform the final accumulator value into the result value. (finalSum) => ((finalSum * inputInts.Count() * (inputInts.Count() + 1)) / ((inputInts.Count() - 1) * (inputInts.Count() - 2) * (inputInts.Count() - 3))) - (3 * Math.Pow((inputInts.Count() - 2), 2)) / ((inputInts.Count() - 2) * (inputInts.Count() - 3)) ); Console.WriteLine("============================================================"); Console.WriteLine("TEST Parallel LINQ Calculate Result"); Console.WriteLine("============================================================"); Console.WriteLine("Mean : {0}", mean); Console.WriteLine("Standard Deviaton : {0}", standarddeviation); Console.WriteLine("Skewness : {0}", skewness); Console.WriteLine("Kurtosis : {0}", kurtosis); Console.ReadLine(); } }}Cancel concurrent PLINQ tasks and tasks

PLINQ can also be used with other concurrent tasks. For example, when calculating standard deviations, skewness, and peaks.
The actual execution order is average => standard deviation => skewness => Kurtosis

However, based on the calculation formula, the skewness and peaks can be processed in parallel. Standard deviation is their common input.
Average value => standard deviation => skewness
=> Kurtosis

They can use the ContinueWith operation. If time-out control or cancellation is required, you can use the WithCancellation () interface.

Sample Code:
The PLINQ operations are encapsulated by functions in the code, and then called in parallel using tasks. DeferredCancelTask is a messy task. If you open the comment, it sends a Cancel signal in 2 seconds to Cancel the task execution and print the task status during exception handling.

using System;using System.Text;using System.Threading;using System.Threading.Tasks;using System.Diagnostics;using System.Linq;using System.IO;using System.Collections.Generic;namespace Sample6_4_parallel_task_with_plinq{    class Program    {        private static ParallelQuery
  
    inputInts =            ParallelEnumerable.Range(1, 100000000);        private static double CalculateMean(System.Threading.CancellationToken ct)        {            return inputInts.AsParallel().WithCancellation(ct).Average();        }        private static double CalculateStandardDeviation(System.Threading.CancellationToken ct, double mean)        {            return inputInts.AsParallel().WithCancellation(ct).Aggregate(                0d, // seed                // update accumulator function                // An accumulator function to be invoked on each element in a partition                (subTotal, thisNumber) => subTotal + Math.Pow((thisNumber - mean), 2),                // combine accumulator function                // An accumulator function to be invoked on the yielded accumulator result from each partition.                (total, thisTask) => total + thisTask,                // result selector                // A function to transform the final accumulator value into the result value.                (finalSum) => Math.Sqrt((finalSum / (inputInts.Count() - 1)))                );        }        private static double CalculateSkewness(System.Threading.CancellationToken ct, double mean, double standarddeviation)        {            return inputInts.AsParallel().WithCancellation(ct).Aggregate(                0d, // seed                // update accumulator function                // An accumulator function to be invoked on each element in a partition                (subTotal, thisNumber) => subTotal + Math.Pow(((thisNumber - mean) / standarddeviation), 3),                // combine accumulator function                // An accumulator function to be invoked on the yielded accumulator result from each partition.                (total, thisTask) => total + thisTask,                // result selector                // A function to transform the final accumulator value into the result value.                (finalSum) => (finalSum * inputInts.Count()) / ((inputInts.Count() - 1) * (inputInts.Count() - 2))                );        }        private static double CalculateKurtosis(System.Threading.CancellationToken ct, double mean, double standarddeviation)        {            return inputInts.AsParallel().WithCancellation(ct).Aggregate(                0d, // seed                // update accumulator function                // An accumulator function to be invoked on each element in a partition                (subTotal, thisNumber) => subTotal + Math.Pow(((thisNumber - mean) / standarddeviation), 4),                // combine accumulator function                // An accumulator function to be invoked on the yielded accumulator result from each partition.                (total, thisTask) => total + thisTask,                // result selector                // A function to transform the final accumulator value into the result value.                (finalSum) => ((finalSum * inputInts.Count() * (inputInts.Count() + 1)) /                    ((inputInts.Count() - 1) * (inputInts.Count() - 2) * (inputInts.Count() - 3))) -                    (3 * Math.Pow((inputInts.Count() - 2), 2)) /                    ((inputInts.Count() - 2) * (inputInts.Count() - 3))                );        }        static void Main(string[] args)        {            Console.WriteLine("============================================================");            Console.WriteLine("TEST Parallel TASK work with PLINQ");            Console.WriteLine("============================================================");            var cts = new System.Threading.CancellationTokenSource();            var ct = cts.Token;            var TaskMean = new Task
   
    (()=> CalculateMean(ct), ct);            var TaskSTDev = TaskMean.ContinueWith
    
     ((t) => { return CalculateStandardDeviation(ct, t.Result); },                                                          TaskContinuationOptions.OnlyOnRanToCompletion);            var TaskSkewness = TaskSTDev.ContinueWith
     
      ((t) => { return CalculateSkewness(ct, TaskMean.Result, t.Result); },                                                          TaskContinuationOptions.OnlyOnRanToCompletion);            var TaskKurtosis = TaskSTDev.ContinueWith
      
       ((t) => { return CalculateKurtosis(ct, TaskMean.Result, t.Result); }, TaskContinuationOptions.OnlyOnRanToCompletion); //var deferredCancelTask = Task.Factory.StartNew(() => { System.Threading.Thread.Sleep(2000); cts.Cancel();}); try { TaskMean.Start(); Task.WaitAll(TaskSkewness, TaskKurtosis); Console.WriteLine("Mean : {0}", TaskMean.Result); Console.WriteLine("Standard Deviaton : {0}", TaskSTDev.Result); Console.WriteLine("Skewness : {0}", TaskSkewness.Result); Console.WriteLine("Kurtosis : {0}", TaskKurtosis.Result); } catch(AggregateException aex) { foreach (var ex in aex.InnerExceptions) { //Console.WriteLine(ex.ToString()); if (ex is TaskCanceledException) { Console.WriteLine("Mean Task: {0}", TaskMean.Status); Console.WriteLine("Standard Deviation Task: {0}", TaskSTDev.Status); Console.WriteLine("Skewness Task: {0}", TaskSkewness.Status); Console.WriteLine("Kurtosis Task: {0}", TaskKurtosis.Status); } } } Console.ReadLine(); } }}