C # basic use of parallel programming in PLINQ

C # basic use of parallel programming in PLINQ
PLINQ Summary

You can easily Query and process data from different data sources. PLINQ Parallel LINQ not only provides the functions of LINQ, but also adds interfaces for Parallel operations to facilitate use and improve efficiency.

 

A simple example

A simple example is enough to illustrate the use of PLINQ.
Simple Description:
Part 1: first query the words that contain the letter "a" in a document. First, use LINQ for 10000 queries, and then use PLINQ for the same test to see how much efficiency is improved. AsParallel () is the main interface used.

Part 2: Calculate the sum of all numbers of the Short type, and test the use of LINQ and PLINQ twice.

Part 3: test the query and sorting function of PLINQ. The interfaces used are AsOrdered () and orderby.
AsOrdered: the order of data in the data source remains unchanged.
Orderby: sort by user-specified order, in ascending/descending order

Example program:

using System;using System.Text;using System.Threading;using System.Threading.Tasks;using System.Diagnostics;using System.Linq;using System.IO;namespace Sample6_1_plink_basic{    class Program    {        static int SumDefault(int[] array)        {            return array.Sum();        }        static int SumAsParallel(int[] array)        {            return array.AsParallel().Sum();        }        static string[] words = { Day, Car, Land, Road, Sea, Mountain, River};        static void Main(string[] args)        {            var customers = System.IO.File.ReadAllLines(@D:estdirCSParallel_ProgramSample6-1 plink basicarget.txt);            int nCounter = 0;            Console.WriteLine(============================================================);            Console.WriteLine(TEST NORMAL LINQ);            Console.WriteLine(============================================================);            var swatchpn = Stopwatch.StartNew();            for (int i = 0; i < 10000; i++)            {                var normalkeyLetters = from line in customers                                       let keys = line.Split(' ')                                       from key in keys                                       where (key.Contains('a'))                                       select key;                var normalkeyList = normalkeyLetters.ToList();                nCounter = normalkeyList.Count();            }            swatchpn.Stop();            Console.WriteLine(Word with letter a = {0}, nCounter);            Console.WriteLine(LINQ Use Time: {0}, swatchpn.Elapsed);            Console.WriteLine();            Console.WriteLine(============================================================);            Console.WriteLine(TEST PARALLEL LINQ);            Console.WriteLine(============================================================);            nCounter = 0;            var swatchp = Stopwatch.StartNew();            for (int i = 0; i < 10000; i++)            {                var keyLetters = from line in customers.AsParallel()                                 let keys = line.Split(' ')                                 from key in keys.AsParallel()                                 where (key.Contains('a'))                                 select key;                var keyList = keyLetters.ToList();                nCounter = keyList.Count();            }            swatchp.Stop();            Console.WriteLine(Word with letter a = {0}, nCounter);            Console.WriteLine(PLINQ Use Time: {0}, swatchp.Elapsed);            // Generate array.            int[] array = Enumerable.Range(0, short.MaxValue).ToArray();            const int m = 10000;            var s1 = Stopwatch.StartNew();            for (int i = 0; i < m; i++)            {                SumDefault(array);            }            s1.Stop();            var s2 = Stopwatch.StartNew();            for (int i = 0; i < m; i++)            {                SumAsParallel(array);            }            s2.Stop();            Console.WriteLine();            Console.WriteLine(============================================================);            Console.WriteLine(CALCULATE SUMMARY TEST);            Console.WriteLine(============================================================);            Console.WriteLine(Default Summary:  + ((double)(s1.Elapsed.TotalMilliseconds * 1000000) / m).ToString(0.00 ns));            Console.WriteLine(Parallel Summary:  + ((double)(s2.Elapsed.TotalMilliseconds * 1000000) /m).ToString(0.00 ns));            Console.WriteLine();            Console.WriteLine(============================================================);            Console.WriteLine(Parallel ASOrder Test);            Console.WriteLine(============================================================);            var orderwords = from word in words.AsParallel().AsOrdered()                             where (word.Contains('a'))                             select word;            var orderletterList = orderwords.ToList();            for (int i = 0; i < orderletterList.Count; i++)            {                Console.WriteLine(orderletterList[i]);            }            Console.WriteLine();            Console.WriteLine(============================================================);            Console.WriteLine(Parallel OrderBy Test);            Console.WriteLine(============================================================);            var orderbywords = from word in words.AsParallel()                             where (word.Contains('a'))                             orderby word ascending                             select word;            var orderbyletterList = orderbywords.ToList();            for (int i = 0; i < orderbyletterList.Count; i++)            {                Console.WriteLine(orderbyletterList[i]);            }            Console.ReadKey();        }    }}

Test results:

In fact, the test has been run many times. The test result of the word test is between 32 and 34 seconds, and the PLINQ is between 16 and 19 seconds, with an improvement of about 50%.

Data Partition in PLINQ

PLINQ processes the same data source using multiple tasks and then summarizes them to improve efficiency. However, it is very important for a task to process data in parallel and how to reasonably divide the data and assign it to the task, which seriously affects the efficiency. However, developers cannot really control which Partitioning Method PLINQ uses. It helps to understand its internal mechanism when optimizing procedural performance.

There are four data partitioning methods in PLINQ:

Range Division: data sources that can be indexed, such as arrays and lists. PLINQ searches for the IList interface of the data source. If yes, it splits the data into partitions with the same number of available logical kernels. PLINQ can precisely know the data size and can directly access any element in it. Data Block Division: Suitable for any data source that cannot be indexed. Data blocks are obtained for different tasks, and the block size may be different. Staggered partitioning: This method optimizes the processing of data items at the top of the data source. This method is used when the query includes SkipWhile and TakeWhile. In this case, each task corresponds to a group of data (stripe for short ). The task can identify the corresponding data stripe through simple calculation, so the task does not need to be synchronized. Hash partition: optimizes data comparison. It establishes a channel between data and tasks. Data items with the same hash code will be sent to the same task, so that possible matching will be performed in a data partition, this simplifies the comparison process and reduces data sharing between different tasks. This method may cause a large overhead when sending data, but the data comparison timeliness will increase.