C # parallel programming-PLINQ: declarative data parallelism

C # parallel programming-PLINQ: declarative data parallelism
Background through LINQ, you can easily query and process different data sources, and use Parallel LINQ (PLINQ) to fully obtain the advantages of parallelism. PLINQ not only implements the complete LINQ operator, but also adds some operators for executing parallel operations. Compared with the corresponding LINQ, PLINQ can provide significant acceleration, however, the specific acceleration effect depends on the specific scenario, but it will accelerate in parallel. If a query involves a large number of computing and memory-intensive operations, and the order is not important, the acceleration will be very obvious. However, if the order is important, the acceleration will be affected. AsParallel () enables parallel query by pasting code to view the effect. For details, see note: copy the code class MRESDemo {/* code: Sakya Nakhon */static void Main () {ConcurrentQueue <Product> products = new ConcurrentQueue <Product> ();/* add multiple pieces of data to the set to modify the data volume and view the performance of Linq and Plinq */Parallel. for (0, 600000, (num) => {products. enqueue (new Product () {Category = "Category" + num, Name = "Name" + num, SellPrice = num });}); /* use LINQ to query Qualified Data */Stopwatch sw = new Stopwatch (); sw. re Start (); var productListLinq = from product in products where (product. name. contains ("1") & product. name. contains ("2") & product. category. contains ("1") & product. category. contains ("2") select product; Console. writeLine ("the number obtained by using the Linq query is: {0}", productListLinq. count (); sw. stop (); Console. writeLine ("using Linq time consumption: {0}", sw. elapsedMilliseconds);/* use PLINQ to query Qualified Data */sw. restart (); var productListPLinq = From product in products. asParallel ()/* AsParallel tries to take advantage of all available logical kernels during the runtime, so that the running speed is faster than the serial version, but pay attention to the performance loss caused by overhead */where (product. name. contains ("1") & product. name. contains ("2") & product. category. contains ("1") & product. category. contains ("2") select product; Console. writeLine ("the number obtained by PLinq query is: {0}", productListPLinq. count (); sw. stop (); Console. writeLine ("PLinq elapsed time: {0}", sw. elapsedMilliseconds); Console. ReadLine () ;}} class Product {public string Name {get; set;} public string Category {get; set;} public int SellPrice {get; set ;}} currently, the amount of simulated data in the copied code is relatively small. The more data, the more obvious the result of the parallel query is AsOrdered () and orderbyAsOrdered: the query results are retained and sorted by the source sequence. In the parallel query, multiple data entries are queried in multiple regions. After the query, the data results of multiple regions are merged into one result set and returned in the order of the source sequence. Orderby: sort the returned result set in the specified order and paste the following code for your convenience: class MRESDemo {/* code: Sakya Nakhon */static void Main () {ConcurrentQueue <string> products = new ConcurrentQueue <string> (); products. enqueue ("E"); products. enqueue ("F"); products. enqueue ("B"); products. enqueue ("G"); products. enqueue ("A"); products. enqueue ("C"); products. enqueue ("SS"); products. enqueue ("D");/* The data output result is not processed in parallel */var productListLinq = from produc T in products where (product. length = 1) select product; string appendStr = string. empty; foreach (string str in productListLinq) {appendStr + = str + "";} Console. writeLine ("parallel output not used: {0}", appendStr ); /* if no sorting policy is used, the data output result of the partition is directly merged and not processed. */var productListPLinq = from product in products. asParallel () where (product. length = 1) select product; appendStr = string. empty; foreach (string st R in productListPLinq) {appendStr + = str + "";} Console. writeLine ("No AsOrdered output: {0}", appendStr ); /* The data output result of the AsOrdered sorting policy is to directly combine the partition data results and sort them by the original data Order */var productListPLinq1 = from product in products. asParallel (). asOrdered () where (product. length = 1) select product; appendStr = string. empty; foreach (string str in productListPLinq1) {appendStr + = str + "";} Console. writeLine ("using AsOrd Ered output: {0} ", appendStr ); /* The data output result of the orderby sorting policy is to directly combine the partition data results and sort them according to orderby requirements */var productListPLinq2 = from product in products. asParallel () where (product. length = 1) orderby product select product; appendStr = string. empty; foreach (string str in productListPLinq2) {appendStr + = str + "";} Console. writeLine ("using orderby output: {0}", appendStr); Console. readLine () ;}} in PLINQ queries, AsOrdered () and or The derby clause reduces the running speed. Therefore, if the order is not necessary, it is very important to compare the acceleration effect with the performance of serial execution before the results in a specific order are requested. In PLINQ queries, AsOrdered () and orderby clauses both reduce the running speed. Therefore, if the order is not mandatory, it is very important to compare the acceleration effect with the performance of serial execution. The Execution Mode WithExecutionMode is specified to parallelize the serialized code, which brings about some additional overhead. This is also true for the parallel execution of Plinq queries. By default, when PLINQ queries are executed ,. NET mechanism will try to avoid high-end parallel algorithms, these algorithms may reduce the performance of execution to the performance of hell serial execution.. NET will make a decision based on the Query Form, and does not enable the dataset size and the time of the delegated execution, but it can also be forced to execute in parallel without considering the execution of the engine analysis results, you can call the WithExecutionMode method to set the mode. Paste the following code to make it easy for you to understand and copy the code class MRESDemo {/* code: Sakya Nakhon */static void Main () {ConcurrentQueue <Product> products = new ConcurrentQueue <Product> ();/* add multiple data entries to the Set */Parallel. for (0, 6000000, (num) => {products. enqueue (new Product () {Category = "Category" + num, Name = "Name" + num, SellPrice = num });}); /* use parallel query to query the data that meets the conditions */Stopwatch sw = new Stopwatch (); sw. restart (); var productListLinq = from product In products. asParallel (). withExecutionMode (ParallelExecutionMode. forceParallelism) where (product. name. contains ("1") & product. name. contains ("2") & product. category. contains ("1") & product. category. contains ("2") select product; Console. writeLine ("the number obtained by parallelizing the entire query is: {0}", productListLinq. count (); sw. stop (); Console. writeLine ("Time consumed by parallelizing the entire query: {0}", sw. elapsedMilliseconds);/* use the default settings from.. NET */Sw. restart (); var productListPLinq = from product in products. asParallel (). withExecutionMode (ParallelExecutionMode. default) where (product. name. contains ("1") & product. name. contains ("2") & product. category. contains ("1") & product. category. contains ("2") select product; Console. writeLine ("by default. NET to query the number: {0} ", productListPLinq. count (); sw. stop (); Console. writeLine ("by default. NET Decision time: {0} ", sw. elapsedMilliseconds); Console. readLine () ;}} class Product {public string Name {get; set;} public string Category {get; set;} public int SellPrice {get; set ;}} using PLINQ to perform a reduction operation PLINQ can simplify the process of applying a function to all members of a sequence or group. This process is called a reduction operation. For example, in PLINQ queries, a query similar to Average, Max, aggregate functions such as Min and Sum can take full advantage of the benefits of parallelism. Parallel Execution of the protocol and serial execution of the Protocol may have different execution results, because the operation cannot meet the exchange and transfer conditions at the same time, in each execution, the distribution of elements in a sequence or group in different parallel tasks may be different, so different final results may be generated in this case. Therefore, you must define the original data source through the serial version to help PLINQ obtain the optimal execution result. Paste the following code: copy the code class MRESDemo {/* code: Sakya Nakhon */static void Main () {ConcurrentQueue <int> products = new ConcurrentQueue <int> (); /* add multiple data entries to the Set */Parallel. for (0, 6000000, (num) => {products. enqueue (num) ;});/* returns IEumerable <int> */var productListLinq = (from product in products select product) using LINQ ). average (); Console. writeLine ("Average calculated using Average: {0}", productListLinq);/* returns ParallelQuery using PLINQ <int> */var ProductListPLinq = (from product in products. asParallel () select product ). average (); Console. writeLine ("Average calculated using Average: {0}", productListPLinq); Console. readLine () ;}} copies the code as shown in the preceding Code. In the LINQ version, this method returns an IEumerable <int>, that is, calling Eumerable. the Range method generates the results of the specified Range integer sequence. In PLINQ version, this method returns a ParallelQuery <int>, that is, the System in the parallel version is called. linq. parallelEumerable's ParallelEumerable. range method. The result sequence obtained through this method is also a parallel sequence and can be run in parallel in PLINQ. If you want to query a specific data source using LINQ, you can define it as private IEquatable <int> products. If you want to query a specific data source using PLINQ, it can be defined as private ParallelQuery <int> products concurrent PLINQ task. Copy the code class MRESDemo {/* code: Sakya Nakhon */static void Main () {ConcurrentQueue <Product> products = new ConcurrentQueue <Product> ();/* add multiple data entries to the Set */Parallel. for (0, 600000, (num) => {products. enqueue (new Product () {Category = "Category" + num, Name = "Name" + num, SellPrice = num}) ;}); CancellationTokenSource cts = new CancellationTokenSource (); /* Create a tk1 Task to query Qualified Data */Task <ParallelQuery <Product> tk1 = new Task <ParallelQuery <Product> (ct) => {Console. writeLine ("starting to execute the tk1 task", products. count); Console. writeLine ("number of data result sets in tk1 task: {0}", products. count); var result = products. asParallel (). where (p => p. name. contains ("1") & p. name. contains ("2"); return result;}, cts. token);/* Create a tk2 task, query the Qualified Data Based on the results of tk1 in the execution of the tk1 Task */Task <ParallelQuery <Product> tk2 = tk1.ContinueWith <ParallelQuery <Product> (tk) ==>{ Console. writeLine ("starting to execute tk2 task", products. count); Console. writeLine ("number of data result sets in tk2 task: {0}", tk. result. count (); var result = tk. result. where (p => p. category. contains ("1") & p. category. contains ("2"); return result;}, TaskContinuationOptions. onlyOnRanToCompletion);/* Create a tk3 task, query the Qualified Data Based on the results of tk1 in the execution of the tk1 Task */Task <ParallelQuery <Product> tk3 = tk1.ContinueWith <ParallelQuery <Product> (tk) ==>{ Console. writeLine ("starting to execute the tk3 task", products. count); Console. writeLine ("number of data result sets in tk3 task: {0}", tk. result. count (); var result = tk. result. where (p => p. sellPrice> 1111 & p. sellPrice <222222); return result;}, TaskContinuationOptions. onlyOnRanToCompletion); tk1.Start (); Task. waitAll (tk1, tk2, tk3); Console. writeLine ("tk2 task result output, total number of records after filtering: {0}", tk2.Result. count (); Console. writeLine ("tk3 task result output, total number of records after filtering: {0}", tk3.Result. count (); tk1.Dispose (); tk2.Dispose (); tk3.Dispose (); cts. dispose (); Console. readLine () ;}} class Product {public string Name {get; set;} public string Category {get; set;} public int SellPrice {get; set ;}}