Analysis of internal execution principle of LINQ

LINQ (Language Integrated Query) is a leading figure in Visual Studio 2008. With the help of LINQ technology, we can use a syntax similar to SQL to query any form of data. Currently, the data sources supported by LINQ include SQL Server, XML, and data sets in the memory. Developers can also use its extension framework to add more data sources, such as MySQL, Amazon, or even Google Desktop.

Generally, an important feature of such query statements is that they can be executed in parallel. Although parallel processing may cause some problems in some cases, this situation is rare. This naturally leads to the parallel processing of the LINQ class library Plinq.

Plinq, formerly called Parallel LINQ, supports XML and In-memory data sets. The query statement executed on the remote server (such as LINQ to SQL) obviously cannot implement this function.
It is extremely simple to convert a LINQ statement to a Plinq statement-you only need to add. asparallel () at the end of the data source specified by the from clause in the query statement. Then the where, orderby, and select clauses will be automatically changed to calling the parallel version of LINQ.

According to msdn magazine, Plinq can be executed in three ways. The first is MPs queue processing: one thread is used to read the data source, while other threads are used to process query statements, synchronization between the two -- although this single consumption thread may not be so easy to synchronize with multiple production threads. However, if load balancing can be carefully configured, the memory usage will be greatly reduced.

The second mode is called "stop and go", which is used when the result set needs to be returned once (for example, when tolist, toarray, or result sorting is called. In this mode, each processing process is completed in sequence, and the results are returned to the consumption thread in a unified manner. This mode is superior to the first mode in performance because it saves the overhead used to maintain thread synchronization.

The last method is "Inverted enumeration ". This method does not need to collect all the output and process it in a single thread. Instead, it passes the final called function to each thread through the forall extension. This is the fastest processing mode so far, but it is thread-safe to pass the function to forall, and it is better not to include any mutex statements such as lock.

If any thread in Plinq throws an exception, all other threads will be terminated. If multiple exceptions are thrown, these exceptions are combined into a multiplefailuresexception type exception, but the call stack of each exception is retained.