C # Parallel programming

A Task-based programming

1. Shared memory multi-core OS and distributed memory OS

Shared Memory multicore os-A microprocessor consists of multiple cores, and each core shares a piece of private memory;

Distributed memory os--consists of multiple microprocessors, each microprocessor can have its own private memory, the microprocessor can be located on different computers, each computer can have a different communication channel

Message Passing Interface (MPI): The most popular communication protocol used by parallel applications running on distributed memory computer systems;

1. Parallel programming and multi-core programming

Parallel programs are running multiple instructions at the same time, and the code written can take full advantage of the parallel execution capability provided by the underlying hardware, and multi-core programming can take advantage of multiple execution cores to run multiple instructions in parallel;

1. Hardware Threads and Software threads

Physical core (physical core)-a truly independent processing unit with multiple physical cores enabling multiple instructions to run concurrently.

Each physical core can provide multiple hardware threads (also known as logical cores or logical processors);

Symmetric multithreading (simultaneous MULTITHREADING,SMT): Uses Hyper-Threading Technology (HT) to enable microprocessors to provide multiple schema states on each physical core, resulting in a number of physical cores x schema states of several hardware threads;

Software threads: Each software thread shares a private, unique memory space with its parent process, but each software thread has its own stack, register, and private local storage area; The hardware thread can be likened to a swimlane, and the software thread to the swimmer;

Load balancing: The tasks of software threads are distributed across multiple hardware threads, and workloads can be distributed evenly between hardware threads through load balancing. The implementation of load balancing depends on the degree of parallelism of the application, the workload, the number of software threads, the available hardware threads, and the load balancing strategy;

1. Amdahl Law

Predicting maximum theoretical performance gains for multiprocessor systems (SPEEDUP)

Formula: Maximum acceleration ratio =1/((1-p) + (P/N))

P refers to the proportion of code that can perform parallel runs

n refers to the number of compute units available (number of processors or physical cores)

1. Gustafson Law

Measure the amount of work that can be performed within a fixed time by the size of the problem;

Total workload (number of units) =s+ (N*P);

s indicates the number of units of work to be completed in one order;

P indicates the number of working units that each part can execute in full parallel;

n indicates the number of available execution units (number of processors or physical cores)

1. Heavyweight concurrency model and lightweight concurrency model

Heavyweight concurrency Model (multithreaded programming model): Write complex multithreaded code, decompose the algorithm into multiple threads, coordinate individual code units, share information among code units, and collect results of operations, and multithreading models are too complex to handle multi-core revolutions; Because of the lack of support for multi-threaded scopes in the framework hierarchy, Multithreading requires a lot of processing, which makes the code complex and difficult to understand;

Lightweight concurrency Model: reduces the total overhead required to create and execute code on different logical cores, not just on job scheduling between different logical cores, but also adds support for multithreaded access at the framework level;

1. Interleaved concurrency, concurrency, and parallelism

Interleaved Concurrency: Instructions for executing one thread at a time, two-thread instruction interleaved execution

Concurrent: Two-thread instruction executed concurrently

Parallelization Requirements: The division of the work to be done, the concurrent operation of the division of processing, and the ability to integrate the results of the operation, parallel to a problem will produce concurrency;

1. Principles of multi-core parallel programming

Think in a parallel way

Using abstract programming (TPL Task Parallel Library)

Programming according to tasks (things) rather than threads (CPU cores)

When designing, consider shutting down the concurrency situation

Avoid using locks

Consul tools and libraries designed to help with concurrency

Using the extensible memory allocator

Scale up when designing to consider growing workloads

Coreinfo--View processor information program

Two command-style data parallelism

TPL supports parallel pipelining of data parallel tasks (combination of task parallelism and data parallelism)

Parallel.Invoke provides the potential for parallel execution of a given independent task;

An array of arguments needed to pass in an action to be executed in parallel

Methods do not have a specific order of execution and are returned only after all methods have been executed;

Parallel.For provides a potential parallel execution of load balancing for a fixed number of independent for loop iterations;

Loop parallelization,不支持浮点数和步进。无法保证迭代执行的顺序；

The Parallel.ForEach provides a potential parallel execution of load balancing for a fixed number of independent for loop iterations, and supports a custom partitioner that gives you complete control over data concurrency, and provides 20 overloaded methods

Using a range integer as a set of data, a partition is used to divide the data into a set of data blocks. Each chunk of data is processed in a circular way, and these loops are parallel.

Optimize partitioning based on number of cores

Environment.processorcount get the number of logical cores

Exiting from a parallel loop

Using ParallelLoopState in the parameters, you can use Loopstate.break (), or loopstate.stop () to exit. The difference is that, assuming that the iteration 100 is being processed when calling break, the iterations of less than 100 are guaranteed to be executed, and stop does not guarantee this.

Parallelloopresult as the return value, you can know whether it is a normal completion or a break

Catching exceptions for parallel loops

Try
{
Loopresult =parallel.foreach (Inputdata,
(int number, parallelloopstateloopstate) =
{ throw new Exception (); });
}
Catch (AggregateException ex)
{
foreach (Exception Innerex in Ex.) InnerExceptions)
{
Debug.WriteLine (Innerex.tostring ());
}
}

Paralleloption

Used to modify the maximum degree of parallelism.

var paralleloptions = new paralleloptions ();
Paralleloptions.maxdegreeofparallelism = Maxdegree;

Three Command-side task parallelism

System.Threading.Tasks.Task

A task represents an asynchronous operation that is created and executed independently.

The state and life cycle of a task

C # Parallel programming