Parallel prefix summation algorithm

For more information, see principles of parallel programming by Calvin Lin and Lawrence Snyder ).

Operations closely related to summation are prefix summation, which are also called scan in many parallel programming languages ). Like the sum operation, there is still a sequence of N values,

However, we want to calculate the following sequence,

Where, each Yi is the sum of the first I elements of the input, that is,

The prefix obtained in parallel is not as obvious as the sum obtained (the sum is accumulated and the tree layer sum, and the time responsibility is logn), because it needs to solve all the median values in sequence. At first glance, it seems that prefix summation has no advantage, but it is impossible to find a better solution. In fact, prefix summation can be done in parallel.

By observing the pair summation method, we can find that the prefix value can be calculated as long as the method is slightly modified. The solution is that each leaf processor that stores Xi can calculate the Yi value, as long as it knows the sum of all elements on its left, that is, its prefix. In the process of pairing and summation, we know the sum of all child trees, and if we can retain this information, we can determine these prefixes without directly summing them. To do this, we start from the root, and its prefix (that is, all elements and) before the sequence element is 0. This is also the prefix of its left subtree, and the sum of its left subtree is the prefix of its right subtree. To sum up the application of this idea, we can obtain the following rules:

• First, from the bottom up, parallel layers are used to calculate the sum.
• Then, the imaginary root receives a 0 value from its parent node (which does not exist.
• All non-leaf nodes receive a value from their parent node and forward the value to their left child node, and send the parent node values and their left child node values (obtained when these values are summed up in pairs) to the right child node. These values are also the prefixes of their child nodes.
• The leaf node adds the prefix value from above to the input value it saves.

Figure: prefix and calculation. The yellow nodes scan up the tree and use the pair sum algorithm to calculate the values. The light green nodes scan down the tree with the prefix and the following simple rules: sends the value from the parent node to the left child node, adds the sum (uploaded) from the left child node to the value from the parent node, and sends the result to the right child node.The value that moves down the tree is the prefix of the subnode (see the figure, where the value that moves down is indicated by a light green box ). This type of computing is called Parallel prefix computing. It performs an upward and downward scan in the tree, but all operations on each layer in the scan can be completed at the same time. Therefore, you only need to add up to and down once at most twice on each node, and add the logic operation of the route. As can be seen from the above, the sum of parallel prefixes has the time complexity of logarithm. Many similar sequential operations are inferior to parallel prefix methods in this respect. The fundamental difference between sequential and parallel algorithms is that the order of computing needs to be changed when constructing parallel algorithms.