Microsoft Visual C + + floating-point optimization

Summary: Learn how to optimize floating-point code using the floating-point semantics management method of Microsoft Visual C + + 2005 (formerly known as Visual C + + "Whidbey"). Create a quick program, while ensuring that security optimizations are performed only on floating-point code.

Floating-point code optimization in C + +

The C + + Optimizer compiler can not only convert source code to machine code, but also be able to properly arrange machine instructions to improve performance and/or reduce size. Unfortunately, many commonly used optimizations may not be secure when applied to floating-point computations. In the following sum algorithm [1], you can see an appropriate example of this:

float KahanSum( const float A[], int n )　
{
　 float sum=0, C=0, Y, T;
　 for (int i=0; i<n; i++)
　 {
　　　Y = A[i] - C;
　　　T = sum + Y;
　　　C = T - sum - Y;
　　　sum = T;
　 }
　 return sum;
}

This function adds n floating-point values in the array vector A. In the loop body, the algorithm computes a "fix" value and then applies it to the next step of the sum. Compared with the simple summation, this method greatly reduces the cumulative rounding error while maintaining the O (n) time complexity.

An imperfect C + + compiler may assume that the floating-point algorithm follows the same algebraic rules as the real number algorithm. Such a compiler might then erroneously conclude that

C = t-sum-y ==> (sum+y)-sum-y ==> 0;

In other words, C gets a value that is always constant 0. If the constant value is subsequently propagated to a subsequent expression, the loop body is reduced to a simple summation. To be more accurate, that is

Y = A[i] - C ==> Y = A[i]
T = sum + Y ==> T = sum + A[i]
sum = T ==> sum = sum + A[i]

Therefore, for imperfect compilers, the logical conversion of the KahanSum function will be:

float KahanSum( const float A[], int n )
{
　 float sum=0; // C, Y & T are now unused
　 for (int i=0; i<n; i++)
　　　sum = sum + A[i];
　 return sum;
}

Although the converted algorithm is faster, it does not accurately express the programmer's intentions. The carefully designed error correction has been completely eliminated, leaving only a simple direct summation algorithm with all its associated errors.

Of course, the perfect C + + compiler knows that algebraic rules for real algorithms are usually not applicable to floating-point algorithms. However, even the perfect C + + compiler may incorrectly explain the programmer's intentions.

Consider a common optimization measure that attempts to store as many values as possible in registers (called "enlistment" values). In the kahansum example, this optimization might attempt to register variables C, Y, and T, because these variables are only used in the loop body. If the register precision is 52 bits (double precision) instead of 23 bits (single precision), this optimization can effectively raise the type of C, Y, and T to double. If the sum variable is not registered in the same way, it will still be encoded as a single precision. This converts the semantics of kahansum to the following semantics

float KahanSum( const float A[], int n )
{
　 float sum=0;
　 double C=0, Y, T; // now held in-register
　 for (int i=0; i<n; i++)
　 {
　　　Y = A[i] - C;
　　　T = sum + Y;
　　　C = T - sum - Y;
　　　sum = (float) T;
　 }
　 return sum;
}

Although Y, T, and C are now evaluated with higher precision, the new encoding may produce lower-precision results, depending on the value in a[]. Thus, even seemingly harmless optimizations can have negative consequences.

These kinds of optimization problems are not limited to "tricky" floating-point code. Even a simple floating-point algorithm can fail after an error has been optimized. Consider a simple direct sum algorithm:

float Sum( const float A[], int n )
{
　 float sum=0;
　 for (int i=0; i<n; i++)
　　　sum = sum + A[i];
　 return sum;
}