Floating Point Data Precision of Win32 Program

Floating point display problems

 

Problem description:FloatVariable, and assign a very variable,ProgramThe running result is inconsistent with the original value. The following test procedure:

Int main (INT argc, char * argv []) { Float fvalue = 123456789012.0; Float * P = & fvalue; Printf ("% F \ n", fvalue ); Trace ("\ r \ n val = % F", fvalue );   Return 0; }

After the program runs, the output result is:123456790528.0

 

After analyzing the representation of floating point in memory, we find thatFloat.

32The binary representation of a bit floating point is as follows:R = S * m * 2e-127

EAdd a fixed value to the actual index127.

Symbol bit-1bit	Level CodeE-8bit	Tail numberM-23bit
1	1111 1111	111 1111 1111 1111 1111 1111

 

value 123456789012 to binary: 1110010111110100110010001101000010100
total 37 bits, the decimal point is left after 36 bits

110010111110100110010001101000010100., Because32The ending number of a floating point number is23The decimal point is retained.23Bit, that is:1.11001011111010011001000, Followed12The number of digits is rounded up by size, for example1101000010100After conversion6676Greater213-1So proceed1, The final storage is represented:1.11001011111010011001001,Level code is36plus 127 = 163.

 

That is, after conversion, the precision is lost, and the converted data is:1.110010111110100110010010000000000000After the decimal point is removed, that is, the data size stored by the computer. The value is123456790528.

 

Conclusion: If you useFloatWhen the data is greater16777215(24Binary1,111111111111111111111111), There will be precision issues, and the difference between the range of the original data depends on the total number of digits in the binary representation of the data and24Difference, for example37, The range is0-237-24