Range and precision of float and double, number type of Oracle

Source: Internet
Author: User
Tags rounds

double and float are floating point types. Doubles (double type) are more accurate than float (single-precision) data and occupy more space. Double precision is twice times that of float, so a double is often used for more precise calculations.

Single-precision floating-point numbers account for 4 bytes in the machine and are described in 32-bit binary.
Double-precision floating-point numbers account for 8 bytes in the machine, and are described in 64-bit binary.

Floating-point numbers are represented in the machine by exponential type, broken down into: digits, mantissa, exponential character, exponent four part.
A number is a 1-bit binary that represents the positive or negative of a number.
The exponent symbol is a 1-bit binary that represents the positive or negative of the exponent.
The mantissa represents a valid number of floating-point numbers, 0.xxxxxxx, but does not save the beginning of 0 and points
A valid number of index indices.

The number of digits the index occupies, and the number of mantissa, is determined by the computer system.
It is possible that the multibyte Mantissa accounts for 24 bits, and the exponent multibyte index is 8 bits--float.
The number multibyte mantissa occupies 48 bits, the exponent multibyte index occupies 16 bits--double.

Know the four part of the placeholder, according to the binary estimate size range, and then converted to decimal, that is, you want to know the range of values.

For programmers, the difference between double and float is a double with a high precision, a valid number of 16 bits, and a float precision of 7 bits.  But double consumes twice times the memory is float, double operation speed is much slower than float, C language math function name double and float different, do not write wrong, can use single precision do not use double precision (to save memory, speed up the operation speed). Range and precision of float and double

1. Scope
The range of float and double is determined by the number of digits of the exponent.
Float has an exponential position of 8 bits, and a double has a 11-bit exponent, distributed as follows:
Float
1bit (sign bit) 8bits (digit digit) 23bits (trailing digit)
Double
1bit (sign bit) 11bits (digit digit) 52bits (trailing digit)
Thus, the index range of float is -127~+128, and the double index range is -1023~+1024, and the digits are divided in the form of complement.
The negative exponent determines the absolute minimum non-zero that the floating point can express, and the positive exponent determines the maximum number of absolute value that the floating point can express, which determines the range of the floating-point value.
The range of float is -2^128 ~ +2^128, that is, the range of -3.40E+38 ~ +3.40e+38;double is -2^1024 ~ +2^1024, or -1.79E+308 ~ +1.79E+308.

2. Accuracy
The precision of float and double is determined by the number of bits in the mantissa. Floating-point numbers are stored in memory by scientific notation, and the integer part is always an implied "1", because it is invariant and therefore cannot affect precision.
float:2^23 = 8388608, a total of seven, which means that there can be up to 7 valid digits, but the absolute guarantee of 6-bit, that is, the precision of float is 6~7 bit valid number;
double:2^52 = 4503599627370496, altogether 16 bits, in the same vein, the precision of the double is 15~16 bit.

Number type in 3.Oracle

In Oracle, the number type can be used to store 0, positive or negative fixed-point or floating-point numbers, which can represent a range of data in
1.0 * 10 (-130)--9.9...9 * 10 (125) {38 9 + 88}
digits behind. When the value of the mathematical expression in Oracle is >=1.0*10 (126), Oracle will make an error. The data declaration for
number is as follows:
represents         action         Description
Number (p, s)         declares a fixed point         p ( Precision) for precision, s (scale) represents the number of digits to the right of the decimal point, the maximum precision is $,
Number (p)         declares an integer         equals number (p, 0)
number        Declare a floating-point number         its accuracy is 38, note that the value of scale is not applied, that is to say, scale can not be easily understood as 0, or other number.

The precision of the fixed-point number (p) and scale (s) follows the following rules:
? Oracle will error when the length of the integer part of a number > P-s
? When the length of a number of decimal parts is > S, Oracle rounds.
? When s (scale) is negative, Oracle rounds the s number to the left of the decimal point.
? When s > P, p indicates the number of digits to the left of the S bit after the decimal point, and if the Oracle error is greater than P, the number of s bits to the right after the decimal point is rounded

4. Verification
Create or Replace function Func_test (p_type number) return number
Is
/*
Function: Data synchronization based on police degree chart
*/
L_CNT number;
Begin
Select P_type to l_cnt from dual;
return l_cnt;
End Func_test;
/
Show err;

5. Conclusion

The total length of number is 40 bits, which may include: decimal point, minus sign bit.

Select To_char (Func_test ( -987.1234567891234567891234567891234567891234)) from dual;
-987.12345678912345678912345678912345679//Including decimal point and minus sign 40 bits total
Select To_char (Func_test (9876.1234567891234567891234567891234567891234)) from dual;
9876.12345678912345678912345678912345679//4-bit integer + decimal + 35-bit decimal = 40-bit
Select To_char (Func_test (987.1234567891234567891234567891234567891234)) from dual;
987.123456789123456789123456789123456789//3-bit integer + decimal + 36-bit decimal = 40-bit
Select To_char (Func_test (1234567891234567891234567891234567891234)) from dual;
1234567891234567891234567891234567891234//40-bit integers
Select To_char (Func_test (12345678912345678912345678912345678912345)) from dual;
Accuracy loss at 1.2345678912345678912345678912345679E+40//41-bit
1.2345678912345678912345678912345679X10^40 is 12345678912345678912345678912345678900000

Range and precision of float and double, number type of Oracle

Related Article

Contact Us

The content source of this page is from Internet, which doesn't represent Alibaba Cloud's opinion; products and services mentioned on that page don't have any relationship with Alibaba Cloud. If the content of the page makes you feel confusing, please write us an email, we will handle the problem within 5 days after receiving your email.

If you find any instances of plagiarism from the community, please send an email to: info-contact@alibabacloud.com and provide relevant evidence. A staff member will contact you within 5 working days.

A Free Trial That Lets You Build Big!

Start building with 50+ products and up to 12 months usage for Elastic Compute Service

  • Sales Support

    1 on 1 presale consultation

  • After-Sales Support

    24/7 Technical Support 6 Free Tickets per Quarter Faster Response

  • Alibaba Cloud offers highly flexible support services tailored to meet your exact needs.