The solution to the accuracy error of JS digital calculation

This article is mainly on the JavaScript to avoid the accuracy of digital calculation method is introduced, the need for friends can come to the reference, I hope to help you.

What if I ask you 0.1 + 0.2 equals a few? You may give me a supercilious eye, 0.1 + 0.2 = 0.3 Ah, do you still ask? Even the children in kindergarten will answer such a question of pediatrics. But you know, the same problem is in the programming language, perhaps not as simple as it might seem.

Don't believe me? We can do an experiment.

Take a look at JS first.

var num1 = 0.1; var num2 = 0.2; alert (num1+num2 = = = ' 0.3 ');

The execution result is false. Yes, when I first saw this piece of code, I took it for granted that it was true, but the result of the execution made me plunge my glasses, was it my way of opening it wrong? Not also not also. Let's try the following code again to see why the result is false.

var num1 = 0.1var num2 = 0.2; alert (num1+NumB);

Originally, 0.1 + 0.2 = 0.30000000000000004. Isn't it a wonderful flower? In fact, for the arithmetic of floating-point numbers, almost all programming languages will have similar accuracy errors, but in C++/c#/java these languages have encapsulated the method to avoid the problem of precision,

While JavaScript is a weak type of language, from the design of the idea of floating-point number has no strict data type, so the accuracy of the problem of error is particularly prominent. The following is an analysis of why this accuracy error, and how to repair the error.

First of all, we need to stand on the computer's point of view 0.1 + 0.2 This seemingly pediatric problem. We know that it is binary, not decimal, that can be read by a computer, so let's convert 0.1 and 0.2 into binary:

0.1== "0.1.toString (2) = =" 0.0001100110011 (infinite loop ...)

0.2== "0.2.toString (2) = =" 0.001100110011 (infinite loop ...)

The decimal portion of a double-precision floating-point number supports up to 52 bits, so a binary number that is truncated as a result of the limit of the number of floating-point numbers 0.0100110011001100110011001100110011001100110011001100 is added , and then we convert it to decimal, which is 0.30000000000000004.

So, how do we solve this problem? I want the result is 0.1 + 0.2 = = 0.3 Ah!!!

One of the simplest solutions is to give a definite accuracy requirement, and the computer will automatically round up the return value,

Like what:

var num1 = 0.1var num2 = 0.2+ num2). ToFixed (2)) = = = 0.30);

But rounding is obviously not the way to be.

If there is a way to help us solve the problem of the accuracy of these floating point numbers, then how good! So, let's write a method ourselves.

Let's try this method:

function (f, digit) {     var m = Math.pow (ten, digit);      return parseint (f * m, 10)/

var num1 = 0.1;
var num2 = 0.2;

Alert (math.formatfloat (NUM1 + num2, 1) = = = 0.3);

What does this method mean?

In order to avoid the difference in precision, we need to calculate the number multiplied by 10 N power, converted to the computer can be accurately recognized by the integer, and then divided by the power of N of 10, most programming languages are such processing precision differences, we borrowed to deal with the accuracy of the floating point in JS error.

So if the next time someone asks you 0.1 + 0.2 equals a few, you should be careful to answer!!

The solution to the accuracy error of JS digital calculation