Jump to content

wrong calculation


backrom
 Share

Recommended Posts

Hi forum,

I have encountered a calculation error on my PC.

The script is simple and its running in an Adobe PDF but i get the same result on the w3schools-try-it-page (Chrome Browser)

 

<!DOCTYPE html>
<html>
<body>
<p>y = 1.21, calculate x = y - 1, and display x:</p>
<button onclick="myFunction()">Try it</button>
<p id="demo"></p>
<script>
function myFunction() {
var y = 1.21;
var x = y - 1;
document.getElementById("demo").innerHTML = x;
}
</script>
</body>
</html>
the result :

y = 1.21, calculate x = y - 1, and display x:

Try it

0.20999999999999996

is close to correct, but its wrong!

After changing my initial values to 1.25

<!DOCTYPE html>
<html>
<body>
<p>y = 1.25, calculate x = y - 1, and display x:</p>
<button onclick="myFunction()">Try it</button>
<p id="demo"></p>
<script>
function myFunction() {
var y = 1.25;
var x = y - 1;
document.getElementById("demo").innerHTML = x;
}
</script>
</body>
</html>
the result is correct

y = 1.25, calculate x = y - 1, and display x:

Try it

0.25

I know that I could include a math.round fonction, but i'm dealing with many big sheets that come from an official goverment site.

Our PC are win8 and (hope) fully patched. The script is running in latest Adobe reader DC.

My question is:

Is this my local problem ?

or is it a general problem, and you all have to deal with that?

Thanks in advance

Romain

 

Link to comment
Share on other sites

To put it simply, that's a side-effect of trying to do floating-point math with a binary computer, and Javascript doesn't handle it particularly well. There's a discussion here with possible solutions from basic rounding stuff to libraries that will handle floating-point math correctly.

 

http://stackoverflow.com/questions/1458633/how-to-deal-with-floating-point-number-precision-in-javascript

Link to comment
Share on other sites

 

Is this my local problem ?

or is it a general problem, and you all have to deal with that?

 

 

Are you kidding? Everybody knows that 1 + 1 = 1.999

 

https://en.wikipedia.org/wiki/Floating_point#Accuracy_problems

 

As JSG says, there are some special libraries that can provide arbitrary precision/infinite precision math, but these are only used when the extra processing is justified by a real need for that precision. Try a google search.

Link to comment
Share on other sites

Are you kidding? Everybody knows that 1 + 1 = 1.999

That's actually a fact, and it shows how our understanding of math is close but not quite there.

 

Consider the fact that if you have a number X, and you divide that number by Y to get Z, then if you multiply Z times Y you will get back to X. e.g.:

 

X = 10
Y = 5
X / Y == 10 / 5 = Z = 2
Z * Y == 2 * 5 = 10 == X
So then why does this happen:

 

X = 1
Y = 3
X / Y = 1 / 3 = Z = .3333333..
Z * Y = .3333333.. * 3 = .9999999..
Therefore, 1 is equal to .9999999..
Link to comment
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
 Share

×
×
  • Create New...