Hi!
If you execute 0.06 + 0.01 (you can try it in the web browser “javascript:alert(0.01+0.06)”) the result is 0.06999999999999999.
What do you do to solve this? .toFixed()? any external library?
Thanks in advance!
application.output(0.07+0.01);
application.output((0.07+0.01).toFixed(2));
application.output(Math.round(0.07+0.01));
results in
0.08
0.08
0.0
So I am not seeing what you are seeing.
Also note that Servoy uses Rhino. Your browser might use another JavaScript engine.
We use this:
var $nrX = arguments[0]; // first(nr) value
var $nrY = arguments[1]; // second(nr) value
var $math = arguments[2]; // the math(string): '+' '-' '*' '/'
var $precision = arguments[3]; // the nr of desired decimals
var x = $nrX * Math.pow(10, $precision);
var y = $nrY * Math.pow(10, $precision);
var z;
switch ($math)
{
case '+':
z = (x + y)/Math.pow(10, $precision);
break;
case '-':
z = (x - y)/Math.pow(10, $precision);
break;
case '*':
z = (x * y)/Math.pow(10, $precision);
break;
case '/':
z = (x / y)/Math.pow(10, $precision);
break;
}
return z;
Hope this helps!
ROCLASI:
So I am not seeing what you are seeing.
Try it 0.06 + 0.01 and you will see it (isn´t always, only with especifics combiantions of numbers and operations)
@Marc:
Thanks for your trick!
victor.rojo:
Hi!
If you execute 0.06 + 0.01 (you can try it in the web browser “javascript:alert(0.01+0.06)”) the result is 0.06999999999999999.
What do you do to solve this? .toFixed()? any external library?Thanks in advance!
First, you need to know that there is only one number type in js (JavaScript). There is not separate integer type. There is just “number.” It is represented as a 64 bit floating point, aka ‘Double’ (the names comes from FORTRAN).
The problem with floating point is that it doesn’t work well with the common understanding of arithmetic. As you noticed, 0.01 + 0.06 does not equal 0.07. While working with js you really need to be careful while building calculations with cents values (things like money).
Floating point (used in js) will always give you an approximation. What you want to do while dealing with money it’s multiply the values by 100 - so that you are dealing with just natural numbers (as positive integers), and then scale them back (divide them by 100 - see Marc’s tip above).
If I were you I’d take the easier road. Just multiply and divide by 100!
JCarlos
jcarlos:
If I were you I’d take the easier road. Just multiply and divide by 100!
JCarlos
That’s what we do also!!
The tip of Marc, is also exactly doing that! (Nice tip Marc, thanks for sharing.)
Harjo:
That’s what we do also!!The tip of Marc, is also exactly doing that! (Nice tip Marc, thanks for sharing.)
Agree. My intention was to complement Marc’s tip with a small explanation about the issue.
I consider a language where 0.06 + 0.01 equals 0.06999999999999999 a disaster !
I worked for twenty years with Foxpro and never knew that 0.06 + 0.01 could be anything else than 0.07
Now we have to adjust many hundreds of (Servoy) code lines to persist 2 decimal precision, mostly price fields. Also the debugging time involved when table number values are not right explodes with more extensive calculations. So a bit of a show stopper for us…
Does anybody know what the result of 0.06 + 0.01 is in C# ?
Regards,
lwjwillemsen:
Does anybody know what the result of 0.06 + 0.01 is in C# ?
i guess that is the same in java: depends on what you use as a datatype…
if you use a Floating point notation (IEEE 754) type then it just will do the same
But if you have Classes like BigDecimal and so on then it will work like you describe.
This is the same as it is in many languages and has to do with the binary nature of floating point numbers. If you do this exact math in C# you get a result 0.069999999999999993 but as Johan mentioned, if you used a specific data type you can handle this. We actually have a method that we send all our math through to avoid the issues surrounding floating point imprecision.
mattfrizzell:
This is the same as in any language and has to do with the binary nature of floating point numbers. If you do this exact math in C# you get a result 0.069999999999999993 but as Johan mentioned, if you used a specific data type you can handle this. We actually have a method that we send all our math through to avoid the issues surrounding floating point imprecision.
Not true, in Visual Foxpro 0.06 + 0.01 = 0.07
and… 0.06112 + 0.01989 = 0.08101 ! No precision loss up to 16 digits !
It’s all up to compiler construction…
Sure but as I said, many other languages operate this way and certainly at the core of it all do this. In the case of FoxPro, it simply handles the imprecision for you. We could team up to follow a dream I have had for years which is that I have always wanted to make an analog computer instead of the foolish binary computers we use. Then we wouldn’t have to deal with this problem at all.
Also, I think you got a cached version of my message. I fixed my “any” to “many” about 15 seconds after I hit submit the first time because I realized I had an important typo there.
lwjwillemsen:
It’s all up to compiler construction…
doesnt have anything to do with compiler construction.
It has to do with what kind of data type, how do you store the Value internally.
I think VFP has Numeric and Double data types, and i think Double is a true precision data type so i guess (not sure) that thats a IEEE 754 implementation.
Numeric data type (lets say BigDecimal in java) stores it differently that is more like a few integers for the value and one integer for the position of the decimal point.
I consider a language where 0.06 + 0.01 equals 0.06999999999999999 a disaster !
JavaScript is far from a disaster, and it’s gaining more and more momentum all the time. Server-side JS for example is becoming a very hot topic in the web community with new projects sprouting up everywhere. It won’t be too long now before people start choosing JS over PHP or the similar (if they haven’t already). Expect gains not falls on the TIOBE index.
JS is ubiquitous; best to learn its quirks. And for that there is (among others) Crockford’s excellent book: “JavaScript the Good Parts”
JS/Rhino was and continues to be a great choice for Servoy.
jcompagner:
lwjwillemsen:
It’s all up to compiler construction…doesnt have anything to do with compiler construction.
It has to do with what kind of data type, how do you store the Value internally.I think VFP has Numeric and Double data types, and i think Double is a true precision data type so i guess (not sure) that thats a IEEE 754 implementation.
Numeric data type (lets say BigDecimal in java) stores it differently that is more like a few integers for the value and one integer for the position of the decimal point.
Hi Johan,
In VFP you don’t have to define the data type at declaration of a variable (string or numeric), so I don’t how it is handled under the hood.
Thanks for your info…
Regards,
jbader:
I consider a language where 0.06 + 0.01 equals 0.06999999999999999 a disaster !
JavaScript is far from a disaster, and it’s gaining more and more momentum all the time. Server-side JS for example is becoming a very hot topic in the web community with new projects sprouting up everywhere. It won’t be too long now before people start choosing JS over PHP or the similar (if they haven’t already). Expect gains not falls on the TIOBE index.
JS is ubiquitous; best to learn its quirks. And for that there is (among others) Crockford’s excellent book: “JavaScript the Good Parts”
JavaScript: The Good Parts: The Good Parts: Crockford, Douglas: 9780143429036: Amazon.com: Books
JS/Rhino was and continues to be a great choice for Servoy.
Hi, I think the word disaster is indeed a bit too heavy. It emerged from frustration over something in my eyes trivial I did not have to care about in the last 20 years of my programming…
May I say that in case of numerical computations with JavaScript I have quite a productivity loss as compared to VFP ? (We have many hundreds of numerical computations in our solutions which require two digit precision)
Thanks for your contribution…
Foxpro stores numbers as strings on disk, as 6 byte floats in memory. It uses some sort of binary coded decimal arithmetic for internal calculations up to 16 digits with which it achieves it´s precision after which it uses floats and will exhibit the same imprecision issues as other platforms.
Jeroen de Vries:
Foxpro stores numbers as strings on disk, as 6 byte floats in memory. It uses some sort of binary coded decimal arithmetic for internal calculations up to 16 digits with which it achieves it´s precision after which it uses floats and will exhibit the same imprecision issues as other platforms.
Hoi/hi Jeroen,
I know, but with 16 digits I have more digits/precision than I ever will need…
0.01 + 0.06 is no rocket science I would say and the second decimal is very much in use in price and price computations !
I know computer computation precision has his limits but I did not expect to face that with an addition with 2 (or 1) decimal involved
Groeten/regards,