When using field format to display numbers, it doesn’t seem to round properly all the time.
For example, a field with format “0.0” will round as follows:
entered : displayed : should be
0.0490 : 0.0 : 0.0
0.0491 : 0.1 : 0.0
0.1489 : 0.1 : 0.1
0.1490 : 0.2 : 0.1
0.2490 : 0.2 : 0.2
0.2491 : 0.3 : 0.2
As you can see, those red lines all rounded up when they should have rounded down. This happens in both versions 2.2.7 and 3.1.5. Is this a known bug?
Float with no precision specified?
The default float precision was used (53 bit mantissa, 8 bytes total).
Floating point numbers don’t round nicely and certainly not always to the same value. Not sure what database you are using, but I leaned the hard way to use the DECIMAL datatype (MySQL) in place of FLOAT for my solutions, as floating point numbers are extremely difficult to work with when precision and rounding is important.
These floating point numbers should have at least 15 significant digits of accuracy. Rounding to 1 digit beyond the decimal point should have no problems at all.
As another example, if we set the format to just “#” (round to the nearest integer):
0.490 rounds down to 0
0.491 rounds up to 1
1.489 rounds down to 1
1.490 rounds up to 2
This is clearly wrong: anything below 0.5 should round down to 0.
15 significant digits of accuracy
Actually no. Those 15 digits are the problem. They don’t call it “floating” for nothing.
Well, here are the values as stored in SQL:
0.490 is stored as 0.48999999999999999
0.491 is stored as 0.49099999999999999
1.489 is stored as 1.4890000000000001
1.490 is stored as 1.49
Rounding those 4 values to the nearest integer should still give 0, 0, 1, 1 (compared to 0, 1, 1, 2 as seen in Servoy).
You are right, this one is a Servoy issue, not a floating point issue.
Please add it to the support system.
Thank you.
Will do, thanks.
This issue will be fixed in Servoy 3.5.3 (expected within one week), and merged back to a future 3.1.x release
Thanks for fixing it. However, it looks like there is now an issue with borderline numbers rounding down when they should round up. For example:
0.5 rounds to 0 (when display format is “0”)
0.25 rounds to 0.2 (when display format is “0.0”)
Those 2 examples should definitely round up since the binary representation is exactly the same as the decimal value. Can this be fixed as well?
hey all,
Is there any way this problem can be fixed in the next release? This is a pretty significant rounding issue that i’m sure other users will also have problem with. Please help
thanks!
sammy
sammyzheng:
Thanks for fixing it. However, it looks like there is now an issue with borderline numbers rounding down when they should round up. For example:0.5 rounds to 0 (when display format is “0”)
0.25 rounds to 0.2 (when display format is “0.0”)Those 2 examples should definitely round up since the binary representation is exactly the same as the decimal value. Can this be fixed as well?
This new (round-up) problem will be fixed in Servoy 3.5.4.