another (different) rounding problem

Questions and answers on designing your Servoy solutions, database modelling and other 'how do I do this' that don't fit in any of the other categories

another (different) rounding problem

Postby Harjo » Wed Feb 04, 2015 11:52 am

Hi, I have (different) rounding problem

I have an invoice, with invoice lines.
one invoice line has:

quantity * (saleprice - discount %) = total
quantity = 12
saleprice = 8.95
discount = 10 (%)

to calculate the price minus discount the calculation is as follows:
var a = 8.95 - (8.95 * (0.01 * 10)) //==8.055
var b = Math.round(a * 100) / 100 //== 8.06

12 * 8.06 = 96.72

This is correct, and there is further no problem.
But sometimes we duplicate the invoice to create a Credit Invoice.
We duplicate than the invoice-line and set the 8.95 to -8.95

but now the rounding goes wrong (in our case)
var a = -8,95 - (-8.95 * (0.01 * 10)) //== -8.055
var b = Math.round(a * 100) / 100 //== -08.05

12 * -8.05 = -96.60

As you can see in this example, the difference is 0.12.
I was wondering if someone came across this problem, and how to solve this.

I have some ideas, like: setting the quantity to -12, instead of the price.
Than the calculation goes oke, but I can't prevent users, by setting manually the price to a negative number..
Harjo Kompagnie
Direct ICT / Servoy Hosting / ServoyCamp
Servoy Certified Developer
Servoy Valued Professional
SAN Developer
User avatar
Harjo
 
Posts: 4289
Joined: Fri Apr 25, 2003 11:42 pm
Location: DEN HAM OV, The Netherlands

Re: another (different) rounding problem

Postby lwjwillemsen » Wed Feb 04, 2015 1:24 pm

Hi Harjo,

What happens if you do the rounding on a positive number and apply the negate after the rounding ?

var a = 8,95 - (8.95 * (0.01 * 10)) //== 8.055
var b = Math.round(a * 100) / 100 //== 08.06
var c = - b // == -08.06 ?

Regards,
Lambert Willemsen
Vision Development BV
lwjwillemsen
 
Posts: 602
Joined: Sat Mar 14, 2009 5:39 pm
Location: The Netherlands

Re: another (different) rounding problem

Postby jdbruijn » Wed Feb 04, 2015 1:28 pm

not sure if it is a useful suggestion but you could always check if the value is negative and than use Math.abs(x) to convert it to a positive value. At the end you only have to convert the result back to a negative value.
Jos de Bruijn
Welcome CCS
Servoy Certified Developer
Image
jdbruijn
 
Posts: 466
Joined: Sun Apr 11, 2010 6:34 pm

Re: another (different) rounding problem

Postby Harjo » Wed Feb 04, 2015 2:33 pm

I came across, when using a negative number you can use: Math.floor.

So I corrected it, by doing this:

Code: Select all
var vValue = 8.95 //or -8.95
var a = vValue - (vValue * (0.01 * 10))
var b = 0
if(vValue >= 0) {
   b = Math.round(a * 100) / 100
} else {
   b = Math.floor(a * 100) / 100
}


if vValue is positive or negative, the rounding is now always correct.
Harjo Kompagnie
Direct ICT / Servoy Hosting / ServoyCamp
Servoy Certified Developer
Servoy Valued Professional
SAN Developer
User avatar
Harjo
 
Posts: 4289
Joined: Fri Apr 25, 2003 11:42 pm
Location: DEN HAM OV, The Netherlands

Re: another (different) rounding problem

Postby omar » Wed Feb 04, 2015 6:23 pm

Personally I would lookup the value of the original invoice and negate it. If the price of the item changes between invoicing and crediting you may wind up giving back more money than was paid regardless of how you round...
Intrasoft, Founder
Omar van Galen
[email protected]
+31-(0)6-21234586
Servoy Developer
omar
 
Posts: 319
Joined: Sat Feb 12, 2011 4:51 pm
Location: Intrasoft, The Netherlands

Re: another (different) rounding problem

Postby Harjo » Wed Feb 04, 2015 7:25 pm

omar wrote:Personally I would lookup the value of the original invoice and negate it.

That is exactly what I did and do Omar.

I duplicate the invoice-line and negate, the piece price...
but if you calculate further (with quantity and maybe discount you gave) with negative numbers, the rounding goes wrong..
Harjo Kompagnie
Direct ICT / Servoy Hosting / ServoyCamp
Servoy Certified Developer
Servoy Valued Professional
SAN Developer
User avatar
Harjo
 
Posts: 4289
Joined: Fri Apr 25, 2003 11:42 pm
Location: DEN HAM OV, The Netherlands

Re: another (different) rounding problem

Postby lwjwillemsen » Wed Feb 04, 2015 8:18 pm

Hi Harjo,

Math.floor(-8.3) = -9
Math.round(8.3) = 8

I don't think that's what you want...

Math.round(8.5) = 9
- Math.round(8.5) = -9

Math.round(8.3) = 8
- Math.round(8.3) = -8

My advice : take my advice...

From my Foxpro legacy :

Foxpro round(1.5, 0) = 2 (round 1.5 to 0 decimals)
Foxpro round(-1.5, 0) = -2

Regards,
Lambert Willemsen
Vision Development BV
lwjwillemsen
 
Posts: 602
Joined: Sat Mar 14, 2009 5:39 pm
Location: The Netherlands

Re: another (different) rounding problem

Postby Harjo » Wed Feb 04, 2015 9:13 pm

Hi Lambert, I like this discussion! :wink:
I see your point when you don't multiply by 100 and than do / by 100

Than, what would you advice to change this method as shown under, to detect if it's negative number, transform it to a positive number, do the rounding, and put it back to a negative number

Code: Select all
var vValue = 8.95 //or -8.95
var a = vValue - (vValue * (0.01 * 10))
var b = 0
if(vValue >= 0) {
   b = Math.round(a * 100) / 100
} else {
   b = Math.floor(a * 100) / 100
}


I have no knowledge of FoxPro, but do you mean, that rounding in FoxPro, is ok?, despite if you using positive or negative numbers?
Harjo Kompagnie
Direct ICT / Servoy Hosting / ServoyCamp
Servoy Certified Developer
Servoy Valued Professional
SAN Developer
User avatar
Harjo
 
Posts: 4289
Joined: Fri Apr 25, 2003 11:42 pm
Location: DEN HAM OV, The Netherlands

Re: another (different) rounding problem

Postby lwjwillemsen » Wed Feb 04, 2015 10:18 pm

Hi Harjo,

I like this discussion also, I have a weak spot for numerical computations and numerical precision...

Foxpro does the rounding (of negative numbers) in a fashion you asked for 8) .

You can achieve the same fashion in Java(script) by always do the rounding on a positive number and negate
the result if the original value before rounding was negative.

I put that code today in our own global Servoy round() method after reading your question.

There is no mathematical law that says what round(-1.5) must return (-1 or -2) so what is
wrong or what is right is arbitrary. I like round(-1.5) to return -2 because round(1.5) returns 2.

Anyway, thanks for stirring this up...
Lambert Willemsen
Vision Development BV
lwjwillemsen
 
Posts: 602
Joined: Sat Mar 14, 2009 5:39 pm
Location: The Netherlands

Re: another (different) rounding problem

Postby omar » Wed Feb 04, 2015 11:54 pm

Bit kludgy but it works and it supports any number of decimals:

Code: Select all
function round(nExpression, nDecimals) {
   var result = Math.round(Math.abs(nExpression) * Math.pow(10, nDecimals)) / Math.pow(10,nDecimals)
   if (nExpression<0) {return result*-1} else {return result}
}

round(-8.055, 2)  // -8.06

round(8.055, 2)  // 8.06
Intrasoft, Founder
Omar van Galen
[email protected]
+31-(0)6-21234586
Servoy Developer
omar
 
Posts: 319
Joined: Sat Feb 12, 2011 4:51 pm
Location: Intrasoft, The Netherlands

Re: another (different) rounding problem

Postby lwjwillemsen » Thu Feb 05, 2015 12:12 am

Hi Omar,

Yep ! That's my function :)

Regards,
Lambert Willemsen
Vision Development BV
lwjwillemsen
 
Posts: 602
Joined: Sat Mar 14, 2009 5:39 pm
Location: The Netherlands

Re: another (different) rounding problem

Postby ROCLASI » Thu Feb 05, 2015 1:03 am

Interesting discussion :)
Here is a slightly cleaner version of that function:
Code: Select all
function round(_nValue, _nPrecision) {
    var _nAbsValue = Math.abs(_nValue),
        _nDivision = Math.pow(10, _nPrecision);
    return (Math.round(_nAbsValue * _nDivision) / _nDivision) * (_nValue / _nAbsValue);
}


Or if you really wanted to as the (in)famous Oneliner :D :
Code: Select all
function round(_nValue, _nPrecision) {
    return (Math.round(Math.abs(_nValue) * Math.pow(10, _nPrecision)) / Math.pow(10, _nPrecision)) * (_nValue / Math.abs(_nValue));
}
Robert Ivens
ROCLASI Software Solutions / JBS Group, Partner
SAN Developer / Servoy Valued Professional / Servoy Certified Developer
Twitter: @roclasi / @servoyforge
--
ServoyForge - Building Open Source Software.
PostgreSQL - The world's most advanced open source database.
User avatar
ROCLASI
Servoy Expert
 
Posts: 5281
Joined: Thu Oct 02, 2003 9:49 am
Location: Netherlands/Belgium

Re: another (different) rounding problem

Postby lwjwillemsen » Thu Feb 05, 2015 9:14 am

Hi Robert,

Already recovered from the fosdem ?

About the (_nValue / _nAbsValue) :

In Foxpro we had the handy numerical function sign(x) with three possible return values.
x<0 : -1, x==0 : 0, x>0 : 1

We often did a check like sign(a)<>sign(b) in our code.

Regards,
Lambert Willemsen
Vision Development BV
lwjwillemsen
 
Posts: 602
Joined: Sat Mar 14, 2009 5:39 pm
Location: The Netherlands

Re: another (different) rounding problem

Postby omar » Thu Feb 05, 2015 10:32 am

Nice Robert! Now maybe a bugfix/feature request for the round function in Servoy to work like that?

Hi Lambert, VFP had a rich language but has also had it's quirks and flaws. And we have to move on so pretty soon I will be taking down www.visualfoxpro.com. If people haven't left VFP yet they probably never will :-)
Intrasoft, Founder
Omar van Galen
[email protected]
+31-(0)6-21234586
Servoy Developer
omar
 
Posts: 319
Joined: Sat Feb 12, 2011 4:51 pm
Location: Intrasoft, The Netherlands

Re: another (different) rounding problem

Postby ROCLASI » Thu Feb 05, 2015 11:02 am

lwjwillemsen wrote:Already recovered from the fosdem ?


Oh yes, we had fun. Friday we (PgEurope) organized for the third time a PgDay in front of FOSDEM. Which is a full day of talks about PostgreSQL. It was fun.
And the first day of FOSDEM (Saturday) we had a devroom so another full day of talks. The Sunday we had Slonik (the Pg mascotte) walking around.
We even said hi to the MySQL booth :twisted: . Too bad they couldn't be bothered to actually man their booth because they had a devroom to run....(tsk tsk... :roll: )

lwjwillemsen wrote:About the (_nValue / _nAbsValue) :

In Foxpro we had the handy numerical function sign(x) with three possible return values.
x<0 : -1, x==0 : 0, x>0 : 1

We often did a check like sign(a)<>sign(b) in our code.

Essentially all these functions that Servoy uses are Javascript functions, not persé Servoy functions.
And I see that the Math.sign() function is proposed in the ECMAScript 6 specification, still experimental at this stage.
Robert Ivens
ROCLASI Software Solutions / JBS Group, Partner
SAN Developer / Servoy Valued Professional / Servoy Certified Developer
Twitter: @roclasi / @servoyforge
--
ServoyForge - Building Open Source Software.
PostgreSQL - The world's most advanced open source database.
User avatar
ROCLASI
Servoy Expert
 
Posts: 5281
Joined: Thu Oct 02, 2003 9:49 am
Location: Netherlands/Belgium

Next

Return to Programming with Servoy

Who is online

Users browsing this forum: Bing [Bot] and 10 guests