Page 1 of 1
Mysterious counting of the INT() function
Posted: Tue Apr 10, 2018 1:14 pm
by edk
To all.
Do you also get the number 5 instead of the expected 6 as a result of
Code: Select all
msgdebug( INT( ( 129.6 - 129 ) * 10 ) )
Edward.
Re: Mysterious counting of the INT() function
Posted: Tue Apr 10, 2018 1:31 pm
by apais
floating point issues !
Re: Mysterious counting of the INT() function
Posted: Tue Apr 10, 2018 2:04 pm
by serge_girard
Yes, me too...
Serge
Re: Mysterious counting of the INT() function
Posted: Mon Apr 16, 2018 7:16 pm
by SALINETAS24
msgdebug( INT( VAL(STR(( 129.6 - 129 ) * 10 ) )))
Raro, raro .., raro
Re: Mysterious counting of the INT() function
Posted: Tue Apr 17, 2018 10:00 am
by Anand
We are using str() with default values of decimal points and again val() to convert it, so problem can come up at random times. I am not sure when and why, but have faced such situation in Clipper and Xbase++ many times.
If it can be avoided then use int((129.6-129)*10), else use round(x,n) to keep the decimals at known places.
Regards,
Anand
Re: Mysterious counting of the INT() function
Posted: Tue Apr 17, 2018 1:09 pm
by edk
As Angel points out
apais wrote: ↑Tue Apr 10, 2018 1:31 pm
floating point issues !
this is a problem with the binary format of floating point numbers:
https://harbour.github.io/the-oasis/clipper-5.html
Using integers in the calculations performed, solves the problem.
Code: Select all
msgdebug( INT( ( 129.6 - 129 ) * 10 ) ) vs. msgdebug( INT( 129.6*10 - 129*10 ) )