## About Decimals... [EDIT]

Discuss anything else that does not suite other forums.

Moderator: Rathinagiri

Roberto Lopez
HMG Founder
Posts: 3966
Joined: Wed Jul 30, 2008 6:43 pm
Has thanked: 23 times
Been thanked: 161 times

### About Decimals... [EDIT]

Hi All,

Usually I work with two decimals in math operations with no problems.

Now, I need to work with three decimals, converting number to strings and strings to numbers...

The result of the following should be 1.000, but, it is shown as 0.996.

I've played with SET DECIMALS and SET FIXED without results...

What I'm doing wrong?

Code: Select all

``````

N1 := 1/12

CN1 := STR( N1 ,7,3)

VN1 := VAL( CN1 )

N2 := VN1 * 12

MSGINFO( STR( N2 ,7,3) )

``````
If you do this:

Code: Select all

``````	N1 := 1 / 12

N2 := N1 * 12

MSGINFO( STR( N2 ,7,3) )

``````
The result is 1.000

TIA.
Regards/Saludos,

Roberto

(Veritas Filia Temporis)

IMATECH
Posts: 169
Joined: Sun May 27, 2012 9:33 pm
Location: Brazil: Goiânia-GO.
Been thanked: 1 time

Code: Select all

``````	N1 := Round( 1/12, 3 )
CN1 := Str( N1 ,7,3)
VN1 := Val( CN1 )
N2 := Round( VN1 * 12, 3 )
MSGINFO( STR( N2 ,7,3) )
``````
*Not tested
M., Ronaldo

By: IMATECH

Imation Tecnologia

Roberto Lopez
HMG Founder
Posts: 3966
Joined: Wed Jul 30, 2008 6:43 pm
Has thanked: 23 times
Been thanked: 161 times
IMATECH wrote:
Tue Aug 14, 2018 12:03 am

Code: Select all

``````	N1 := Round( 1/12, 3 )
CN1 := Str( N1 ,7,3)
VN1 := Val( CN1 )
N2 := Round( VN1 * 12, 3 )
MSGINFO( STR( N2 ,7,3) )
``````
*Not tested
The result is the same: 0.996

Thanks anyway!
Regards/Saludos,

Roberto

(Veritas Filia Temporis)

andyglezl
Posts: 1149
Joined: Fri Oct 26, 2012 7:58 pm
Location: Guadalajara Jalisco, MX
Has thanked: 35 times
Been thanked: 68 times
Contact:
Maybe this way

FUNCTION ConDeci()
SET FIXED ON
SET DECI TO 15
N1 := 1/12
msgdebug( N1 )
CN1 := ALLTRIM( STR( N1, 15, 12 ) )
msgdebug( CN1 )
VN1 := VAL( CN1 )
msgdebug( VN1 )
SET DECI TO 2
N2 := VN1 * 12
msgdebug( N2 )
msgdebug( STR( N2 ,7,3) )
RETURN Nil
Andrés González López
Desde Guadalajara, Jalisco. México.

Roberto Lopez
HMG Founder
Posts: 3966
Joined: Wed Jul 30, 2008 6:43 pm
Has thanked: 23 times
Been thanked: 161 times
andyglezl wrote:
Tue Aug 14, 2018 2:12 am
Maybe this way

FUNCTION ConDeci()
SET FIXED ON
SET DECI TO 15
N1 := 1/12
msgdebug( N1 )
CN1 := ALLTRIM( STR( N1, 15, 12 ) )
msgdebug( CN1 )
VN1 := VAL( CN1 )
msgdebug( VN1 )
SET DECI TO 2
N2 := VN1 * 12
msgdebug( N2 )
msgdebug( STR( N2 ,7,3) )
RETURN Nil
Thanks Andrés, it works.

Doug answered my question in Harbour users group too.

I'm introducing the problem, limiting the decimals to 3.

I'm doing the conversions, since I'm storing the values in a grid (as string) and then recalling them to store in a table (as numbers).

Thanks again!
Regards/Saludos,

Roberto

(Veritas Filia Temporis)

edk
Posts: 462
Joined: Thu Oct 16, 2014 11:35 am
Location: Poland
Has thanked: 135 times
Been thanked: 377 times
The variable N1: = 1/12 gives the result: 0.08333(3)
When converting:
CN1: = STR (N1, 7, 3)
VN1: = VAL (CN1)
you rounded up to three decimal, which resulted in 0.083
In such a rounded number, multiplying by 12, we get the result: 0.996
Roberto Lopez wrote:
Mon Aug 13, 2018 11:41 pm

Code: Select all

``````

N1 := 1/12		//=> 0.083333333

CN1 := STR( N1 ,7,3)	//=> "  0.083"

VN1 := VAL( CN1 )	//=> 0.083

N2 := VN1 * 12		//=> 0.996

MSGINFO( STR( N2 ,7,3) )	//=> "  0.996"

``````
See that:

Code: Select all

``````SET DECIMALS TO 3
N1 := 1/12
VN1 := VAL (STR( N1 ,7,3))
MSGDebug(N1, VN1, N1 == VN1)
``````
Although SET DECIMALS TO 3 is set, the value of variable N1 is stored with the greatest precision. SET DECIMALS has no effect on the result of the calculation, only on its display.
Therefore, the value of N1== VN1 is returned as .F.

SET DECIMALS
Toggle the console display

Syntax:
SET DECIMALS TO [<nDecimal>]
Arguments:
<nDecimal> Number of decimals places
Description:
This command establishes the number of decimal places that Harbour will display in mathematical calculations, functions, memory variables, and fields.

andyglezl
Posts: 1149
Joined: Fri Oct 26, 2012 7:58 pm
Location: Guadalajara Jalisco, MX
Has thanked: 35 times
Been thanked: 68 times
Contact:
Roberto Lopez wrote:
Tue Aug 14, 2018 6:40 am

Thanks Andrés, it works.

Doug answered my question in Harbour users group too.

I'm introducing the problem, limiting the decimals to 3.

I'm doing the conversions, since I'm storing the values in a grid (as string) and then recalling them to store in a table (as numbers).

Thanks again!
Al contrario Roberto, gracias a ti.
Andrés González López
Desde Guadalajara, Jalisco. México.