About Decimals... [EDIT]

Discuss anything else that does not suite other forums.

Moderator: Rathinagiri

Post Reply
User avatar
Roberto Lopez
HMG Founder
Posts: 4004
Joined: Wed Jul 30, 2008 6:43 pm

About Decimals... [EDIT]

Post by Roberto Lopez »

Hi All,

Usually I work with two decimals in math operations with no problems.

Now, I need to work with three decimals, converting number to strings and strings to numbers...

The result of the following should be 1.000, but, it is shown as 0.996.

I've played with SET DECIMALS and SET FIXED without results...

What I'm doing wrong?

Code: Select all

	
	
	N1 := 1/12

	CN1 := STR( N1 ,7,3)

	VN1 := VAL( CN1 )

	N2 := VN1 * 12

	MSGINFO( STR( N2 ,7,3) )

If you do this:

Code: Select all

	N1 := 1 / 12

	N2 := N1 * 12

	MSGINFO( STR( N2 ,7,3) )
		
The result is 1.000

TIA.
Regards/Saludos,

Roberto


(Veritas Filia Temporis)
User avatar
IMATECH
Posts: 188
Joined: Sun May 27, 2012 9:33 pm
Location: Brazil: Goiânia-GO.

Re: About Decimals...

Post by IMATECH »

Code: Select all

	N1 := Round( 1/12, 3 )
	CN1 := Str( N1 ,7,3)
	VN1 := Val( CN1 )
	N2 := Round( VN1 * 12, 3 )
	MSGINFO( STR( N2 ,7,3) )
*Not tested
M., Ronaldo

By: IMATECH

Imation Tecnologia
User avatar
Roberto Lopez
HMG Founder
Posts: 4004
Joined: Wed Jul 30, 2008 6:43 pm

Re: About Decimals...

Post by Roberto Lopez »

IMATECH wrote: Tue Aug 14, 2018 12:03 am

Code: Select all

	N1 := Round( 1/12, 3 )
	CN1 := Str( N1 ,7,3)
	VN1 := Val( CN1 )
	N2 := Round( VN1 * 12, 3 )
	MSGINFO( STR( N2 ,7,3) )
*Not tested
The result is the same: 0.996

Thanks anyway!
Regards/Saludos,

Roberto


(Veritas Filia Temporis)
User avatar
andyglezl
Posts: 1461
Joined: Fri Oct 26, 2012 7:58 pm
Location: Guadalajara Jalisco, MX
Contact:

Re: About Decimals...

Post by andyglezl »

Maybe this way

FUNCTION ConDeci()
SET FIXED ON
SET DECI TO 15
N1 := 1/12
msgdebug( N1 )
CN1 := ALLTRIM( STR( N1, 15, 12 ) )
msgdebug( CN1 )
VN1 := VAL( CN1 )
msgdebug( VN1 )
SET DECI TO 2
N2 := VN1 * 12
msgdebug( N2 )
msgdebug( STR( N2 ,7,3) )
RETURN Nil
Andrés González López
Desde Guadalajara, Jalisco. México.
User avatar
Roberto Lopez
HMG Founder
Posts: 4004
Joined: Wed Jul 30, 2008 6:43 pm

Re: About Decimals...

Post by Roberto Lopez »

andyglezl wrote: Tue Aug 14, 2018 2:12 am Maybe this way

FUNCTION ConDeci()
SET FIXED ON
SET DECI TO 15
N1 := 1/12
msgdebug( N1 )
CN1 := ALLTRIM( STR( N1, 15, 12 ) )
msgdebug( CN1 )
VN1 := VAL( CN1 )
msgdebug( VN1 )
SET DECI TO 2
N2 := VN1 * 12
msgdebug( N2 )
msgdebug( STR( N2 ,7,3) )
RETURN Nil
Thanks Andrés, it works.

Doug answered my question in Harbour users group too.

I'm introducing the problem, limiting the decimals to 3.

I'm doing the conversions, since I'm storing the values in a grid (as string) and then recalling them to store in a table (as numbers).

Thanks again!
Regards/Saludos,

Roberto


(Veritas Filia Temporis)
edk
Posts: 909
Joined: Thu Oct 16, 2014 11:35 am
Location: Poland

Re: About Decimals... [EDIT]

Post by edk »

The variable N1: = 1/12 gives the result: 0.08333(3)
When converting:
CN1: = STR (N1, 7, 3)
VN1: = VAL (CN1)
you rounded up to three decimal, which resulted in 0.083
In such a rounded number, multiplying by 12, we get the result: 0.996
Roberto Lopez wrote: Mon Aug 13, 2018 11:41 pm

Code: Select all

	
	
	N1 := 1/12		//=> 0.083333333

	CN1 := STR( N1 ,7,3)	//=> "  0.083"

	VN1 := VAL( CN1 )	//=> 0.083

	N2 := VN1 * 12		//=> 0.996

	MSGINFO( STR( N2 ,7,3) )	//=> "  0.996"

See that:

Code: Select all

SET DECIMALS TO 3
N1 := 1/12
VN1 := VAL (STR( N1 ,7,3))
MSGDebug(N1, VN1, N1 == VN1)
Although SET DECIMALS TO 3 is set, the value of variable N1 is stored with the greatest precision. SET DECIMALS has no effect on the result of the calculation, only on its display.
Therefore, the value of N1== VN1 is returned as .F.


SET DECIMALS
Toggle the console display

Syntax:
SET DECIMALS TO [<nDecimal>]
Arguments:
<nDecimal> Number of decimals places
Description:
This command establishes the number of decimal places that Harbour will display in mathematical calculations, functions, memory variables, and fields.
User avatar
andyglezl
Posts: 1461
Joined: Fri Oct 26, 2012 7:58 pm
Location: Guadalajara Jalisco, MX
Contact:

Re: About Decimals...

Post by andyglezl »

Roberto Lopez wrote: Tue Aug 14, 2018 6:40 am

Thanks Andrés, it works.

Doug answered my question in Harbour users group too.

I'm introducing the problem, limiting the decimals to 3.

I'm doing the conversions, since I'm storing the values in a grid (as string) and then recalling them to store in a table (as numbers).

Thanks again!
Al contrario Roberto, gracias a ti.
Andrés González López
Desde Guadalajara, Jalisco. México.
Post Reply