About Decimals... [EDIT]

Discuss anything else that does not suite other forums.

Moderator: Rathinagiri

Post Reply
User avatar
Roberto Lopez
HMG Founder
Posts: 3919
Joined: Wed Jul 30, 2008 6:43 pm
Has thanked: 15 times
Been thanked: 140 times

About Decimals... [EDIT]

Post by Roberto Lopez » Mon Aug 13, 2018 11:41 pm

Hi All,

Usually I work with two decimals in math operations with no problems.

Now, I need to work with three decimals, converting number to strings and strings to numbers...

The result of the following should be 1.000, but, it is shown as 0.996.

I've played with SET DECIMALS and SET FIXED without results...

What I'm doing wrong?

Code: Select all

	
	
	N1 := 1/12

	CN1 := STR( N1 ,7,3)

	VN1 := VAL( CN1 )

	N2 := VN1 * 12

	MSGINFO( STR( N2 ,7,3) )

If you do this:

Code: Select all

	N1 := 1 / 12

	N2 := N1 * 12

	MSGINFO( STR( N2 ,7,3) )
		
The result is 1.000

TIA.
Regards/Saludos,

Roberto


(Veritas Filia Temporis)

User avatar
IMATECH
Posts: 168
Joined: Sun May 27, 2012 9:33 pm
Location: Brazil: Goiânia-GO.

Post by IMATECH » Tue Aug 14, 2018 12:03 am

Code: Select all

	N1 := Round( 1/12, 3 )
	CN1 := Str( N1 ,7,3)
	VN1 := Val( CN1 )
	N2 := Round( VN1 * 12, 3 )
	MSGINFO( STR( N2 ,7,3) )
*Not tested
M., Ronaldo

By: IMATECH

Imation Tecnologia

User avatar
Roberto Lopez
HMG Founder
Posts: 3919
Joined: Wed Jul 30, 2008 6:43 pm
Has thanked: 15 times
Been thanked: 140 times

Post by Roberto Lopez » Tue Aug 14, 2018 12:52 am

IMATECH wrote:
Tue Aug 14, 2018 12:03 am

Code: Select all

	N1 := Round( 1/12, 3 )
	CN1 := Str( N1 ,7,3)
	VN1 := Val( CN1 )
	N2 := Round( VN1 * 12, 3 )
	MSGINFO( STR( N2 ,7,3) )
*Not tested
The result is the same: 0.996

Thanks anyway!
Regards/Saludos,

Roberto


(Veritas Filia Temporis)

User avatar
andyglezl
Posts: 1033
Joined: Fri Oct 26, 2012 7:58 pm
Location: Guadalajara Jalisco, MX
Has thanked: 22 times
Been thanked: 44 times
Contact:

Post by andyglezl » Tue Aug 14, 2018 2:12 am

Maybe this way

FUNCTION ConDeci()
SET FIXED ON
SET DECI TO 15
N1 := 1/12
msgdebug( N1 )
CN1 := ALLTRIM( STR( N1, 15, 12 ) )
msgdebug( CN1 )
VN1 := VAL( CN1 )
msgdebug( VN1 )
SET DECI TO 2
N2 := VN1 * 12
msgdebug( N2 )
msgdebug( STR( N2 ,7,3) )
RETURN Nil
Andrés González López
Desde Guadalajara, Jalisco. México.

User avatar
Roberto Lopez
HMG Founder
Posts: 3919
Joined: Wed Jul 30, 2008 6:43 pm
Has thanked: 15 times
Been thanked: 140 times

Post by Roberto Lopez » Tue Aug 14, 2018 6:40 am

andyglezl wrote:
Tue Aug 14, 2018 2:12 am
Maybe this way

FUNCTION ConDeci()
SET FIXED ON
SET DECI TO 15
N1 := 1/12
msgdebug( N1 )
CN1 := ALLTRIM( STR( N1, 15, 12 ) )
msgdebug( CN1 )
VN1 := VAL( CN1 )
msgdebug( VN1 )
SET DECI TO 2
N2 := VN1 * 12
msgdebug( N2 )
msgdebug( STR( N2 ,7,3) )
RETURN Nil
Thanks Andrés, it works.

Doug answered my question in Harbour users group too.

I'm introducing the problem, limiting the decimals to 3.

I'm doing the conversions, since I'm storing the values in a grid (as string) and then recalling them to store in a table (as numbers).

Thanks again!
Regards/Saludos,

Roberto


(Veritas Filia Temporis)

edk
Posts: 362
Joined: Thu Oct 16, 2014 11:35 am
Location: Poland
Has thanked: 107 times
Been thanked: 262 times

Post by edk » Tue Aug 14, 2018 9:24 am

The variable N1: = 1/12 gives the result: 0.08333(3)
When converting:
CN1: = STR (N1, 7, 3)
VN1: = VAL (CN1)
you rounded up to three decimal, which resulted in 0.083
In such a rounded number, multiplying by 12, we get the result: 0.996
Roberto Lopez wrote:
Mon Aug 13, 2018 11:41 pm

Code: Select all

	
	
	N1 := 1/12		//=> 0.083333333

	CN1 := STR( N1 ,7,3)	//=> "  0.083"

	VN1 := VAL( CN1 )	//=> 0.083

	N2 := VN1 * 12		//=> 0.996

	MSGINFO( STR( N2 ,7,3) )	//=> "  0.996"

See that:

Code: Select all

SET DECIMALS TO 3
N1 := 1/12
VN1 := VAL (STR( N1 ,7,3))
MSGDebug(N1, VN1, N1 == VN1)
Although SET DECIMALS TO 3 is set, the value of variable N1 is stored with the greatest precision. SET DECIMALS has no effect on the result of the calculation, only on its display.
Therefore, the value of N1== VN1 is returned as .F.


SET DECIMALS
Toggle the console display

Syntax:
SET DECIMALS TO [<nDecimal>]
Arguments:
<nDecimal> Number of decimals places
Description:
This command establishes the number of decimal places that Harbour will display in mathematical calculations, functions, memory variables, and fields.

User avatar
andyglezl
Posts: 1033
Joined: Fri Oct 26, 2012 7:58 pm
Location: Guadalajara Jalisco, MX
Has thanked: 22 times
Been thanked: 44 times
Contact:

Post by andyglezl » Tue Aug 14, 2018 6:35 pm

Roberto Lopez wrote:
Tue Aug 14, 2018 6:40 am


Thanks Andrés, it works.

Doug answered my question in Harbour users group too.

I'm introducing the problem, limiting the decimals to 3.

I'm doing the conversions, since I'm storing the values in a grid (as string) and then recalling them to store in a table (as numbers).

Thanks again!
Al contrario Roberto, gracias a ti.
Andrés González López
Desde Guadalajara, Jalisco. México.

Post Reply