topic Re: Doubt in type p in Application Development and Automation Discussions

Doubt in type p

Former Member — Wed, 20 Aug 2008 12:55:58 GMT

Dear experts,

I have declared the variable as

Data betrg type p decimal 3.

But for this it is showing the wrong ouput

e.g:-

1/30 = 0.033333333333

but it is taking only 0.033

so afer multipication value is decresing from the orginl value

similaily

8/30 = 0.26666666666666666

but it is taking as .267

and i m not getting the actual value.

What type of declaration I will do to get the actual value.

Regards

Ansuman Parhi

Re: Doubt in type p

peter_ruiz2 — Wed, 20 Aug 2008 13:00:45 GMT

hi,

This is because you have declare a variable with the P DECIMALS 3 meaning it will only accept 3 decimal places. try changing the data type to P DECIMALS 11 to be able to achieve your desired output.

regards,

Peter

Re: Doubt in type p

Former Member — Wed, 20 Aug 2008 13:03:21 GMT

Please write in this way

data xyz type p decimal 8 .

Now it will take upto 8 decimal places .

~hitesh

Re: Doubt in type p

Former Member — Wed, 20 Aug 2008 13:04:26 GMT

Hi,

since ur specifying the decimal precision , it is rounding of to your req no of decimals.if you want to have more precision , increase the no of decimals and do a offset moving to another variable.

Raj D

Re: Doubt in type p

vinod_vemuru2 — Wed, 20 Aug 2008 13:37:05 GMT

Hi Ansuman,

As others told increase the number of decimals.

Other thing i understand from ur query is why

0.033333333333 as 0.033 and 0.26666666666666666 as 0.027.

This is because of rounding is happening.

Here 6666 is GT 5 that is why rounding to 7

3333 is LT 5 so rounding to 3 itself.

Thanks,

Vinod.

Re: Doubt in type p

Former Member — Thu, 21 Aug 2008 04:08:40 GMT

Dear Vinod,

Even if I am increasing the decimal place it is not giving me the actual value.

I have increased it to 11 places and still it is giving me 214.999 and for other cases it is giving me 1064.666666.But my desired outpu must be 215 and 1064.67.

So is there any other way for getting the actual value.

Regards

Ansuman

Re: Doubt in type p

Former Member — Thu, 21 Aug 2008 04:13:49 GMT

Hi,

U can use ROUND function for that.

Re: Doubt in type p

Former Member — Thu, 21 Aug 2008 04:24:52 GMT

Hi ,

from you post i understand that you req a precision of 2 decimals.

like if it is 214.9999 you want it as 215.00 and it it is 1064.66666 you want it to be 1064.67.

So specify the length as 2 instead of 3 for p data type.

Hope this solves your problem'

Raj D

Re: Doubt in type p

Former Member — Thu, 21 Aug 2008 04:48:58 GMT

hi,

i think this is due to your decimal precision

if you define data like this:

data:   num type p decimals 3.

of course you will get only 3 decimals.

so if you want to get more decimals, you should increase the decimal precision.

Re: Doubt in type p

Former Member — Thu, 21 Aug 2008 06:04:11 GMT

hi anshumaan,

as u told that ur query is giving wrong output. it is due to the declaration part where u are declaring the data type as BETRG TYPE P DECIMAL 3. since u have put 3 after DECIMAL so it will give the value upto 3 decimal places with rounding off the last digit. So if u want to increase the accuracy u will have to do like

BETRG TYPE P DECIMAL 10 as much places u want upto but still it will round off.

thanx

nitesh