strange data conversion problem

[Former Member]

Hi all,

I have a strange data conversion problem. I have a few fields of type QUAN, with runtime values of Var1 - "499.000" and Var2 - "12.500". I divide Var1 by Var2 and push the result in another field of type QUAN, Var3. The result should be '39.920'

The problem is, the result in Var3 is '0.040'. The value getsw rounded off to 40 and then gets assigned to this variable. The funny part is that the value gets copied to Var3 directly without conversion, and ends up in the decimal part of the variable.

I tried this on another program and it worked fine. But here, there seems to be some other problem. I tried with packed decimals as well but got the same result.

Another strange thing is that usually if you assign a value like '123.456' from a packed decimal of dec. length 3 to another of length 2 the result would be '123.46', but when I do it in this program, it becomes '1234.56'. I fail to understand why this is happening.

I believe this has something to do with the program's attributes/environment. The same code works perfectly in other programs.

Please Help!!

Thanks in advance,

Madhur

[RichHeilman]

Your right it might have something to with the program attributes. You need to check the "Fix Point Arithimatic" Check box on the attributes screen on the program. If it is not checked, then this is why you are seeing this behavior. You may be able to just click this checkbox on and it will solve the problem, but there may be a reason why it is off. If you don't check the box, you will need to handle manully which means that you will need to multiple by 100 or 1000 depending on the number of decimals places.

Regards,

Rich Heilman

[Former Member]

Hi Rich/Vikram,

Thanks a ton.

Regards,

Madhur

[Former Member]

Hi Madhur,

Check in the program attributes, if the Fixed point arithmatic is checked.

Sometimes, such kind of errors happen if the checkbox is not checked then all QUAN, packed decimals will treated as integers only.

Cheers,

Vikram

Pls reward for helpful replies!!