‎2007 Oct 15 3:17 AM
Hi,
I have a question on how integers are converted to floating point variables. I have a formula that involves a floating point variable but it reacts differently in different boxes.
data: x_floating_pt type f.
data: x_var1 like xxxx ---> This has a data type of quan 13 dec 3
data: x_type_p type p decimals 0.
x_var1 = 100.00
x_floating_pt = x_var1.
add x_floating_pt to x_type_p.
In one box, x_type_p will result into 100. While in another box, it will result into 100,000 (which requires me to divide it by 1000 to get the correct result).
Thanks.
‎2007 Oct 15 3:27 AM
Hi, DWade,
What do you mean by "different boxes"?
I'm not sure, if the different situations are in different programs, maybe it is caused by the different program attribute set on "fixed point arithmetic".
‎2007 Oct 15 3:27 AM
Hi, DWade,
What do you mean by "different boxes"?
I'm not sure, if the different situations are in different programs, maybe it is caused by the different program attribute set on "fixed point arithmetic".
‎2007 Oct 15 4:36 AM
Thanks for your reply ! yes, the fixed point checkbox change does it