Application Development and Automation Discussions
Join the discussions or start your own on all things application development, including tools and APIs, programming models, and keeping your skills sharp.
cancel
Showing results for 
Search instead for 
Did you mean: 
Read only

Floating Point Conversion

Former Member
0 Likes
460

Hi,

I have a question on how integers are converted to floating point variables. I have a formula that involves a floating point variable but it reacts differently in different boxes.

data: x_floating_pt type f.

data: x_var1 like xxxx ---> This has a data type of quan 13 dec 3

data: x_type_p type p decimals 0.

x_var1 = 100.00

x_floating_pt = x_var1.

add x_floating_pt to x_type_p.

In one box, x_type_p will result into 100. While in another box, it will result into 100,000 (which requires me to divide it by 1000 to get the correct result).

Thanks.

1 ACCEPTED SOLUTION
Read only

Former Member
0 Likes
427

Hi, DWade,

What do you mean by "different boxes"?

I'm not sure, if the different situations are in different programs, maybe it is caused by the different program attribute set on "fixed point arithmetic".

2 REPLIES 2
Read only

Former Member
0 Likes
428

Hi, DWade,

What do you mean by "different boxes"?

I'm not sure, if the different situations are in different programs, maybe it is caused by the different program attribute set on "fixed point arithmetic".

Read only

Former Member
0 Likes
427

Thanks for your reply ! yes, the fixed point checkbox change does it