Application Development and Automation Discussions
Join the discussions or start your own on all things application development, including tools and APIs, programming models, and keeping your skills sharp.
cancel
Showing results for 
Search instead for 
Did you mean: 
Read only

How big is the Float Type

Former Member
0 Likes
1,265

What is the maximum value that a P and a F can take?

And another question, can I assign a Float type to a Package type (if it is inferior to the package type)

1 ACCEPTED SOLUTION
Read only

Former Member
0 Likes
1,014

This may be a bit long...It is from F1 on Type.

**********************************************************

The value range of type P fields depends on their length and the number of decimal places. P fields can be 1 to 16 bytes long, with two decimal digits packed into each byte, and one decimal digit and the sign packed into the last byte. There can be up to 14 decimal places. Auxiliary fields for intermediate results are always 16 bytes long and can thus hold up to 31 decimal digits. To ensure that the decimal point is correctly calculated, you should always set the program attribute "fixed point arithmetic". Otherwise, all numbers are specified as integers and all intermediate results for the next integer are rounded. If "fixed point arithmetic" is not set, the decimal places defined for the number only appear when outputting with WRITE.

Fixed point arithmetic is decimal arithmetic and is similar to using a pocket calculator or calculating with paper and pencil.

Type P is typically used for sizes, lengths, weights and sums of money.

Rule: If you want to calculate "down to the last penny", you should use the type P.

Type F values range from /- 2.2250738585072014E-308 to 1.7976931348623157E308, as well as the number 0, with an accuracy of at least 15 decimal places.

You cannot enter floating point numbers directly in your programs. Instead, you must use text literals that can be interpreted as floating point numbers. You may use the following formats:

Decimal numbers with or without sign, with or without decimal point. The form <mantissa> E<exponent>, where the mantissa is a decimal. The exponent may be specified either with or without sign. You may also use spaces before or after the number. Examples of text literals with "floating point numbers":

'1', '-12.34567', '-765E-04', '1234E5', '12E34', '+12.3E-4', '1E160'.

Use floating point arithmetic if you need a very large value range or you are making decimal calculations, but be aware of the following features of floating point arithmetic.

What is the maximum value that a P and a F can take?

And another question, can I assign a Float type to a Package type (if it is inferior to the package type)

2 REPLIES 2
Read only

RichHeilman
Developer Advocate
Developer Advocate
0 Likes
1,014

It appears that for type F that the limit is 8 and for type P, the limit is 15.



report zrich_0002 .


data: f type f,
      p type p.


      f = 99999999.
      p = 999999999999999.

      write:/ f.
      write:/ p.


      p = f.

      write:/ p.

Yes, you can move a float to a packed field.

Regards,

Rich Heilman

Read only

Former Member
0 Likes
1,015

This may be a bit long...It is from F1 on Type.

**********************************************************

The value range of type P fields depends on their length and the number of decimal places. P fields can be 1 to 16 bytes long, with two decimal digits packed into each byte, and one decimal digit and the sign packed into the last byte. There can be up to 14 decimal places. Auxiliary fields for intermediate results are always 16 bytes long and can thus hold up to 31 decimal digits. To ensure that the decimal point is correctly calculated, you should always set the program attribute "fixed point arithmetic". Otherwise, all numbers are specified as integers and all intermediate results for the next integer are rounded. If "fixed point arithmetic" is not set, the decimal places defined for the number only appear when outputting with WRITE.

Fixed point arithmetic is decimal arithmetic and is similar to using a pocket calculator or calculating with paper and pencil.

Type P is typically used for sizes, lengths, weights and sums of money.

Rule: If you want to calculate "down to the last penny", you should use the type P.

Type F values range from /- 2.2250738585072014E-308 to 1.7976931348623157E308, as well as the number 0, with an accuracy of at least 15 decimal places.

You cannot enter floating point numbers directly in your programs. Instead, you must use text literals that can be interpreted as floating point numbers. You may use the following formats:

Decimal numbers with or without sign, with or without decimal point. The form <mantissa> E<exponent>, where the mantissa is a decimal. The exponent may be specified either with or without sign. You may also use spaces before or after the number. Examples of text literals with "floating point numbers":

'1', '-12.34567', '-765E-04', '1234E5', '12E34', '+12.3E-4', '1E160'.

Use floating point arithmetic if you need a very large value range or you are making decimal calculations, but be aware of the following features of floating point arithmetic.