Dynamic Create Data (length) / file record transfer

[Former Member]

Hi,

I need a data variable at runtime (of type X) where I do not know the length at compile time. It is like

   DATA: length              TYPE I VALUE 33.
   DATA: xDataStream(length) TYPE X.

Of course this example doesn't really work, though I looked through

   CREATE DATA

but this doesn't help. I can write that it will be of TYPE X (or any other) but I can't specifiy the length at runtime.

The cause is, and I would like to combine this with the question related to it:

I have a physical file that has any number of given records and a header that gives the record length, e.g.

30
<record-1 length 30>
<record-2 length 30>
...

I would like to read from this file record-by-record. But when using the simple

   ...
   READ DATASET file INTO record.
   ...

the runtime system will transfer as much bytes as the record is defined. So if you had a

   DATA: record(30)   TYPE X.

it works fine but statically.

I do not wish to read the record byte-wise. This is not sexy enough, like:

   DATA: byte               TYPE X,
         recordLength       TYPE I,
         recordBuffer(1024) TYPE X. " Some maximum length

   PERFORM read_header CHANGING recordLength.
   ...
   DO recordLength TIMES.
      DATA: i TYPE I.
      i = sy-index - 1.
      READ DATASET file INTO byte.
      recordBuffer+i(1) = byte.
   ENDDO.
   ...

.

This would perform a system call for each byte of the file and I want to avoid this due to performance issue. Well, you could create a recordBuffer, reading it block-wise and cut-out your own records from the buffer, but I was thinking if anyone has a smart solution.

Best regards,

Florin

[former_member183804]

Hello Florian,

normally XString should do, on older releases an internal table is a common solution.

Regards

Klaus

[RichHeilman]

Is reading the file into an internal table of type string not going to work for you?

data: itab type table of string with header line.


.....


   READ DATASET file INTO itab.
....

append itab.

REgards,

RIch Heilman

[Former Member]

Hello Klaus,

an XString grows dynamically on size but I need a fixed size of a variable that is set at runtime.

I cannot use an internal table for the READ DATASET as well (if you look at the second part of the question).

Hmmm..

Thx & best regards,

Florin

[Former Member]

Hi Rich,

no, because I have mentioned that the amount of bytes that is to be transferred in the READ DATASET statement is defined at runtime. I'm also using the BINARY MODE, so when does the String end when I read the file?

btw: The itab doesn't has a header-line and the read dataset statement should yield a syntax error.

Best regards,

Florin

[former_member183804]

Hello Florin,

creating data requires a full type specification. As x/c require a length specification they can not be used for create data. On 7.00 onwards you may use the RTTS (CL_ABAP_TYPE_DESCR .. ) to create first a type handle and then a data item.

But I do not fully understand why xString is not suitable. Can you give some additional hint?

the dataset implementation uses an internal buffer, so even reading it bytewise thus not mean that each byte is causing an OS call. So using an xString table with a resonable length of the records (say 256 bytes) should be fine from view of performance.

Best Regards

Klaus

[Former Member]

Hi Klaus,

   
   DATA: xRecord   TYPE XSTRING.
   READ DATASET p_file INTO xRecord.

... reads the complete file. My file could be like a couple of megabytes and I would like to avoid reading in into the memory in one go.

Best regards,

Florin

[former_member183804]

Hello Florin,

in that case the extensions MAX LENGTH / ACTUAL LENGTH might be the solution.

Best Regards

Klaus

[Former Member]

Hello Klaus,

the statement READ DATASET does not have any additions beside LENGTH len that stores the read bytes into variable len.

What do you mean? Though I haven't found under the term XSTRING something that refers to the actual size there.

Best regards,

Florin

[former_member183804]

I guess I am on a newer version (7.00)

Sorry

Klaus

[Former Member]

Hello Klaus,

thanks for this info. Good to know that his feature is available in newer version...

Best regards,

Florin

[Former Member]

I have used the workaround to implement a buffered file reader that assignes the record length dynamically.

CLASS file_buffered_reader DEFINITION.

   PUBLIC SECTION.

      " Class assumes that the given filename is correct.
      " There is
      " no error handling when file cannot be opened.
      " The read function will return an empty record on
      " such an
      " attempt.
      METHODS CONSTRUCTOR IMPORTING i_filename   TYPE SAPB-SAPFILES
                                    i_isOpen     TYPE boole-boole.


      METHODS read        IMPORTING i_numBytes     TYPE I
                          EXPORTING e_xRecord      TYPE ANY
                                    e_EOF          TYPE boole-boole.


   PRIVATE SECTION.

      CONSTANTS: RECORDSIZE        TYPE I VALUE 4096.
      DATA: handle                 TYPE SAPB-SAPFILES,
            xBuffer(8192)          TYPE X,  " see constant RECORDSIZE
            xBuffer_fill           TYPE I,
            xBufferReadAhead(4096) TYPE X,
            eof                    TYPE boole-boole.

      METHODS xBuffer_read.

ENDCLASS.

CLASS file_buffered_reader IMPLEMENTATION.

   METHOD CONSTRUCTOR.

      CLEAR xBuffer.
      xBuffer_fill = 0.
      CLEAR xBufferReadAhead.
      CLEAR: eof.
      handle = i_filename.
      IF i_isOpen IS INITIAL.
         OPEN DATASET handle FOR INPUT IN BINARY MODE.
      ENDIF.

   ENDMETHOD.

   METHOD read.

      DATA: bytes_remaining    TYPE I,
            bytes_assigned     TYPE I.

      bytes_assigned  = 0.
      bytes_remaining = i_numBytes.

      WHILE bytes_remaining > 0.


         IF xBuffer_fill < RECORDSIZE.
            CALL METHOD xBuffer_read.
         ENDIF.

         DATA: max_or_buffer    TYPE I.
         IF bytes_remaining > xBuffer_fill.
            max_or_buffer = xBuffer_fill.
         ELSE.
            max_or_buffer = bytes_remaining.
         ENDIF.

         e_xRecord+bytes_assigned(max_or_buffer)
              = xBuffer+0(max_or_buffer).
         ADD max_or_buffer TO bytes_assigned.

         " Move buffer up
         DATA: buffer_remaining   TYPE I.
         buffer_remaining = xBuffer_fill - max_or_buffer.

         xBuffer+0(buffer_remaining)
                = xBuffer+max_or_buffer(buffer_remaining).
         xBuffer_fill = buffer_remaining.
         IF xBuffer_fill < 0.
            break-point.
         ENDIF.

         SUBTRACT bytes_assigned FROM bytes_remaining.

         " Check, if the record could be filled up
         IF eof = 'X'  AND bytes_remaining > xBuffer_fill.
            e_eof = 'X'.
            EXIT.
         ENDIF.


      ENDWHILE.


   ENDMETHOD.


   METHOD xBuffer_read.

      DATA: len   TYPE I.
      READ DATASET handle INTO xBufferReadAhead LENGTH len.
      IF sy-subrc <> 0.
         eof = 'X'.
      ENDIF.

      IF len > 0.
         xBuffer+xBuffer_fill(len) = xBufferReadAhead+0(len).
      ENDIF.
      ADD len TO xBuffer_fill.

   ENDMETHOD.

ENDCLASS.