Flat File Processing using Batch Job

[Former Member]

Hello,

I need some ABAP advice...

The scenario: There's a batch job that runs every hour and picks up all the files from APP server and processes them to creates idocs. Sometimes, while a batch job is running, a file is being created on the app server. But although the file is being written to and is not yet complete, the batch job picks up this file to create an idoc and hence leads to errors. Is there a way I can check if a file is already complete and then only the batch job should process the file.

I am using SAP 4.6c and it doesnt allow me to use GET DATASET attributes...

Any suggestions on how to check.....

[Former Member]

Hi,

Not sure about FM's to get dataset attributes. However you might try a workaround. For example create a little Z table with the filename and status, and make the batch job process to check if the status for the file is completed, if so, then process it. Otherwise, leave it.

Hope it helps.

Regards,

Gilberto Li

[Former Member]

Thank you for your reply.

The Flat files come from Power Vision to SAP randomly at any time, so if we create a Z table how do we keep a track of updating it ?

[Former Member]

At the moment there is only one workaround that is coming to my mind.

After you do READ dataset and transfer to the internal table.

WAIT FOR 5 seconds.

Again read the file with the same procedure.

Compare the two internal table using DESCRIBE TABLE 1 LINES and similarly LINES of table 2.

Now if there is no difference proceed. If there is a difference you could either ignore this run of batch job so that when the program runs next time definitely file would be complete OR i dont think writing the file from whatever source could be taking more than 10 seconds, so after another WAIT of 5 seconds read the file and process without checking anything.

Revert if u think this is not feasible.

[Former Member]

That sounds like a good idea but is there also a way I can check the timestamp of the file ? I mean instead of reading the whole file and then taking a count of lines twice in the program, if I could check for the timestamp of the file and then wait for 5 seconds and again check for timestamp of file. If the file timestamp remains the same that means the file is not being edited anymore, right?

Can you suggest something.

Thanks,

Shipra.

[Former Member]

Hi Shipra,

Check this code taken from another post to get the creation time of the file:

*&---------------------------------------------------------------------*
*& Report  ZFILE_CREATE_DATE
*&
*&---------------------------------------------------------------------*
*&
*&
*&---------------------------------------------------------------------*

REPORT  zfile_create_date.

TABLES epsf.

PARAMETERS dir
  LIKE epsf-epsdirnam DEFAULT 'directory here'.

PARAMETERS file LIKE epsf-epsfilnam DEFAULT 'file here'.

DATA : mtime TYPE p DECIMALS 0,
            time(10),
           date LIKE sy-datum.

CALL FUNCTION 'EPS_GET_FILE_ATTRIBUTES'
     EXPORTING
          file_name              = file
          dir_name               = dir
     IMPORTING
          file_size              = epsf-epsfilsiz
          file_owner             = epsf-epsfilown
          file_mode              = epsf-epsfilmod
          file_type              = epsf-epsfiltyp
          file_mtime             = mtime
     EXCEPTIONS
          read_directory_failed  = 1
          read_attributes_failed = 2
          OTHERS                 = 3.

IF sy-subrc NE 0.

  WRITE: / 'error:', sy-subrc.

ELSE.

*The subroutine p6_to_date_time_tz is sap std present in rstr0400.

  PERFORM p6_to_date_time_tz(rstr0400) USING mtime
                                             time
                                             date.
  WRITE: / 'mtime:',mtime.
  WRITE: / 'date: ', date.
  WRITE: /  'time:',time.
ENDIF.

Regards,

Gilberto Li

[Former Member]

Hi Shipra,

There could be one more way.

Whatever dataset name you get, before doing OPEN DATASET FOR input do an OPEN DATASET FOR APPENDING....

Now if the file cannot be opened for appending then sy-subrc = 4 else 0.

also i am pretty sure that while one source is writing to the file it wont allow second to open for appending... Hence the check u need...

Explore this possibility.... For testing you may OPEN DATASET FOR INPUT and then APPENDING in the same program...

Also if this thing works you could have this in DO loop to wait till the file is completely written to and proceed only afterwards..

Revert if you feel like.

Cheers !

[Former Member]

Sure I will try this out. Sounds like it should work

Thanks,

Regards,

Shipra.

[Former Member]

Hi Shipra,

All the best to you..I gotta be leaving..

If things do work then do update accordingly..

Also if you get time then you could explore RSPO_READ_DATASET to read the dataset directly.. Its just for your info.

Cheers!

[Former Member]

Hi Shipra,

I assumed that the file was generated by a program that you got control.

You can try using the FM EPS_GET_FILE_ATTRIBUTES. As Ankesh said, use the FM and check the File size. Wait for few seconds, and check again. If file size differ, don't process it.

This way you won't have to open the file and read the lines and compare them, just use the size attributes of the FM.

Hope it helps.

Regards,

Gilberto Li

Edited by: Gilberto Li on Jan 7, 2009 7:08 PM

[Former Member]

From the help on OPEN Dataset:

In Unicode programs, the file must not yet be open in the current program; otherwise a treatable exception occurs. In non-Unicode programs, the file may already be open. The statement OPEN DATASET then does not reopen the file but moves the read or write position depending on the access mode. In this case, you should not change the access or storage mode.

Rob

[Former Member]

Do you mean if I try to reopen an already open file in Append mode, it would give an error? What do you mean by a unicode / non unicode program...

please explain.

Thanks,

Shipra

[Former Member]

It was taken directly from the help - you probably should have a look at it.

Rob

[Former Member]

Shipra,

Can you make use of this FM, RZL_READ_DIR_LOCAL, this will give you list of the files and then you can do some more coding to resolve your probelm?

[Former Member]

Hi,

Try to tell the other system which write file....to first write in the other folder and then move or FTP to the SAP application server.

If this is not possible....have them write some End of File Character in the last line.

Then in the reading program when it completes reading the file.......if the last line matches with the End of File character....means file is complete....

[Former Member]

Thanks Ankesh & Gilberto, your replies helped alot. Thanks all for your help.