Open Dataset Query

[Former Member]

Hi Experts,

I am just curious, is there any time limitation  as in up to what time limit the file can remain open.  The development scenario is that I am opening a large file using OPEN DATASET command , then the file contents are read using READ DATASET command. As soon a first x data is read from the file, parallel processing of the records starts(processing includes database commit and rollback) . After the x records are processed, I go for next x records, read and processing. The process is suppose to run for 12-15 hours.

I understand opening the file for a long time can restrict the access of the file for other programs, but is there any chance I can loose data from the file due to long run time. My goal is to reduce the memory intake by processing the file data in packets, rather than reading all the file data into an internal table and start processing.

Please advice.

[Former Member]

Since you opened your file for reading (I suppose) I think the data is not put at risk. However in my opinion it would be neater to open/close for each processing step if it takes that much time.

Have you considered to saving the position and next time use OPEN DATASET with addition AT POSITION?

[Former Member]

Hi Gabor,

Yes, I have considered AT POSITION but just wanted to know if long run time could lead to data loss.

[Former Member]

Hi Ritesh,

From my experience i have not run into a timeout scenario with OPEN DATASET in long running programs or data loss. I think what your wanting todo is possible and you should not have any issues with that as long as you can guarentee that no other programs will attempt to change the file your working on until your finished.

Now having said that i agree with Gabor that it would be more efficient to use the AT POSITION code and do a open/close data set as this will ensure your resources stay free.You could even look into doing parallel processing with this method if you wanted to help speed up processing time.

Geoffery

[Former Member]

Hi Geoffery,

Thanks for the input.

As for the AT POSITION syntax, SAP recommends to use this when we are opening the file in binary mode, but in my case I am opening the file in text mode. So, I am in a fix whether to use it or not.

[Former Member]

Hi Ritesh,

Do you have a specific requirement to do this in text mode? Because it seems like what you are wanting to do would be best done using the default mode in sap which is binary.

Geoffery

[Former Member]

Hi Geoffery, its just that I am enhancing an existing logic and the file contents are being read in text mode.

[Jelena_Perfiljeva]

I would suggest to run this scenario by your Basis team. "I will use a file to avoid using memory" sounds like something we would do in the good old mainframe days when hardware resources were at premium. In these days of in-memory databases such approach seems questionable, to say the least.

As for data loss - yes, it is, of course, possible if a program gets interrupted unexpectedly. Just like if anything was in memory and not in the database it would be lost as well. Files do not have any magical properties.

[Former Member]

Thanks All. Appreciate your inputs.