Uploading text file into internal table

[Former Member]

Hello All,

I am trying to upload a text file having 6 million records into internal table and it is giving short dump.

Any ways to read some 10,000 records from file and load into internal table and while reading next time it shouldread from 10001 th records to 20,000 records. Or any other way to handle this.

[Former Member]

Hello All,

I am trying to upload a text file having 6 million records into internal table and it is giving short dump.

Any ways to read some 10,000 records from file and load into internal table and while reading next time it shouldread from 10001 th records to 20,000 records. Or any other way to handle this.

[SantoshKallem]

try it in background.

reward if useful

[Former Member]

i am trying in background only.

[Former Member]

Try to upload this file to the application server and then read it into an internal table using the OPEN DATASET and READ DATASET commands.

Regards,

Manoj

[Former Member]

I have written this program to split the file into 10,000 records and update Ztable. The file is already in the application server only and i have used open dataset only. The problem i am getting is while reading the file from the app server after some time it is giving short dump because of the file size and heavy usage of memory.

[Former Member]

Reply by Ali:

Hi Kumar,

As you told you are uploading the records in to a Ztable, then you can try processing of your records using SELECT statement.

The following is the SELECT statement using 'Pacakge Size' .

Package size can be used if you for example only want to finish processing a limited amount of data at a time due to lack of memory.

<b> SELECT <field1> <field2 <...>

FROM <table1>

INTO TABLE <itab> PACKAGE SIZE 50</b>

Best Regards

Ali S

Kumaraswamy Subramanian

Posts: 14

Registered: 1/12/06

I have written this program to split the file into 10,000 records and update Ztable. The file is already in the application server only and i have used open dataset only. The problem i am getting is while reading the file from the app server after some time it is giving short dump because of the file size and heavy usage of memory.

[Former Member]

Read back and reengineer the process. You don't want to read the whole file to memory.

What you want to do is create a control table that will hold the file location and the current restart/recovery point...

You open the file and read it up to the restart recovery point. You then read no more than 100,000 records into memory and write them to the DB. There you commit. DO not forget to update your restart recovery point in the transaction writing the 100,000 records to the DB.

This way you can treat files of a very large size.

Have fun...

[Former Member]

the sap default database is oracle.in oracle we can write..

select * from <table> where rownum <1001.

then u can read first 1000 records.

just check it in sap.

it may or may not have.

[Former Member]

Try splitting the file into six files of one million entries each. Then try uploading them using GUI_UPLOAD separately. This will decrease the memory consumption.

[Former Member]

try declaring internal table with occurs 1000 clause.