topic Re: Uploading text file into internal table in Application Development and Automation Discussions

Uploading text file into internal table

Former Member — Thu, 28 Dec 2006 06:21:35 GMT

Hello All,

I am trying to upload a text file having 6 million records into internal table and it is giving short dump.

Any ways to read some 10,000 records from file and load into internal table and while reading next time it shouldread from 10001 th records to 20,000 records. Or any other way to handle this.

Re: Uploading text file into internal table

SantoshKallem — Thu, 28 Dec 2006 06:25:51 GMT

try it in background.

reward if useful

Re: Uploading text file into internal table

Former Member — Thu, 28 Dec 2006 06:32:04 GMT

i am trying in background only.

Re: Uploading text file into internal table

Former Member — Thu, 28 Dec 2006 07:26:21 GMT

Try to upload this file to the application server and then read it into an internal table using the OPEN DATASET and READ DATASET commands.

Regards,

Manoj

Re: Uploading text file into internal table

Former Member — Thu, 28 Dec 2006 08:19:31 GMT

I have written this program to split the file into 10,000 records and update Ztable. The file is already in the application server only and i have used open dataset only. The problem i am getting is while reading the file from the app server after some time it is giving short dump because of the file size and heavy usage of memory.

Re: Uploading text file into internal table

Former Member — Thu, 28 Dec 2006 08:36:12 GMT

the sap default database is oracle.in oracle we can write..

select * from <table> where rownum <1001.

then u can read first 1000 records.

just check it in sap.

it may or may not have.

Re: Uploading text file into internal table

Former Member — Thu, 28 Dec 2006 09:23:34 GMT

Try splitting the file into six files of one million entries each. Then try uploading them using GUI_UPLOAD separately. This will decrease the memory consumption.

Re: Uploading text file into internal table

Former Member — Thu, 28 Dec 2006 10:20:48 GMT

try declaring internal table with occurs 1000 clause.

Re: Uploading text file into internal table

Former Member — Thu, 28 Dec 2006 10:49:09 GMT

Reply by Ali:

Hi Kumar,

As you told you are uploading the records in to a Ztable, then you can try processing of your records using SELECT statement.

The following is the SELECT statement using 'Pacakge Size' .

Package size can be used if you for example only want to finish processing a limited amount of data at a time due to lack of memory.

<b> SELECT <field1> <field2 <...>

FROM <table1>

INTO TABLE <itab> PACKAGE SIZE 50</b>

Best Regards

Ali S

Kumaraswamy Subramanian

Posts: 14

Registered: 1/12/06

Re: Uploading text file into internal table

Former Member — Mon, 01 Jan 2007 06:02:22 GMT

Read back and reengineer the process. You don't want to read the whole file to memory.

What you want to do is create a control table that will hold the file location and the current restart/recovery point...

You open the file and read it up to the restart recovery point. You then read no more than 100,000 records into memory and write them to the DB. There you commit. DO not forget to update your restart recovery point in the transaction writing the 100,000 records to the DB.

This way you can treat files of a very large size.

Have fun...