Exporting and importing huge data through CSV files

[sanjay_ram]

Hi,

Can ABAP program be used to export 10GB worth of data and also import it into another table? Will it crash or complete running? Is there any limits to this?

[sanjay_ram]

Hi,

Can ABAP program be used to export 10GB worth of data and also import it into another table? Will it crash or complete running? Is there any limits to this?

[matt]

Just the usuals. Memory, performance and filesize.

[tom_wan]

Hi sanjay.ram

When you read the 10GB file, please try with OPEN DATASET, GUI_UPLOAD should be impossible due to the large file size:

SAP note 872457 -Gui_Upload: Restricting the file size.

As mentioned in this note, Unlike Ws_Download, the Gui_Download supports code pages including Unicode. Due to the conversion that occurs during the processing, the main memory requirement during the upload can be two to three times the size of the uploaded file. In this case, the maximum main memory size of the work process can be exceeded and this terminates the transfer. Basically, the front-end file modules are not intended for transferring large volumes of data. Instead, large files should be loaded directly from the application server by using OPEN DATASET. If the files can only be read by using gui_upload, you need to adjust the memory space restriction of the work process correspondingly.

When you process the big data in ABAP program, if memory dump happens, contact your basis to adjust the memory parameters.

Regards

Tom

[former_member660513]

Hi, any machine(not only servers) in the world has physical limits, right?

by 10 GB worth of data you mean

-> data size, as compressed on the DB is 10 GB

-> after the export in let's say CSV file, that text file's size is 10GB?

Do you need to transfer that data between two different (ABAP/DB) instances or you need to move it from table A to table B in the same DB schema and instance?

If it is just moving from A to B within the same DB instance, you should be able to use packages/ cursors, and so on, right?

Then it should not matter if it is 10 GB...