cancel
Showing results for 
Search instead for 
Did you mean: 

Uploading of large amount of data

Former Member
0 Kudos

Hi all,

i really hope you can help me. I have to upload quite large amount of data from flat files to ODS (via PSA of course). But the process takes very long time. I used method of loadin to PSA and then packet by packet into ODS. Loading of cca 1.300.000 lines from flat file takes about 6 or more hours. It seems strange for me. Is it normal or not?? Or should I use another uploading method or set up ODS some way ?? thanks

Accepted Solutions (1)

Accepted Solutions (1)

Former Member
0 Kudos

hi jj,

welcome to the SDN!

in my limited experience, 6hrs for 1.3M records is a bit too long. here are some things you could try and look into:

- load from the application server, not from the client computer (meaning, move your file to the server where BW is running, to minimize network traffic).

- check your transfer rules and any customer exits related to loading, as the smallest performance-inefficient bits of code can cause a lot of problems.

- check the size of data packets you're transmitting, as it could also cause problems, via tcode RSCUSTA2 (i think, but i'm not 100% sure).

hope ths helps you out - please remember to give out points as a way of saying thanks to those that help you out okay? 😃

ryan.

Former Member
0 Kudos

hi ryan,

i am already loading from files located on application server. and i have some routins in transfer structure.

So may be that it takes this time. the file with 1.3M of records is cca 250 GB large. I think it is not so much

and I think that the process is slower and slower. May be I should use another type uf ODS or I will try to set up

the packet sizes via RSCUSTA2.

jj

Former Member
0 Kudos

JJ,

i'm very, very alarmed at the size of your file; 1.3M records @ 250GB is VERY big. isn't there a way to limit your file to contain only the fields that are actually relevant for your ODS to begin with?

in experience, larger packet sizes help, but they will result in larger memory and space overheads.

ryan.

Former Member
0 Kudos

Hi Ryan,

unfortunatelly it is not possible to limit the file, because it necessary for migration of old datawarehouse into new SAP BW. So the file must be loaded complete. And there are even larger files

jj

Former Member
0 Kudos

Hi JJ,

Just wanted to add the following

1) Limit your flat file to smaller number of records initially (10,000) and then have you Basis/DBA built the database statistics, and then continue further with your remaining loads

2) try loading first into PSA and then update into ODS.

Hope it helps you somewhat,

Sree

Former Member
0 Kudos

Hi JJ,

Just to see where the culprit is. Why dont you try loading at least 10 line items only (produce a small file w/ only 10 line items in it). If it is still taking too long then file size is not the problem. I think its something else.

Regards,

--Jkyle

Former Member
0 Kudos

sorry wrong thread

Message was edited by: Mary Huang

Message was edited by: Mary Huang

Answers (0)