Delta load in data service 4.0

[Former Member]

Hi,

We want to pull the data from ECC(6.0) to SQL Server 2008 using data services 4.0. We have used table comparison for the same. But the problem here is it takes lot of time to pull the data as it reads the data row by row.

Can anyone suggest how can we increase the performance of this job or any alternative for the same ?

Thanks in advance.

Regards,

Vidula

[Former Member · Accepted solution]

Hi ,

How you are extracting the data from SAP ECC? Extractors? IDocs?

I suggest to use a ABAP data flow, this is faster in extracitng the data.

- Naresh

[Former Member · Accepted solution]

Hi Naresh,

We are directly pulling data from tables using dataflow. Below is the process that we used. Here, in MARA there are around 90,000 records which takes almost 30min to finish the job. I cant understand where am I going wrong. It is 1st time I am using SAP as source. I have worked on SQL as source.

Regards,

Vidula

[Former Member · Accepted solution]

Hi,

Whats the Array fetch size on MARA table? Can you try ABAP dataflow? Its faster compare to this.

- Naresh

[Former Member · Accepted solution]

Hi Vidula,

ABAP Flow is fast and easy way of moving data from ECC to any databases, we have been using ABAP flows and it takes about 2 min to transfer about the 50K records.

Thanks,

Vinod

[Former Member · Accepted solution]

Hi Vidula,

It seems that your are using RFC interface to pull the data from SAP table. It is not the recommended way to pull the data from SAP systems. This process is good for pulling very less volume of data, not for bulk data.  You have to use ABAP interface to pull the data from SAP, means place the table in ABAP data flow, then use time stamp delta mechanism (i.e. pull data based on Material Changed data), which is the right process to pull changed data from SAP.

Thanks,

Ramakrishna Kamurthy

[Former Member · Accepted solution]

Hi All,

Thank you so much for all the replies.

I tried using ABAP dataflow but I am confused with what details should I enter in APAB generate file and program name. I have no knowledge about ABAP. Dont know how to use it and what to use in it.

Vidula

[Former Member · Accepted solution]

1. You can give a name as per your company naming standards like 'Z_XXXXXX'.  For all the name fields, you cn give the same name. No harm in that.

2. Then in that ABAP dataflow, drag the MARA table as source, then Query and drag the 'Data Transport' object. In that Transport object, you just need to mantion the filename.

**** Remember, this generates a ABAP program on SAP side, so when ever you want to migrate the code to other env's, you have to move that ABAP program also. So you have to coordinate with your SAP team for that transport.

After you are done with this ABAP dataflow....

3. Call this ABAP dataflow object in the Dataflow. Map the ABAP dataflow to Target table.

Thats it..you are done.....

I suggest to read the documentation to get more knowledge on this.

-Naresh

[Former Member · Accepted solution]

Hi Naresh,

Thank you for the detailed explanation.

The ABAP program that has been generated on SAP side, do I need to execute/schedule it on SAP side too or running just the ETL job would be enough?

-Vidula

[Former Member · Accepted solution]

ETl job will take care of calling that ABAP program. Remember my *** note while migrating the code.

[Former Member · Accepted solution]

Hi Naresh,

I used ABAP data flow as you explained. I get an error "Error opening file <C:/Program Files/SAP BusinessObjects/Data Services/ZW_ABAPDF>."

ZW_ABAPDF is the file name that i have used for ABAP program name and Generated ABAP file name.

-Vidula

[Former Member · Accepted solution]

Hi Vidula,

If you are using the ABAP data flow it should be working like this:

1) DataServices creates an abap code program with the filename/parameters you set in the workflow.
The code will contain the commands that will tell SAP to execute a dump of the table according to your data flow settings and put it into a file.(you can open the code and search for the filename you put. Even if you don't program ABAP (neither do I) you will see filename, separator character,path...

2) You load (and execute?) that ABAP program file onto the SAP system (various options, like load + execute, execute only ..)

3) The result is a .dat file that is nothing else than a text file containing the info you need.   In the DataStore you configured how DataServices will access that file ZW_ABAPDF.dat

Did you choose a shared directory on the SAP Server? an FTP transport from the SAP to the DS Server?

http://wiki.sdn.sap.com/wiki/display/EIM/Chosing+the+Transport+Method

Norbert

[Former Member]

Hi Vidula,

You may check this for more information.

http://wiki.sdn.sap.com/wiki/display/EIM/Material+Dimension+Delta+Load+with+SQL

Thanks,
Arun