SAP DI SLT Parallel

[former_member866218]

We are using SAP Data Intelligence Gen1 Operators to replicate SAP data to AWS. The goal is to transfer 250 Million data sets every day. The SLT Connector V2 is being utilized to access the SAP system from DI. Unfortunately, the connector is very slow, taking approximately 5 minutes for a batch of 500,000 records to arrive in DI. We also attempted to use the faster SLT Connector V1, which only took about 2 minutes. Additionally, I discovered that parallelizing the calculation jobs for replication or initial load is possible. I tried implementing it, but it caused DI to fail. I received the following error message: 'Failed to start the graph. Failed to initiate process sltconnector1. An exception was raised. More than one job is defined for table ..., mass transfer ID 001; see long text. A fatal error has been encountered, leading to the termination of the graph.' I followed the steps outlined in sections 3.1 to 3.4 of the blog post for parallelization (https://blogs.sap.com/2021/01/03/parallelization-of-access-plan-calculation-job-in-slt/).

Is it normal for the SLT Connectors to be slow? Is it possible to start a parallelized connector from DI? What needs to be configured for parallelization to work? Thank you

[jimgiffin]

The error you get is because in SLT the table name is already registered to another graph for that MTID 001. Graphs are always single threaded so it does the calculation job then sends the portions to DI in serial, which can make the extraction seem slower.

The most efficient way to extract data is via Replications (RMS) in DI. It does parallel extraction from the source.