CRM Middleware performance Optimization for Initial Load

[#1]

Hi All,

We are performing our dry run for data migration now. The customer data load from crm to r/3 is currently happening at 32000 records per hour. We see this is a very low throughput. I have made the following settings in Middleware:

1. Parallelization of queues. We are currently running about 10 queues at a time

2. Increased the block size of BUPA_MAIN

3. Increased the number of work processes to 38. We are currently running only middleware jobs in the system. No other users are using the system

I have looked at OSS notes for Performance improvement. Most of them speak about these 3 settings. Is there anything else we can do to improve the performance?

Also, as of now, the system is using one WP per queue. Like if a queue has 6000 entries it, the middleware is using only one work process to process all the 6000 entries. With the parallelization of queues, we have increased the number of queues running at a time, But still each queue is using only one WP ( 5 queues = 5 WP). Is there a way to make one queue use more than one WP?

Any help would be higly appreciated.

Thanks and Regards,

Shwetha

[Former Member]

In previous projects I would have been happy with that performance, but apparently you need more throughput. The download itself features the following restrictions which also limit performance:

- SAP System power (of course)

- DB Power (to a lesser extent)

- As CRM always works on the FIFO-principle on queues, there can only be one work process per queue. But as you did you can increase the number of processed queues in parallel.

- Number of work processes - although this one only speeds up if processes are scarce and all are limited

- As every single BP is validated against the CRM system it is important to keep the validation slim and performant.

Esp. the DB performance and the last point is the one next to queue parallelization which is increasing performance considerably. So

- go through the DB / SQL log and check which tables are referenced and loaded from the DB - these you should load into the buffer instead.

- Disable any not needed checks during download (e.g. duplicate check addons or self-programmed ones)

- Also check that the DB index on the partner tables does not work against the inserts - sometimes a rebuid of indexes here helps alot.

Regards,

Kai