topic process an internal table with huge data volume in Application Development and Automation Discussions

process an internal table with huge data volume

raffinkira — Fri, 09 Jan 2015 15:34:45 GMT

Implementing Parallel Processing - Background Processing - SAP Library

I have been gone through the above document about parallel processing.

My problem may be a typical one. For example， in my report, the structure of which is:

PERFORM get_data."get data from DB

LOOP AT internal_table.

PERFORM process_data."process every line of internal table

ENDLOOP.

PERFORM output_data."download the internal table as a txt file

How could I use parallel tech in my report?

Re: process an internal table with huge data volume

rajkumarnarasimman — Fri, 09 Jan 2015 16:16:08 GMT

Hi Ming,

Can you please share your full coding(statements inside subroutine), that help us to know more about the problem.

Regards

Rajkumar Narasimman.

Re: process an internal table with huge data volume

Former Member — Fri, 09 Jan 2015 16:26:56 GMT

Hi Ming Yu,

Can you please give détails about the "process of data"?

Just calculations/transformations of data or any call of transaction/function?..

Cheers

Thomas

Re: process an internal table with huge data volume

rajkumarnarasimman — Fri, 09 Jan 2015 16:41:51 GMT

This message was moderated.

Re: process an internal table with huge data volume

raffinkira — Sat, 10 Jan 2015 08:24:22 GMT

Hi,

perform of processing data is like:

LOOP AT lt_ab1 INTO ls_ab1.

ls_ab2-xxx = ls_ab1-xxx.

APPEND ls_ab2 TO lt_ab2.

ENDLOOP.

This is a sample, the real logic is much more complicated.

My problem is if lt_ab1 is large, can I use aRFC tech to reduce the time consumption?

Re: process an internal table with huge data volume

raffinkira — Sat, 10 Jan 2015 08:25:29 GMT

Hi, please refer to the above, thanks.

Re: process an internal table with huge data volume

rajkumarnarasimman — Sat, 10 Jan 2015 09:13:46 GMT

Hi Ming,


Ming Yu wrote:

My problem is if lt_ab1 is large, can I use a RFC tech to reduce the time consumption?

You are expecting for performance in the program, in order to do that, use runtime analysis tool(SM30) that helps to find at where it is taking more time.

Parallel Cursor method increases the performance during nested loop. It is based on index search. Please find the below link for Parallel cursor .

ABAP Code for Parallel Cursor - Loop Processing - Code Gallery - SCN Wiki

Use subroutines instead of RFC FM inside loop, that helps to increase the performance.

Before using Internal table, sort the internal table based on key. Use Binary search in READ Statement, that helps to increase the performance.

Regards

Rajkumar Narasimman

Re: process an internal table with huge data volume

Former Member — Tue, 13 Jan 2015 09:27:54 GMT

Hi Ming,

Yes, you can implement parallel processing of RFC, dividing your internal table in different groups and then calling parallel RFC, like that:

http://help.sap.com/erp2005_ehp_03/helpdata/EN/fa/096e92543b11d1898e0000e8322d00/content.htm

The problem is:

If you "simply" need to populate another internal table, you will have to wait to receive the different results and then you'll have to collect all of them in 1 unique table, so I'm not sure it will improve so much your performances.

Maybe it would be better first to rework your code for example with field symbols, mass moves (lt_ab2[] = lt_ab1[])...

Regards,

Thomas