ABAP looping forever

[Former Member]

<b>Brief explanation of problem:</b>

I have a report that processes a lot of data. The amount of data is dependent on what the user selects on the selection screen, but even for one entry in selection criteria, there could be thousands of records and millions of loops.

The problem, in the report, seems to be that when a user selects a range of entries (A* or A00 – DZZ), the program (scheduled in background) runs for a very long time. Typically it the execution time is less than 30-40 seconds for one entry, but even for 30 entries, the report only averages about 1,000 seconds.

When the report is selecting a large amount of data, it seems the report just keeps on running, sometimes well over 100,000 seconds. I’m not using any DO or WHILE loops, which can run infinitely, these are simply multiple nested (large) loops, up to 8 nested loops. I have also monitored the memory usage of the background work process; it seems the memory levels-out after a certain point (about 500MB for large amount of data) and doesn’t increase or decrease. I just cancel the report when it gets to that point and re-run it for smaller set of entries, and that works just fine.

<b>Now the questions:</b>

1) Can anyone diagnose the problem? Or am I just canceling the report too early?

2) Is there a limit, when reached, the ABAP program goes in some sort of infinite loop?

[Former Member]

<b>Brief explanation of problem:</b>

I have a report that processes a lot of data. The amount of data is dependent on what the user selects on the selection screen, but even for one entry in selection criteria, there could be thousands of records and millions of loops.

The problem, in the report, seems to be that when a user selects a range of entries (A* or A00 – DZZ), the program (scheduled in background) runs for a very long time. Typically it the execution time is less than 30-40 seconds for one entry, but even for 30 entries, the report only averages about 1,000 seconds.

When the report is selecting a large amount of data, it seems the report just keeps on running, sometimes well over 100,000 seconds. I’m not using any DO or WHILE loops, which can run infinitely, these are simply multiple nested (large) loops, up to 8 nested loops. I have also monitored the memory usage of the background work process; it seems the memory levels-out after a certain point (about 500MB for large amount of data) and doesn’t increase or decrease. I just cancel the report when it gets to that point and re-run it for smaller set of entries, and that works just fine.

<b>Now the questions:</b>

1) Can anyone diagnose the problem? Or am I just canceling the report too early?

2) Is there a limit, when reached, the ABAP program goes in some sort of infinite loop?

[christian_wohlfahrt]

Hi!

You're are looking for the explanation, why you get this step in runtime:

your memory usage jumps from 'everything in RAM' to 'swapped on harddrive'.

500MB seems to be your limit, there are other lines counting up the rest -> your runtime is more or less infinite when looping on harddisk instead of memory.

To improve performance:

- don't select information twice into different internal tables

- if you don't need some tables any longer, use FREE-command

- have a look, if you have declared your tables as SORTED (to get some key access in LOOP ... WHERE)

Regards,

Christian

[Former Member]

No limit that I have ever heard of that would put your program into an infinite loop.

Try writing some S messages from your program that will tell you where the program is at different times. These message will show up in the job log.

Try invoking the debugger while the program is running. You can do this when it reaches the point where you have been canceling it.

You can invoke the debugger several ways:

1. Execute SM37, select the job you want to debug,

enter the function code =JDBG in the command field

or

2. Go to SM50 / SM51 and select the job when it is running and in the menu Program -> Program -> debugging

Message was edited by: Charles Folwell

[Former Member]

Charles,

I inserted S messages in the program, but those messages don't appear until after the program is entirely executed. Is there any way to make them appear while the program is running?

[Former Member]

Hi,

As your program is running in the background, as Charles mentioned: <b>Try writing some S messages from your program that will tell you where the program is at different times. These message will show up in the job log.</b> To be able to see messages while the program is running, try to execute it in the foreground with a workable set of data to evaluate its performance and any other factors affecting its execution.

Regards

[thomas_jung]

I'm sure this is your nested loops. The loops will have to do full table reads if you are doing loop at where. Usually this isn't any problem. However when you start nesting loops within each other, this starts to show as a problem. Because you are in a loop on a memory structure, you don't see activity details in the process overview.

The best thing to do is recode your loops. Turn your tables into sorted tables. If your tables are sorted tables and your loop at where conditions batch the sorted criteria, the system will perform binary reads instead of full table reads. The other option is to resort the tables and do a read binary search instead of the loop at where. I have seen programs that were running for hours finish within seconds after this kind of change. If you need some code samples, just let me know.

[Former Member]

I'm working in telefonica(spain) .We process a large quantity of documents every day .We tuned all the reports ,etc because of perfomance and time.

You need to reduce the greater number of operations possibles .You need to reduce the conditions .You need to put the conditions more restricted first .You need to access the tables by primary key .If you simplified code you improve the process.

After this you can process information partially throwing jobs in parallel like in our project and then concatenate results

[ssimsekler]

Hi

There is not so much to add. It is obvious that some tuning measures should be taken. Some of the following will repeat the above recommendations. However let me write:

<b>1.</b> "BINARY SEARCH" addition may make great positive effect, however do not forget to sort your tables (The standard table must be sorted in ascending order by the specified search key).

<b>2.</b> Try giving your loop conditions with 'WHERE' statement instead of using 'CHECK'.

e.g.

LOOP AT gt_itab WHERE field_1 = 'A' .
...
ENDLOOP .

<u>is better than</u>

LOOP AT gt_itab .
CHECK gt_itab-field_1 = 'A' .
...
ENDLOOP .

<b>3.</b> If it is possible to specify keys, prefer using "READ TABLE...WITH KEY..." instead of "LOOP...ENDLOOP" to read single records.

<b>4.</b> Try using "WHILE...ENDWHILE" or "DO...ENDDO" if possible for looping. "WHILE...ENDWHILE" is better if you can prefer it.

<b>5.</b> Optimize your SELECT statements if any exists inside your loops.

__a. Prefer "SELECT SINGLE..." for reading single records and "SELECT...INTO TABLE..."(array fetch) to read multiple records.

__b. Try explicitly defining the fields you want to select. e.g. "SELECT field_1 field_2" is better than "SELECT * ".

__c. Eliminate redundant selection conditions.

__d. Generate secondary indices for the relevant tables if required.

__e. Hesitate from sophisticated selection conditions.

<b>6.</b> You can code some "WRITE" statements to trace the program execution. You can see the list output at debugging session by pressing the relevant button.

<b>7.</b> Use the "Runtime Analysis Tool" (SE30) to analyze your session for a reasonable set of data to see the relational execution time comparison between logical execution blocks of your program.

<b>8.</b> Try parallel processing. You can also consider exporting/importing generated lists.

Hope you can solve your problem...

*--Serdar

[email protected]

[Former Member]

Thanks guys for all your input.

My tables, all 8 of them, are sorted or hashed internal tables. On top of that, I am also using secondary indices on the sorted tables (which makes a lot of difference). I want to use parallel cursors, but since I just learned that trick I am trying to figure out where/how I can use them in my loops. From the looks of it, parallel cursors don’t seem to apply here.

Another thing I did was, combined 3 tables into one longer-linear table (the 1st table) instead of more nested loops. Also, right before I get into this massive loop, I refresh all tables except these 8 and few others that I need (these are small).

Charles, thanks I didn’t even know that was possible. I will try to debug and see where the loop goes in an infinite loop. Also will write S messages to display status in message log.

Jorge, how can I process data in parallel? Do you mean use two work processes at once? This is very interesting, can you please expand on how I can run two (or more) work process in parallel? I had heard about this, but never really knew how to do it.

Thanks again for the help!

Sumit

[Former Member]

Hi,

For parallel processing, take a look at the documentation for statement <b>CALL FUNCTION <function> STARTING NEW TASK <task name></b>.

You can additionally take a look at http://www.kabai.com/abaps/z83.htm for a sample program.

Regards

Message was edited by: Shehryar Khan

[Former Member]

dear Sumit Sangha

if neted loops are necessry then create index on those columns which are used in where clause. it is very useful for large amount of data.

Thanks

Rai Zeeshan Zaffar Khan.