Parallel Processing

[Former Member]

Hi, Does anyone impletementation of Parallel processing? Is the statement 'WAIT UNTIL rec_jobs >= snd_jobs working? We started to implement this 2 month ago, it was working fine, but recently it looks this statement not working properly. Anybody have this experience before. Do System configuration or others factors affect this statemen? Very much apprecite for your input. Thanks.

[Former Member]

Hi,

Your question is not clear.

Parallel processing of what? Detail your process please.

Raj

[Former Member]

Hi Furong

I have worked on parallel processing. Depending on the number of backgroung processes set by basis adminstrators the number of jobs will be scheduled in background.

Supposing you have 10 jobs to execute in parallel and 4 background processes, the jobs will execute in sequence. If you are like submitting some other program internally within a program that will take precedence to the next job in sequence.

Do revert back if you need more info.

Kind Regards

Eswar

[Former Member]

Hi Eswar,

Thank you very much for your reply.

We have more than 1000 jobs and have about 200 processes. WE define 4 servers in a group.

In my program, it looks fine to submit these jobs (by loop and endloop. After submit the jobs, i use WAIT UNTIL rec_jobs > = send_jobs to wait all jobs completed, afetr get all data from these 1000 jobs, program cintinues to process data. It worked good for 2 months. But recently rec_jobs > = send_jo did not satisfy. what is possible reasons?

1) some process got lost due to some other heavy jobs (other parallel programs) are running at the same time? system issue?

2) WAIT UNTIL rec_jobs >= send_jobs is not working properly? if this is the case, any SAP notes can help? i did some research, but did not find out any info.

thanks again.

[Former Member]

Hi Furong

Can that happen due to the load balancing in your system. As your system is having 4 servers in a group and if jobs are going to different servers, will that effect your condition rec_jobs >= send_jobs.

Itz just a wild guess, am not sure of how the system reacts for this scenario.

Kind Regards

Eswar

[Former Member]

Hi Eswar,

Thanks. I fully agree with you, I do not think there is no bug for WAIT UNTIL statemnet. but our basis team insists that WAIT UNTIL statement is not working properly. As an ABAP developer, what should i do to find out this solution? We have requested SAP, not yet get feed back.

Thanks.

[Former Member]

Hi Furong

I am not on SAP System now, so can not advice on the exact solution. But i think we have an option to forcifully execute the job on a particular server. Please try to explore on the same. Similarly ask your BASIS adminstrator if itz possible to make one server out of 4 specific for background jobs so whenever a job is triggered it will go to a specific server.

Kind Regards

Eswar

[sridhar_k1]

You've mentioned "rec_jobs > = send_jo did not satisfy" so your original program is waiting forever?

One possible reason for that condition being not true is, some of the parallel tasks might be timing out and not executing the RECEIVE subroutine. Check ST22 for timeouts.

Regards

Sridhar

[Former Member]

Hi,

Thanks a lot. I will ask Basis to specific one or two server to run one program. Currently 3 parallel programs are runing at same time. first program is running on 5 servers (we have 5 servers totally), second program program is running in 4 server, and last program use same as second one.

[Former Member]

Hi Sridhar,

Thank you for your reply. I do think so, but there is no dump when i check ST22.

[sridhar_k1]

Did you check dumps in all appservers? I think ST22 by default shown dumps in the appserver you loged into.

Regards

Sridhar

[Former Member]

Hi Furong

Can you tell us whatz the requirement for waiting until all the 1000 jobs are finished? Maybe if we try to get that way, someone here can propose a better solution if any for your case.

Kind Regards

Eswar

[Former Member]

Dear Sridhar

I guess what your assumption of encountering a short dump for TIME OUT might be right if the initiating program is executed online and the called programs take huge amount of time for execution but that does not happen if the initiating program is executed in a background.

Please correct me if i am wrong.

Kind Regards

Eswar

[Former Member]

Hi,

I checked all the server, there is no dump. thanks. Is it possible in system log?

[Former Member]

hi,

all parallel programs are running in backgroud.

[Former Member]

Dear Furong

   As you are saying that you have to wait until all the 
jobs are finished, how about creating one single job 
with all the programs to be executed as steps and then 
doing the later processing as a last step in the job though i agree that this will effect your concept of parallel processing.

  For this do the following steps: 
1. Use FM: JOB_OPEN to open a new job reterive the job 
number with Proposed Jobname
2. submit the programs(1000).
3. Use FM: JOB_CLOSE with Job Name and Job Number from 
FM JOB_OPEN.

Note: You have to set 'X' to start immediately in export 
parameters of FM:JOB_CLOSE.

Kind Regards
Eswar

[sridhar_k1]

Do you have these exceptions in your ARFC call:

system_failure

communication_failure

resource_failure.

How are you incrementing snd_jobs variable? is it immediately after the arfc call or after checking sy-subrc for above exceptions?

Regards

Sridhar

[Former Member]

Hi,

The code is:

LOOP AT it_sum.

it_sum_temp = it_sum.

DO.

CALL FUNCTION 'ZPP_CHECK_CONDITION'

STARTING NEW TASK taskname

DESTINATION IN GROUP p_srvgrp

PERFORMING task_receive ON END OF TASK

TABLES

p_pharse = it_pharse

p_sum = it_sum_temp

EXCEPTIONS

system_failure = 1

communication_failure = 2

RESOURCE_FAILURE = 3.

CASE sy-subrc.

WHEN 0.

taskname = taskname + 1.

snd_jobs = snd_jobs + 1.

CLEAR: excep_flag.

EXIT.

WHEN 1 OR 2.

excep_flag = 'X'.

WHEN 3.

IF excep_flag = space.

excep_flag = 'X'.

WAIT UNTIL rcv_jobs >= snd_jobs UP TO '0.1' SECONDS.

ELSE.

WAIT UNTIL rcv_jobs >= snd_jobs UP TO '1.0' SECONDS.

ENDIF.

IF sy-subrc EQ 0.

CLEAR excep_flag.

ENDIF.

ENDCASE.

ENDDO.

CLEAR: it_sum_temp, it_sum_temp[].

ENDLOOP.

WAIT UNTIL rcv_jobs >= snd_jobs.

[sridhar_k1]

One possible issue with the do loop in the code is, if one of the exception is raised every time because of unavailable work process, the program keeps on looping and never reach the last WAIT statement, this might be happening in your case. To test, exit the do loop after some time if it's looping for the same it_sum record.

This is just FYI:

If resource_failed exception raised when rcv_jobs = snd_jobs (first loop pass or after some loops) the WAIT in the if condition does not wait the specefied amount of time.

Regards

Sridhar

[Former Member]

Hi Sridhar,

Thank you very much for your valuable information.

You are right. it may never come to last wait rcv_jobs = snd_jobs.

Please give me some more detail explaination for your send paragragh. i am not really geeting your point. Thanks again.

***********

If resource_failed exception raised when rcv_jobs = snd_jobs (first loop pass or after some loops) the WAIT in the if condition does not wait the specefied amount of time

[sridhar_k1]

What I meant was:

statement WAIT UNTIL rcv_jobs >= snd_jobs UP TO '0.1' SECONDS.

does not wait for 0.1 seconds when rcv_jobs = snd_jobs.

rcv_jobs and snd_jobs variables are equal during first loop pass and at any time when all submitted jobs finish in the loop.

Regards

Sridhar

[Former Member]

Hi,

Thanks for your patient to explain to me.

i agree with you the code can casue deadlock, so the last wait statement never reach. But if this is the case, the program is running forever, or system will abort it with error? So far in production system it looks sucessfully finished everytime, but the data is not right sometimes.

i will change program (as you recommended) to test the loop issue.

[sridhar_k1]

You can try executing fm ZPP_CHECK_CONDITION without STARTING NEW TASK when resource_failure exception raised by the arfc call several times in the do loop and exit the do loop.

Regards

Sridhar

[Manohar2u]

Refer this section of prerequisites

"SAP system resources: In order to process tasks from parallel jobs, a server in your SAP system must have at least 3 dialog work processes. It must also meet the workload criteria of the parallel processing system: Dispatcher queue less than 10% full, at least one dialog work process free for processing tasks from the parallel job."

Refer this link

http://help.sap.com/saphelp_nw2004s/helpdata/en/fa/096e92543b11d1898e0000e8322d00/frameset.htm

Regds

Manohar