topic Re: Parallel Processing in Application Development and Automation Discussions

Parallel Processing

Former Member — Fri, 11 Aug 2006 15:16:29 GMT

Hi, Does anyone impletementation of Parallel processing? Is the statement 'WAIT UNTIL rec_jobs >= snd_jobs working? We started to implement this 2 month ago, it was working fine, but recently it looks this statement not working properly. Anybody have this experience before. Do System configuration or others factors affect this statemen? Very much apprecite for your input. Thanks.

Re: Parallel Processing

Former Member — Fri, 11 Aug 2006 19:05:25 GMT

Hi,

Your question is not clear.

Parallel processing of what? Detail your process please.

Raj

Re: Parallel Processing

Former Member — Fri, 11 Aug 2006 19:08:33 GMT

Hi Furong

I have worked on parallel processing. Depending on the number of backgroung processes set by basis adminstrators the number of jobs will be scheduled in background.

Supposing you have 10 jobs to execute in parallel and 4 background processes, the jobs will execute in sequence. If you are like submitting some other program internally within a program that will take precedence to the next job in sequence.

Do revert back if you need more info.

Kind Regards

Eswar

Re: Parallel Processing

Manohar2u — Fri, 11 Aug 2006 19:15:48 GMT

Refer this section of prerequisites

"SAP system resources: In order to process tasks from parallel jobs, a server in your SAP system must have at least 3 dialog work processes. It must also meet the workload criteria of the parallel processing system: Dispatcher queue less than 10% full, at least one dialog work process free for processing tasks from the parallel job."

Refer this link

http://help.sap.com/saphelp_nw2004s/helpdata/en/fa/096e92543b11d1898e0000e8322d00/frameset.htm

Regds

Manohar

Re: Parallel Processing

Former Member — Fri, 11 Aug 2006 19:35:06 GMT

Hi Eswar,

Thank you very much for your reply.

We have more than 1000 jobs and have about 200 processes. WE define 4 servers in a group.

In my program, it looks fine to submit these jobs (by loop and endloop. After submit the jobs, i use WAIT UNTIL rec_jobs > = send_jobs to wait all jobs completed, afetr get all data from these 1000 jobs, program cintinues to process data. It worked good for 2 months. But recently rec_jobs > = send_jo did not satisfy. what is possible reasons?

1) some process got lost due to some other heavy jobs (other parallel programs) are running at the same time? system issue?

2) WAIT UNTIL rec_jobs >= send_jobs is not working properly? if this is the case, any SAP notes can help? i did some research, but did not find out any info.

thanks again.

Re: Parallel Processing

Former Member — Fri, 11 Aug 2006 19:50:09 GMT

Hi Furong

Can that happen due to the load balancing in your system. As your system is having 4 servers in a group and if jobs are going to different servers, will that effect your condition rec_jobs >= send_jobs.

Itz just a wild guess, am not sure of how the system reacts for this scenario.

Kind Regards

Eswar

Re: Parallel Processing

Former Member — Fri, 11 Aug 2006 20:05:28 GMT

Hi Eswar,

Thanks. I fully agree with you, I do not think there is no bug for WAIT UNTIL statemnet. but our basis team insists that WAIT UNTIL statement is not working properly. As an ABAP developer, what should i do to find out this solution? We have requested SAP, not yet get feed back.

Thanks.

Re: Parallel Processing

Former Member — Fri, 11 Aug 2006 20:12:33 GMT

Hi Furong

I am not on SAP System now, so can not advice on the exact solution. But i think we have an option to forcifully execute the job on a particular server. Please try to explore on the same. Similarly ask your BASIS adminstrator if itz possible to make one server out of 4 specific for background jobs so whenever a job is triggered it will go to a specific server.

Kind Regards

Eswar

Re: Parallel Processing

sridhar_k1 — Fri, 11 Aug 2006 20:24:18 GMT

You've mentioned "rec_jobs > = send_jo did not satisfy" so your original program is waiting forever?

One possible reason for that condition being not true is, some of the parallel tasks might be timing out and not executing the RECEIVE subroutine. Check ST22 for timeouts.

Regards

Sridhar

Re: Parallel Processing

Former Member — Fri, 11 Aug 2006 20:25:38 GMT

Hi,

Thanks a lot. I will ask Basis to specific one or two server to run one program. Currently 3 parallel programs are runing at same time. first program is running on 5 servers (we have 5 servers totally), second program program is running in 4 server, and last program use same as second one.

Re: Parallel Processing

Former Member — Fri, 11 Aug 2006 20:30:59 GMT

Hi Sridhar,

Thank you for your reply. I do think so, but there is no dump when i check ST22.

Re: Parallel Processing

sridhar_k1 — Fri, 11 Aug 2006 20:38:39 GMT

Did you check dumps in all appservers? I think ST22 by default shown dumps in the appserver you loged into.

Regards

Sridhar

Re: Parallel Processing

Former Member — Fri, 11 Aug 2006 20:41:03 GMT

Hi Furong

Can you tell us whatz the requirement for waiting until all the 1000 jobs are finished? Maybe if we try to get that way, someone here can propose a better solution if any for your case.

Kind Regards

Eswar

Re: Parallel Processing

Former Member — Fri, 11 Aug 2006 20:44:45 GMT

Dear Sridhar

I guess what your assumption of encountering a short dump for TIME OUT might be right if the initiating program is executed online and the called programs take huge amount of time for execution but that does not happen if the initiating program is executed in a background.

Please correct me if i am wrong.

Kind Regards

Eswar

Re: Parallel Processing

Former Member — Fri, 11 Aug 2006 20:53:58 GMT

Hi,

I checked all the server, there is no dump. thanks. Is it possible in system log?

Re: Parallel Processing

Former Member — Fri, 11 Aug 2006 20:55:09 GMT

hi,

all parallel programs are running in backgroud.

Re: Parallel Processing

Former Member — Fri, 11 Aug 2006 21:06:22 GMT


Dear Furong

   As you are saying that you have to wait until all the 
jobs are finished, how about creating one single job 
with all the programs to be executed as steps and then 
doing the later processing as a last step in the job though i agree that this will effect your concept of parallel processing.

  For this do the following steps: 
1. Use FM: JOB_OPEN to open a new job reterive the job 
number with Proposed Jobname
2. submit the programs(1000).
3. Use FM: JOB_CLOSE with Job Name and Job Number from 
FM JOB_OPEN.

Note: You have to set 'X' to start immediately in export 
parameters of FM:JOB_CLOSE.

Kind Regards
Eswar

Re: Parallel Processing

sridhar_k1 — Fri, 11 Aug 2006 22:40:50 GMT

Do you have these exceptions in your ARFC call:

system_failure

communication_failure

resource_failure.

How are you incrementing snd_jobs variable? is it immediately after the arfc call or after checking sy-subrc for above exceptions?

Regards

Sridhar

Re: Parallel Processing

Former Member — Mon, 14 Aug 2006 12:38:08 GMT

Hi,

The code is:

LOOP AT it_sum.

it_sum_temp = it_sum.

DO.

CALL FUNCTION 'ZPP_CHECK_CONDITION'

STARTING NEW TASK taskname

DESTINATION IN GROUP p_srvgrp

PERFORMING task_receive ON END OF TASK

TABLES

p_pharse = it_pharse

p_sum = it_sum_temp

EXCEPTIONS

system_failure = 1

communication_failure = 2

RESOURCE_FAILURE = 3.

CASE sy-subrc.

WHEN 0.

taskname = taskname + 1.

snd_jobs = snd_jobs + 1.

CLEAR: excep_flag.

EXIT.

WHEN 1 OR 2.

excep_flag = 'X'.

WHEN 3.

IF excep_flag = space.

excep_flag = 'X'.

WAIT UNTIL rcv_jobs >= snd_jobs UP TO '0.1' SECONDS.

ELSE.

WAIT UNTIL rcv_jobs >= snd_jobs UP TO '1.0' SECONDS.

ENDIF.

IF sy-subrc EQ 0.

CLEAR excep_flag.

ENDIF.

ENDCASE.

ENDDO.

CLEAR: it_sum_temp, it_sum_temp[].

ENDLOOP.

WAIT UNTIL rcv_jobs >= snd_jobs.

Re: Parallel Processing

sridhar_k1 — Mon, 14 Aug 2006 14:24:02 GMT

One possible issue with the do loop in the code is, if one of the exception is raised every time because of unavailable work process, the program keeps on looping and never reach the last WAIT statement, this might be happening in your case. To test, exit the do loop after some time if it's looping for the same it_sum record.

This is just FYI:

If resource_failed exception raised when rcv_jobs = snd_jobs (first loop pass or after some loops) the WAIT in the if condition does not wait the specefied amount of time.

Regards

Sridhar