topic Re: Parallel processing question in Application Development and Automation Discussions

Parallel processing question

Former Member — Wed, 25 Oct 2006 14:27:01 GMT

I have written a module to read data from an external source, convert input to IDOCS, and process these IDOCS.

Due to performance of IDOC processing, I decided to use parallel processing to spread the IDOC creation load among several processes using server groups.

My main program sets up the parallel processing environment with a call to function module SPBT_INITIALIZE, then make repeated calls to a function module that processes the IDOCS, using the STARTING NEW TASK and DESTINATION IN GROUP xxx keywords.

What I'm seeing is the main process running as a background task and one dialog process running the parallel function module, even though there are still 8 or 9 other dialog processes available.

What I expected to see was several dialog process running my child function modules, not just the one.

Does anyone know why the other dialog processes are not being used?

Thanks for any input,

Dorian.

Re: Parallel processing question

sridhar_k1 — Wed, 25 Oct 2006 16:20:36 GMT

Do you have Multiple app servers in the group? then the parallel taskes might be executing on other app servers.

Go to SM51 and double click on each appserver, check the work process.

Did you use only one group name after DESTINATION IN GROUP in the main program. Runtime environment does not support multiple group names in the same program.

Add the following exceptions to the fm call:

EXCEPTIONS

system_failure = 1 MESSAGE lv_msg

communication_failure = 2 MESSAGE lv_msg.

resource_failure = 3.

where lv_msg is of type char100.

Check sy-subrc and lv_msg to check if the fm call is raising any resource failure error.

Regards

Sridhar

Re: Parallel processing question

Former Member — Wed, 25 Oct 2006 17:42:41 GMT

check the logic in RBDAPP01 which is used by SAP to post IDOCs. It has options for parallel processing, including limitation to server group, so you can use a similar logic in your code.

Re: Parallel processing question

Dorian — Wed, 25 Oct 2006 18:05:28 GMT

Sridhar:

There are no errors - I have coded these exceptions and they never get called. The child task is running, but only one. I expected to see as many child tasks as there are open dialog processes (less the number of tasks defined in RZ12 as "spare").

Re: Parallel processing question

Former Member — Mon, 30 Oct 2006 16:42:26 GMT

When you make repeated calls to a function module that processes the IDOCS ( using the STARTING NEW TASK and DESTINATION IN GROUP xxx keywords ) check whether your passing new task id everytime ?

Re: Parallel processing question

Former Member — Thu, 02 Nov 2006 09:44:17 GMT

How did you check, that there is no resource_Failure? In the debugger? That's of course not the way you can check that. Write an entry to the application log, if there is a resource failure and check then the application log. I made the experience, that you will find entries. The behaviour of 'starting new task' strongly depends on the RFC-parameters. Maybe you could also check them.

Regards,

Thomas

Re: Parallel processing question

Dorian — Thu, 02 Nov 2006 15:16:41 GMT

Thomas:

I'm logging any errors that occur; I'm not seeing any resource failures - in fact no errors at all, other than expected application data errors. It seems that the RFC calls are all being made in a single child process that queues up the parallel jobs and uses just one dialog process to run them all. I expected to see as many dialog tasks being used as were available.

As far as the RFC parameters go - are you referring to the RZ10 values? I looked at all of the parameters containing "rfc" as part of their name, and nothing looked as though it was restricting the parallel task behaviour. Do you have any advice as to suitable settings?

I'm also wondering if what I am seeing is just the way SAP is supposed to work? Although I expected to see lots of child processes running in multiple dialogs if processes are available, maybe by design only one remote process per server is allowed? I checked the documentation I could find on the "starting new task" keyword, and nowhere does it say that multiple processes will be started on each server in the server group; only that a child process will NOT be started if the number of unused processes fall below a defined threshold.

Re: Parallel processing question

Former Member — Thu, 02 Nov 2006 17:31:44 GMT

Hi,

"I'm also wondering if what I am seeing is just the way SAP is supposed to work? Although I expected to see lots of child processes running in multiple dialogs if processes are available, maybe by design only one remote process per server is allowed?"

Not from my experience, although it's been a while. We were seeing a lot of child processes. In fact, to avoid those processes from 'flooding' the servers, we restricted parallel processing to only a number of app servers (by the RFC group definition). Sorry this is not much help, but wanted to confirm back to you that what you're seeing is not typical. Good luck.

Re: Parallel processing question

Former Member — Thu, 02 Nov 2006 20:03:19 GMT

Maybe your processes are to fast: Try the following: In the RFC function module, which is to be started in new task, insert the line of code 'WAIT UP TO 20 SECONDS.'. Then you have enough time to check in sm50 if there are more parallel processes.

Best regards,

Thomas

Re: Parallel processing question

Former Member — Thu, 02 Nov 2006 20:22:04 GMT

if you just save IDOCs and then start SAP program to post them in parallel - do you have the same issue or it's limited to the case when FM is called from your code only ?

Re: Parallel processing question

Dorian — Thu, 02 Nov 2006 20:53:54 GMT

Thomas:

Turns out my problem was waiting too long; I set the wait period to 0.25, thinking that I needed to space the processes out a bit (afriad of the call-back getting tied up in knots).

After setting the wait to 20 seconds, I saw the wait on the RFC response really dragging, and wondered if my wait time was too big to fire off more than one child at a time. I went back and set the wait to 0.001 - now I'm seeing 3 or four running at a time <g>.

The issues I thought I would have with so many call-backs returning data together don't seem to have come up yet. I think I'll leave the wait setting as-is, and increase it only if I get problems.....

Thanks for getting me looking in the right direction!

Regards,