‎2007 Nov 19 12:52 PM
Hi,
I'm about to develop a class for task management.
It should have methods like GET_TASK returning a taskid and task_return importing task_id received and possibly task_fail importing task_id sent with a bad sy-subrc when calling function with starting new task.
The idea behind is: New task is started by system using a dialog process with time limit restriction applying. Thus I have to split the whole task into more small packets than processes available. So I think I have to use the same task again after it returned successfully.
My assumption is I can't use more tasks than processes initially available and I can use the same tsak again after it has successsfully returned results.
I want some ideas and information on that exceeding the SAP documentation.
TIA.
Regards,
Clemens
‎2007 Nov 19 12:53 PM
Hi Clemens,
Here is a sample program for Parallel processing.
Hope this helps.
REPORT PARAJOB.
*
Data declarations
DATA: GROUP LIKE RZLLITAB-CLASSNAME VALUE ' ',
"Parallel processing group.
"SPACE = group default (all
"servers)
WP_AVAILABLE TYPE I, "Number of dialog work processes
"available for parallel processing
"(free work processes)
WP_TOTAL TYPE I, "Total number of dialog work
"processes in the group
MSG(80) VALUE SPACE, "Container for error message in
"case of remote RFC exception.
INFO LIKE RFCSI, C, "Message text
JOBS TYPE I VALUE 10, "Number of parallel jobs
SND_JOBS TYPE I VALUE 1, "Work packets sent for processing
RCV_JOBS TYPE I VALUE 1, "Work packet replies received
EXCP_FLAG(1) TYPE C, "Number of RESOURCE_FAILUREs
TASKNAME(4) TYPE N VALUE '0001', "Task name (name of
"parallel processing work unit)
BEGIN OF TASKLIST OCCURS 10, "Task administration
TASKNAME(4) TYPE C,
RFCDEST LIKE RFCSI-RFCDEST,
RFCHOST LIKE RFCSI-RFCHOST,
END OF TASKLIST.
*
Optional call to SBPT_INITIALIZE to check the
group in which parallel processing is to take place.
Could be used to optimize sizing of work packets
work / WP_AVAILABLE).
*
CALL FUNCTION 'SPBT_INITIALIZE'
EXPORTING
GROUP_NAME = GROUP
"Name of group to check
IMPORTING
MAX_PBT_WPS = WP_TOTAL
"Total number of dialog work
"processes available in group
"for parallel processing
FREE_PBT_WPS = WP_AVAILABLE
"Number of work processes
"available in group for
"parallel processing at this
"moment
EXCEPTIONS
INVALID_GROUP_NAME = 1
"Incorrect group name; RFC
"group not defined. See
"transaction RZ12
INTERNAL_ERROR = 2
"R/3 System error; see the
"system log (transaction
"SM21) for diagnostic info
PBT_ENV_ALREADY_INITIALIZED = 3
"Function module may be
"called only once; is called
"automatically by R/3 if you
"do not call before starting
"parallel processing
CURRENTLY_NO_RESOURCES_AVAIL = 4
"No dialog work processes
"in the group are available;
"they are busy or server load
"is too high
NO_PBT_RESOURCES_FOUND = 5
"No servers in the group
"met the criteria of >
"two work processes
"defined.
CANT_INIT_DIFFERENT_PBT_GROUPS = 6
"You have already initialized
"one group and have now tried
"initialize a different group.
OTHERS = 7..
CASE SY-SUBRC.
WHEN 0.
"Everythings ok. Optionally set up for optimizing size of
"work packets.
WHEN 1.
"Non-existent group name. Stop report.
MESSAGE E836. "Group not defined.
WHEN 2.
"System error. Stop and check system log for error
"analysis.
WHEN 3.
"Programming error. Stop and correct program.
MESSAGE E833. "PBT environment was already initialized.
WHEN 4.
"No resources: this may be a temporary problem. You
"may wish to pause briefly and repeat the call. Otherwise
"check your RFC group administration: Group defined
"in accordance with your requirements?
MESSAGE E837. "All servers currently busy.
WHEN 5.
"Check your servers, network, operation modes.
WHEN 6.
Do parallel processing. Use CALL FUNCTION STARTING NEW TASK
DESTINATION IN GROUP to call the function module that does the
work. Make a call for each record that is to be processed, or
divide the records into work packets. In each case, provide the
set of records as an internal table in the CALL FUNCTION
keyword (EXPORT, TABLES arguments).
DO.
CALL FUNCTION 'RFC_SYSTEM_INFO' "Function module to perform
"in parallel
STARTING NEW TASK TASKNAME "Name for identifying this
"RFC call
DESTINATION IN GROUP group "Name of group of servers to
"use for parallel processing.
"Enter group name exactly
"as it appears in transaction
"RZ12 (all caps). You may
"use only one group name in a
"particular ABAP program.
PERFORMING RETURN_INFO ON END OF TASK
"This form is called when the
"RFC call completes. It can
"collect IMPORT and TABLES
"parameters from the called
"function with RECEIVE.
EXCEPTIONS
COMMUNICATION_FAILURE = 1 MESSAGE msg
"Destination server not
"reached or communication
"interrupted. MESSAGE msg
"captures any message
"returned with this
"exception (E or A messages
"from the called FM, for
"example. After exception
"1 or 2, instead of aborting
"your program, you could use
"SPBT_GET_PP_DESTINATION and
"SPBT_DO_NOT_USE_SERVER to
"exclude this server from
"further parallel processing.
"You could then re-try this
"call using a different
"server.
SYSTEM_FAILURE = 2 MESSAGE msg
"Program or other internal
"R/3 error. MESSAGE msg
"captures any message
"returned with this
"exception.
RESOURCE_FAILURE = 3. "No work processes are
"currently available. Your
"program MUST handle this
"exception.
YOUR_EXCEPTIONS = X. "Add exceptions generated by
"the called function module
"here. Exceptions are
"returned to you and you can
"respond to them here.
CASE SY-SUBRC.
WHEN 0.
"Administration of asynchronous RFC tasks
"Save name of task...
TASKLIST-TASKNAME = TASKNAME.
"... and get server that is performing RFC call.
CALL FUNCTION 'SPBT_GET_PP_DESTINATION'
EXPORTING
RFCDEST = TASKLIST-RFCDEST
EXCEPTIONS
OTHERS = 1.
APPEND TASKLIST.
WRITE: / 'Started task: ', TASKLIST-TASKNAME COLOR 2.
TASKNAME = TASKNAME + 1.
SND_JOBS = SND_JOBS + 1.
"Mechanism for determining when to leave the loop. Here, a
"simple counter of the number of parallel processing tasks.
"In production use, you would end the loop when you have
"finished dispatching the data that is to be processed.
JOBS = JOBS - 1. "Number of existing jobs
IF JOBS = 0.
EXIT. "Job processing finished
ENDIF.
WHEN 1 OR 2.
"Handle communication and system failure. Your program must
"catch these exceptions and arrange for a recoverable
"termination of the background processing job.
"Recommendation: Log the data that has been processed when
"an RFC task is started and when it returns, so that the
"job can be restarted with unprocessed data.
WRITE msg.
"Remove server from further consideration for
"parallel processing tasks in this program.
"Get name of server just called...
CALL FUNCTION 'SPBT_GET_PP_DESTINATION'
EXPORTING
RFCDEST = TASKLIST-RFCDEST
EXCEPTIONS
OTHERS = 1.
"Then remove from list of available servers.
CALL FUNCTION 'SPBT_DO_NOT_USE_SERVER'
IMPORTING
SERVERNAME = TASKLIST-RFCDEST
EXCEPTIONS
INVALID_SERVER_NAME = 1
NO_MORE_RESOURCES_LEFT = 2
"No servers left in group.
PBT_ENV_NOT_INITIALIZED_YET = 3
OTHERS = 4.
WHEN 3.
"No resources (dialog work processes) available at
"present. You need to handle this exception, waiting
"and repeating the CALL FUNCTION until processing
"can continue or it is apparent that there is a
"problem that prevents continuation.
MESSAGE I837. "All servers currently busy.
"Wait for replies to asynchronous RFC calls. Each
"reply should make a dialog work process available again.
IF EXCP_FLAG = SPACE.
EXCP_FLAG = 'X'.
"First attempt at RESOURCE_FAILURE handling. Wait
"until all RFC calls have returned or up to 1 second.
"Then repeat CALL FUNCTION.
WAIT UNTIL RCV_JOBS >= SND_JOBS UP TO '1' SECONDS.
ELSE.
"Second attempt at RESOURCE_FAILURE handling
WAIT UNTIL RCV_JOBS >= SND_JOBS UP TO '5' SECONDS.
"SY-SUBRC 0 from WAIT shows that replies have returned.
"The resource problem was therefore probably temporary
"and due to the workload. A non-zero RC suggests that
"no RFC calls have been completed, and there may be
"problems.
IF SY-SUBRC = 0.
CLEAR EXCP_FLAG.
ELSE. "No replies
"Endless loop handling
...
ENDIF.
ENDIF.
ENDCASE.
ENDDO.
...
*
Wait for end of job: replies from all RFC tasks.
Receive remaining asynchronous replies
WAIT UNTIL RCV_JOBS >= SND_JOBS.
LOOP AT TASKLIST.
WRITE:/ 'Received task:', TASKLIST-TASKNAME COLOR 1,
30 'Destination: ', TASKLIST-RFCDEST COLOR 1.
ENDLOOP.
...
*
This routine is triggered when an RFC call completes and
returns. The routine uses RECEIVE to collect IMPORT and TABLE
data from the RFC function module.
*
Note that the WRITE keyword is not supported in asynchronous
RFC. If you need to generate a list, then your RFC function
module should return the list data in an internal table. You
can then collect this data and output the list at the conclusion
of processing.
*
FORM RETURN_INFO USING TASKNAME.
DATA: INFO_RFCDEST LIKE TASKLIST-RFCDEST.
RECEIVE RESULTS FROM FUNCTION 'RFC_SYSTEM_INFO'
IMPORTING RFCSI_EXPORT = INFO
EXCEPTIONS
COMMUNICATION_FAILURE = 1
SYSTEM_FAILURE = 2.
RCV_JOBS = RCV_JOBS + 1. "Receiving data
IF SY-SUBRC NE 0.
Handle communication and system failure
...
ELSE.
READ TABLE TASKLIST WITH KEY TASKNAME = TASKNAME.
IF SY-SUBRC = 0. "Register data
TASKLIST-RFCHOST = INFO_RFCHOST.
MODIFY TASKLIST INDEX SY-TABIX.
ENDIF.
ENDIF.
...
ENDFORM
<b>Reward points if this helps.
Manish</b>
‎2007 Nov 19 1:29 PM
Hi Manish,
thank you for pasting the SAP documentation without saying so.
I said: "Exceeding SAP documentation"
The docu text says what I think is not enough:
"In production use, you would end the loop when you have
"finished dispatching the data that is to be processed.
Your answer just wastes my time [and nerves].
[Grrr],
Clemens
‎2007 Nov 28 6:53 PM
Hey, I got it:
The dispatched processes do not start to process the form given with PERFORMING ... ON END OF TASK before the WAIT statement has been reached.
Then the tasks return their results in the RECEIVING statement in the form.
I tried to start a new task imediately after receiving a result from the first task started. But this did not work, I had too many process fails.
Now I start as many processes as possible, then I wait for all results having been received, then I start again until everything is finished.
You may try my sample program which was developed under 46C.
‎2007 Nov 28 6:56 PM
... and here comes the program (no formatting available with the answer given with "solved".
The programs retrieves billing documents with function BAPI_BILLINGDOC_GETLIST - this was the only BAPI I found quickly that has ranges as import parameters. This allows giving packages to the tasks.
*&---------------------------------------------------------------------*
*& Report ZZZPARTEST *
*& *
*&---------------------------------------------------------------------*
*& *
*& *
*&---------------------------------------------------------------------*
REPORT zzzpartest.
PARAMETERS:
p_dbcnt TYPE sydbcnt DEFAULT 1010,
p_pacsz TYPE sydbcnt DEFAULT 95.
CONSTANTS:
gc_function TYPE tfdir-funcname
VALUE 'BAPI_BILLINGDOC_GETLIST'.
DATA:
gt_bapivbrksuccess TYPE TABLE OF
bapivbrksuccess,
gv_activ TYPE i,
gv_rcv TYPE i,
gv_snd TYPE i,
BEGIN OF ls_intval,
task TYPE numc4,
idxfr TYPE i,
idxto TYPE i,
activ TYPE flag,
fails TYPE i,
END OF ls_intval,
gt_intval LIKE TABLE OF ls_intval.
START-OF-SELECTION.
PERFORM paralleltest.
*---------------------------------------------------------------------*
* CLASS task DEFINITION
*---------------------------------------------------------------------*
* ........ *
*---------------------------------------------------------------------*
CLASS task DEFINITION.
PUBLIC SECTION.
CLASS-METHODS:
provide
RETURNING
value(name) TYPE numc4,
return
IMPORTING
name TYPE numc4,
initialize
RETURNING
value(group) TYPE rzllitab-classname.
PRIVATE SECTION.
CLASS-DATA:
gv_group TYPE rzllitab-classname,
BEGIN OF ls_task,
name TYPE numc4,
used TYPE flag,
END OF ls_task,
gt_task LIKE TABLE OF ls_task.
ENDCLASS. "itab DEFINITION
*** CLASS itab IMPLEMENTATION ***
CLASS task IMPLEMENTATION.
METHOD initialize.
DATA:
lv_max TYPE i,
lv_inc TYPE numc7,
lv_free TYPE i.
CHECK gt_task IS INITIAL.
SELECT classname
INTO gv_group
FROM rzllitab UP TO 1 ROWS
WHERE grouptype = 'S'.
ENDSELECT.
CALL FUNCTION 'SPBT_INITIALIZE'
EXPORTING
group_name = gv_group
IMPORTING
max_pbt_wps = lv_max
free_pbt_wps = lv_free
EXCEPTIONS
invalid_group_name = 1
internal_error = 2
pbt_env_already_initialized = 3
currently_no_resources_avail = 4
no_pbt_resources_found = 5
cant_init_different_pbt_groups = 6
OTHERS = 7.
IF sy-subrc <> 0.
MESSAGE ID sy-msgid TYPE sy-msgty NUMBER sy-msgno
WITH sy-msgv1 sy-msgv2 sy-msgv3 sy-msgv4.
ENDIF.
SUBTRACT 2 FROM lv_free.
IF lv_free >= 1.
DO lv_free TIMES.
ls_task-name = sy-index.
APPEND ls_task TO gt_task.
ENDDO.
group = gv_group.
MESSAGE s000(r1)
WITH
'Parallelverarbeitung benutzt'
lv_free
'Prozesse in Gruppe'
gv_group.
* & & & &
ELSE.
MESSAGE e000(r1)
WITH
'Parallelverarbeitung abgebrochen,'
lv_free
'Prozesse in Gruppe'
gv_group.
* & & & &
ENDIF.
ENDMETHOD. "initialize
METHOD provide.
FIELD-SYMBOLS:
<task> LIKE ls_task.
IF gv_group IS INITIAL.
MESSAGE e000(r1)
WITH 'Task group not initialized'.
ENDIF.
LOOP AT gt_task ASSIGNING <task>
WHERE used IS initial.
EXIT.
ENDLOOP.
CHECK sy-subrc = 0.
<task>-used = 'X'.
name = <task>-name.
ENDMETHOD.
METHOD return.
LOOP AT gt_task INTO ls_task
WHERE
name = name
AND used = 'X'.
DELETE gt_task.
ENDLOOP.
IF sy-subrc = 0.
CLEAR ls_task-used.
APPEND ls_task TO gt_task.
ELSE.
* fatal error
ENDIF.
ENDMETHOD.
ENDCLASS. "itab IMPLEMENTATION
*&---------------------------------------------------------------------*
*& Form paralleltest
*&---------------------------------------------------------------------*
FORM paralleltest.
DATA:
ls_bapi_ref_doc_range TYPE bapi_ref_doc_range,
lv_done TYPE flag,
lv_group TYPE rzllitab-classname,
lv_task TYPE numc4,
lv_msg TYPE text255,
lv_grid_title TYPE lvc_title,
lv_tfill TYPE sytfill,
lv_vbelv TYPE vbelv,
lv_npacs TYPE i,
lt_vbelv TYPE SORTED TABLE OF vbelv
WITH UNIQUE KEY table_line,
lv_mod TYPE i.
FIELD-SYMBOLS:
<intval> LIKE LINE OF gt_intval.
* build intervals
SELECT vbelv INTO lv_vbelv
FROM vbfa.
INSERT lv_vbelv INTO TABLE lt_vbelv.
CHECK sy-subrc = 0.
ADD 1 TO lv_tfill.
CHECK:
p_dbcnt > 0,
lv_tfill >= p_dbcnt.
EXIT.
ENDSELECT.
DESCRIBE TABLE lt_vbelv LINES lv_tfill.
IF (
p_pacsz < p_dbcnt OR
p_dbcnt = 0
) AND
p_pacsz > 0.
* p_dbcnt > 0 ).
lv_npacs = lv_tfill DIV p_pacsz.
lv_mod = lv_tfill MOD p_pacsz.
IF lv_mod <> 0.
ADD 1 TO lv_npacs.
ENDIF.
DO lv_npacs TIMES.
ls_intval-idxfr = ls_intval-idxto + 1.
ls_intval-idxto = ls_intval-idxfr - 1
+ p_pacsz.
IF ls_intval-idxto > lv_tfill.
ls_intval-idxto = lv_tfill.
ENDIF.
APPEND ls_intval TO gt_intval.
ENDDO.
ELSE.
ls_intval-idxfr = 1.
ls_intval-idxto = lv_tfill.
APPEND ls_intval TO gt_intval.
ENDIF.
WHILE lv_done IS INITIAL.
* find an interval to be processed
LOOP AT gt_intval ASSIGNING <intval>
WHERE activ = space
AND fails BETWEEN 0 AND 5.
EXIT.
ENDLOOP.
IF sy-subrc <> 0.
* no inactive unprocessed interval. All complete or must wait?
* check for intervals with unsuccesful tries
LOOP AT gt_intval ASSIGNING <intval>
WHERE fails BETWEEN 0 AND 5.
EXIT.
ENDLOOP.
IF sy-subrc = 0.
* wait until all started processes have been received.
* Note: No receive is executed without WAIT
WAIT UNTIL gv_activ IS INITIAL UP TO 600 SECONDS.
ELSE.
* all done
lv_done = 'X'.
ENDIF.
UNASSIGN <intval>.
ENDIF.
* process interval if provided
IF <intval> IS ASSIGNED.
WHILE lv_task IS INITIAL.
IF lv_group IS INITIAL.
* init parallel processing
lv_group = task=>initialize( ).
ENDIF.
* get unused task
lv_task = task=>provide( ).
CHECK lv_task IS INITIAL.
* no unused task? wait for all started task are received
WAIT UNTIL gv_activ IS INITIAL UP TO 600 SECONDS.
ENDWHILE.
* call if task assigned
CHECK NOT lv_task IS INITIAL.
* prepare function parameters
ls_bapi_ref_doc_range = 'IBT'.
READ TABLE lt_vbelv INTO ls_bapi_ref_doc_range-ref_doc_low
INDEX <intval>-idxfr.
READ TABLE lt_vbelv INTO ls_bapi_ref_doc_range-ref_doc_high
INDEX <intval>-idxto.
* mark interval as failed
ADD 1 TO <intval>-fails.
ADD 1 TO gv_snd.
CALL FUNCTION gc_function
STARTING NEW TASK lv_task
DESTINATION IN GROUP lv_group
PERFORMING bapi_receive ON END OF TASK
EXPORTING
refdocrange = ls_bapi_ref_doc_range
EXCEPTIONS
communication_failure = 1 MESSAGE lv_msg
system_failure = 2 MESSAGE lv_msg
RESOURCE_FAILURE = 3.
IF sy-subrc = 0.
<intval>-activ = 'X'.
<intval>-task = lv_task.
ADD 1 TO gv_activ.
ELSE.
CALL METHOD task=>return EXPORTING name = lv_task.
ENDIF.
CLEAR lv_task.
ENDIF.
ENDWHILE.
* wait for pending processes
MESSAGE s000(r1) WITH 'Wait for pending processes'.
WAIT UNTIL gv_activ IS INITIAL.
* report unfinished intervals
LOOP AT gt_intval ASSIGNING <intval>
WHERE fails >= 0.
READ TABLE lt_vbelv INTO ls_bapi_ref_doc_range-ref_doc_low
INDEX <intval>-idxfr.
READ TABLE lt_vbelv INTO ls_bapi_ref_doc_range-ref_doc_high
INDEX <intval>-idxto.
MESSAGE i000(r1)
WITH
'Unverarbeitetes Intervall von'
ls_bapi_ref_doc_range-ref_doc_low
'bis'
ls_bapi_ref_doc_range-ref_doc_high.
ENDLOOP.
MESSAGE s000(r1) WITH 'start ALV'.
* transfer results to standard table
WRITE gv_rcv TO lv_grid_title LEFT-JUSTIFIED.
lv_grid_title+40(1) = '+'.
WRITE gv_snd TO lv_grid_title+50 LEFT-JUSTIFIED.
REPLACE '+' WITH 'RCV/SND' INTO lv_grid_title.
CONDENSE lv_grid_title.
CALL FUNCTION 'REUSE_ALV_GRID_DISPLAY'
EXPORTING
i_structure_name = 'BAPIVBRKSUCCESS'
i_grid_title = lv_grid_title
TABLES
t_outtab = gt_bapivbrksuccess.
ENDFORM. " paralleltest
*&---------------------------------------------------------------------*
*& Form bapi_receive
*&---------------------------------------------------------------------*
FORM bapi_receive USING pv_task TYPE any.
DATA:
lv_task TYPE numc4,
lt_bapivbrksuccess TYPE TABLE OF bapivbrksuccess,
lv_msg TYPE text80,
lv_subrc TYPE sy-subrc.
FIELD-SYMBOLS:
<intval> LIKE LINE OF gt_intval.
CLEAR lt_bapivbrksuccess.
RECEIVE RESULTS FROM FUNCTION gc_function
TABLES
success = lt_bapivbrksuccess
EXCEPTIONS
communication_failure = 1 MESSAGE lv_msg
system_failure = 2 MESSAGE lv_msg .
lv_subrc = sy-subrc.
lv_task = pv_task.
CALL METHOD task=>return EXPORTING name = lv_task.
LOOP AT gt_intval ASSIGNING <intval>
WHERE task = lv_task
AND fails <> -1.
EXIT.
ENDLOOP.
IF sy-subrc <> 0.
* fatal error
MESSAGE e000(r1)
WITH 'returned task' lv_task 'not in task table'.
ENDIF.
CLEAR <intval>-activ.
CASE lv_subrc.
WHEN 0.
<intval>-fails = -1.
APPEND LINES OF lt_bapivbrksuccess TO gt_bapivbrksuccess.
ADD 1 TO gv_rcv.
WHEN 1.
ADD 1 TO <intval>-fails.
WRITE: 'communication_failure for task', lv_task, lv_msg.
WHEN 2.
WRITE: 'system_failure', lv_task, lv_msg.
ADD 1 TO <intval>-fails.
ENDCASE.
SUBTRACT 1 FROM gv_activ.
ENDFORM. " bapi_receive
Regards,
Clemens