This blog describes how to configure exception handling in a splitter scenario. It describes the splitter in some sample scenarios using different exception handling configurations.
Usage of General and Iterating Splitter with Exception Handling
In many Cloud Integration scenarios big messages are split into smaller parts using a splitter pattern. The smaller chunks are then processed in cloud integration. In such scenarios multiple errors can occur, which can be handled in different ways. This blog gives some general recommendations and describes the options available in some sample scenarios.
General Recommendations
For using the splitter with exception handling there are some important recommendations:
1. Combination of Stop on Exception with End Events in Exception Subprocess
The exception handling defined in the
Splitter takes precedence over the end event defined in the Exception Subprocess with respect to continuation with the next split. This means the next split is only executed if
Stop on Exception is not set. If
Stop on Exception is set, the next split is not executed anymore independently of the end event used in an Exception Subprocess. Because of this runtime behavior, we recommend using only the following combinations:
Combine Stop on Exception with Error End Event
If you set the
Stop on Exception flag in the
Splitter either use no
Exception Subprocess, or if you need a specific error handling, configure the
Exception Subprocess to end with an
Error End event.
Using the
Message End event would, in this combination, lead to a probably unexpected behavior because the end event is only relevant for the single split, not for the overall split processing. This means, the overall split processing would stop.
- In case no Gather is used, the processing would end with a Completed message, even if not all splits were executed or tried to be executed because the split processing stops.
- If a Gather is used, only the single split would be completed, but the next split processing would not be executed anymore. The processing would continue with the splits already executed in the Gather. The final status of the message will then depend on the further message processing.
Because of this runtime behavior we recommend to
always use the Error End event when Stop on Exception is set.
Do not set Stop on Exception when using Message End Event
If you do not set the
Stop on Exception flag in the
Splitter either use no
Exception Subprocess, or, if you need a specific error handling, configure the
Exception Subprocess to end with a
Message End event.
Using an
Error End event would, in this combination, lead to a probably unexpected behavior because the end event is only relevant for the single split, not for the overall split processing: The overall split processing would continue.
- In case no Gather is used the processing would end with a Failed message, but the next split would still be executed because Stop on Exception is not set in the Splitter.
- If a Gather step is used, only the single split would end, but the overall processing would continue. The final message status will then depend on the further message processing.
Because of this runtime behavior we recommend to
always use the Message End event when Stop on Exception is not set.
2. Use Separate Local Process for Split Processing
Do the processing of the single splits (between
Splitter and
Gather or after the
Splitter if no
Gather is used
) in a separate
Local Process. Doing so, you can configure a dedicated exception handling for errors occurring in the split processing using an
Exception Subprocess in this local process.
Additional Considerations when Using the Gather Step
When using the
Splitter together with a
Gather step the following recommendations should be followed additionally.
1. Create Valid XML in Exception Subprocess
If you handle split errors in an
Exception Subprocess and use a
Gather step after the
Splitter, create a valid XML to be passed to the
Gather step. Otherwise the
Gather step could stop with an error because of the wrong format and the whole message processing would end with a
Failed status.
2. Avoid Splitter - Gather Scenarios without Exception Subprocess
Avoid configuring
Splitter - Gather scenarios without error handling in an
Exception Subprocess when using
Combine XMLs of the same or different format in the
Gather step. Otherwise the
Gather step may stop with an error because the payload at the time the exception occurs has the wrong format (wrong XML or no XML at all) and the whole message processing would end with a
Failed status.
Let's discuss how to follow the recommendations in some sample scenarios. First, I show you some sample scenarios with
Splitter, but without
Gather. There are different scenarios with different exception handling depending on the overall use case for the scenario. Further down I will describe the specific recommendations in sample scenarios using
Splitter and
Gather.
Splitter Scenarios without Gather
The following scenarios are simple sample scenarios using a
Splitter without a
Gather step after the
Splitter.
Scenario 1: Splitter without Gather with Stop on Exception
Scenario:
If you want to stop the complete message processing in case of an error in the processing of one of the splits, the easiest way would be to configure this using the
Stop on Exception option in the
Splitter:
In this sample scenario a bulk message is received by the SOAP adapter, this message is then split into single chunks. The processing of the single chunks is done in a separate
Local Process. In the sample scenario there simply is a
Request-Reply call fetching additional data. Afterwards the single splits are sent to the
Receiver.
Note the following important point:
- In this scenario, no Exception Subprocess is configured. If you need to configure a specific exception handling in an Exception Subprocess and want to stop the complete processing using Stop on Exception follow the recommendations in scenario 2.
Runtime Execution
If an error occurs during message processing, the processing stops, and the message gets a
Failed status. If some splits were executed successfully before the error occurred, their data is already sent to the receiver, no roll-back of the executed calls is possible. This means, parts of the original message have been sent to the receiver, some not.
The sender system usually gets the failed status back and resends the message (if configured for retry). When the sender system resends the message, the whole message is executed again.
In this scenario, re-executing the whole message means that also those split parts that had been successfully sent out at the first time are sent to the receiver for the second time. This leads to duplicate messages in the receiver system. So, either the receiver system can handle duplicates, or you have to configure your scenario differently to avoid those duplicates. One option would be to do a dedicated exception handling in an
Exception Subprocess as described in scenario 3.
Scenario 2: Splitter without Gather with Stop on Exception and Exception Subprocess
Scenario:
As already mentioned, you need to use an
Exception Subprocess if you want to do a specific error handling. If we want to end the complete processing, but would like to do a specific error handling before that, like writing a notification to someone, we can combine the
Stop on Exception option in the
Splitter and an
Exception Subprocess:
Like in scenario 1, a bulk message is received by the SOAP adapter, which is then split into single chunks. The processing of the single chunks is done in a separate
Local Process, where we fetch additional data via a
Request-Reply call and then send the single splits to the
Receiver.
The difference to scenario 1 is that we add an
Exception Subprocess in the
Local Process, where we configure the specific error handling. As shown above ,in the
Exception Subprocess added to the
Local Process, as a simple error handling we write a notification mail to someone. We end the
Exception Subprocess with an
Error End event because we want to end the complete processing anyway and do not continue with the next split.
Note the following important point:
- In this scenario, an Exception Subprocess is configured with an Error End event together with Stop on Exception. If you need to configure a specific exception handling in an Exception Subprocess and do not want to stop the complete processing, but handle the single erroneous splits, follow the recommendations in scenario 3.
Runtime Execution
The runtime processing in case of an error in this scenario is a little different from scenario 1. When an error occurs during split processing, the exception is caught by the
Exception Subprocess. The error handling is executed, which means that the notification mail is sent to the
Receiver. Afterwards the complete processing ends because of the
Stop on Exception option in the
Splitter. The next split is not executed. The message gets a
Failed status in the monitoring.
Similar to scenario 1, if some splits were executed successfully before the error occurred, their data is already sent to the receiver, no roll-back of the executed calls is possible. This means, parts of the original message have been sent to the receiver, some not.
The sender system usually gets the failed status back and resends the message (if configured for retry). When the sender system resends the message, the whole message is executed again.
Also, in this scenario, re-executing the whole message means that the already sent split parts are sent to the receiver again. Which leads to duplicate messages in the receiver system. So, either the receiver system can handle duplicates, or you have to configure your scenario differently to avoid those duplicates. One option would be to do a different exception handling in an
Exception Subprocess as described in scenario 3.
Scenario 3: Splitter without Gather with Exception Subprocess
Scenario:
As already mentioned, the
Exception Subprocess can be used to configure a specific error handling and then continue the overall processing. Let's take the sample scenario 2 but configure the
Exception Subprocess with an
Message End event. Make sure that
Stop on Exception is not selected in the
Splitter:
Like in scenarios 1 and 2, a bulk message is received by the SOAP adapter, which is then split into single chunks. The processing of the single chunks is done in a separate
Local Process. In the sample scenario there simply is a
Request-Reply call fetching additional data. Afterwards the single splits are sent to the
Receiver.
The difference to scenario 1 and 2 is that we add an
Exception Subprocess in the
Local Process, where we configure the specific error handling for the single split parts. To be able to do a specific error handling for the single chunks we retrieve a unique id for the chunk using a
Content Modifier and write it into a property. In this sample scenario we use a property with the name
'CharacterName':
In the
Exception Subprocess added to the
Local Process we write the chunk to a data store using a
Data Store Write step with the unique ID (property
'CharacterName') as
Entry ID. Note, that we select the option
Overwrite Existing Message. With this setting we can make sure that another execution of the same split does not create a new entry for the same split but overwrites the existing entry.
The important thing is that we end the
Exception Subprocess with a
Message End event. From runtime perspective, this means the error raised by the split processing is caught by the
Exception Subprocess and handled there. The overall processing continues with the next split because
Stop On Exception is not selected.
Note the following important point:
- In this scenario, an Exception Subprocess is configured with a Message End event. For this configuration Stop on Exception should not be selected.
Runtime Execution
The runtime processing in case of an error in this scenario is different from scenarios 1 and 2: When an error occurs during split processing, the exception is caught by the
Exception Subprocess. The error handling is executed, which means that this split message is written to the data store and is not sent out to the
Receiver. The overall processing continues until all splits are executed. The message gets a
Completed status in the monitoring.
The splits which caused an error, are stored in a data store and can, for example, be polled by a different integration flow and handled differently.
Note, that this is only a simple sample error handling configuration, in the
Exception Subprocess you can configure the specific handling required for your scenario. But keep one important aspects in mind:
- Do not set Stop on Exception in the splitter and end the processing in the Exception Subprocess with a Message End event if you want to continue processing of the next splits. The continuation of the split execution depends on the Stop on Exception setting.
Scenario 4: Splitter without Gather without Stop on Exception and without Exception Subprocess
Scenario:
If you don't need to take care of errors for single splits but just want to continue processing of all possible splits, you could use the option to configure the scenario without
Stop on Exception in
Splitter and without
Exception Subprocess. The scenario would be the same as scenario 1 but
Stop on Exception would not be selected.
Runtime Execution
If an error occurs during split processing, the single split is not completed but the whole processing continues. This means that all successful splits are sent out to the
Receiver. The splits that could not be sent successfully are just ignored.
The message gets a
Failed status, because not all splits could be executed successfully.
Splitter Scenarios with Gather
The following scenarios are simple sample scenarios using a
Splitter step combined with a
Gather step which combines the single splits using an aggregation strategy.
Scenario 5: Splitter with Gather with Stop on Exception
Scenario:
Also, with the
Gather step the splitter scenario can be configured to stop the complete message processing if an error occurs. Again, we configure this using the
Stop on Exception option in the
Splitter:
In the small sample scenario, a bulk message is received by the SOAP adapter, this message is then split into single chunks. The processing of the single chunks is done in a separate
Local Process. In the small sample scenario there simply is a
Request-Reply call fetching additional data. All the splits are combined in a
Gather step configured with
Combine option:
Afterwards an
XSLT Mapping is executed, and the message is sent to the
Receiver. The whole scenario looks like shown above.
Note the following important point:
- In this scenario, no Exception Subprocess is configured. If you need to configure a specific exception handling in an Exception Subprocess and want to stop the complete processing using Stop on Exception follow the recommendations in scenario 6.
Runtime Execution
If an error occurs during message processing, the whole processing stops, and the message gets a
Failed status. The sender system usually gets the failed status back and would resend the message (if configured for retry). If the sender system resends the message, the whole message is executed again, including all the split parts that were already executed in the previous execution.
Scenario 6: Splitter with Gather with Stop on Exception and Exception Subprocess
Scenario:
Also, with the
Gather step the splitter scenario can be configured with
Stop on Exception and an
Exception Subprocess to end the complete processing, but still do a specific error handling before, like for example writing a notification to someone.
Like in scenario 5, a bulk message is received by the SOAP adapter, which is then split into single chunks. The processing of the single chunks is done in a separate
Local Process. In the sample scenario there simply is a
Request-Reply call fetching additional data. All the splits are combined in a
Gather step configured with
Combine option.
After the
Gather an
XSLT Mapping is executed and the message is sent to the
Receiver.
The difference to scenario 5 is that we add an
Exception Subprocess in the
Local Process, where we configure the specific error handling. As shown above in the
Exception Subprocess added to the
Local Process, as a simple error handling we write a notification mail to someone. We end the
Exception Subprocess with an
Error End event because we want to end the complete processing.
Note the following important point:
- In this scenario, an Exception Subprocess is configured with an Error End event together with Stop on Exception. If you need to configure a specific exception handling in an Exception Subprocess and do not want to stop the complete processing, but handle the single erroneous splits, follow the recommendations in scenario 7.
Runtime Execution
The runtime processing in case of an error in this scenario is a little different from scenario 5. When an error occurs during split processing, the exception is caught by the
Exception Subprocess. The error handling is executed, which means that the notification mail is sent to the Receiver. Afterwards the complete processing ends. The message gets a
Failed status in the monitoring because the
Stop on Exception option sets the message to
Failed in case of an error independently of the kind of end event used in the
Exception Subprocess.
Scenario 7: Splitter with Gather with Exception Subprocess
Scenario:
As already mentioned, you can use an Exception Subprocess if you want to do a specific error handling. This time we really want to handle the exception and do not end the whole processing. Make sure
Stop on Exception is not selected in the
Splitter:
Like in scenarios 5 and 6, a bulk message is received by the SOAP adapter, which is then split into single chunks. The processing of the single chunks is done in a separate
Local Process. In the sample scenario there simply is a
Request-Reply call fetching additional data. All the splits are combined in a
Gather step configured with
Combine option:
After the
Gather an
XSLT Mapping is executed and the message is sent to the
Receiver.
The difference to scenarios 5 and 6 is that we add an
Exception Subprocess in the
Local Process, where we configure the specific error handling for the single splits. To be able to do a specific error handling for the single chunks we retrieve a unique id for the chunk using a
Content Modifier and write it into a property. In this sample scenario we use a property with the name
'CharacterName':
In the
Exception Subprocess added to the
Local Process as a simple error handling we write the split message to a data store using a
Data Store Write step with the unique ID (property
'CharacterName') as
Entry ID. Note, that we select the option
Overwrite Existing Message. With this setting we can make sure that another execution of the same split does not create a new entry for the same split but overwrites the existing entry.
In the
Content Modifier after the
Data Store Write step we need to create a valid XML, that can be understood by the
Gather step. It needs to have the same format as the other split responses that are combined in the
Gather. In my scenario I simply define an empty book tag, because this is the format expected by the
Gather step
.
We end the
Exception Subprocess with a
Message End event. From runtime perspective, this means that the error raised by the split processing is caught by the
Exception Subprocess and handled there. The overall processing continues with the next split because
Stop On Exception is not selected.
Note the following important point:
- In this scenario, an Exception Subprocess is configured with a Message End event. For this Stop on Exception should not be selected.
Runtime Execution
The runtime processing in case of an error in this scenario is different from scenario 5 and 6. When an error occurs during split processing, the exception is caught by the
Exception Subprocess. The error handling is executed, which means that this split message is written to the data store and an empty, but valid XML payload is created which is then passed to the
Gather. The overall processing continues until all splits are executed. The single splits, including the empty ones, are combined in the
Gather step. An
XSLT Mapping is executed, and the message is sent to the
Receiver. The message gets a
Completed status in the monitoring.
The splits which caused an error, are stored in a data store and can, for example, be polled by a different integration flow and handled differently.
Note, that this is only a simple sample error handling configuration, in the
Exception Subprocess you can configure the specific handling required for your scenario. But keep two important aspects in mind:
- Do not set Stop on Exception in the splitter and end the processing in the Exception Subprocess with a Message End event to continue processing of the next splits.
- Create a valid XML to be passed to the Gather step, otherwise the Gather step would stop with an error because of the wrong format and the whole message processing would end with a Failed status.
Scenario 8: Splitter with Gather without Stop on Exception and without Exception Subprocess - Not Recommended
Scenario:
Also, in a scenario with
Gather you could configure the scenario without
Stop on Exception in
Splitter and without
Exception Subprocess. The scenario would be the same as scenario 5, but
Stop on Exception is not selected.
Note, that this scenario setup
cannot be recommended because of the following reason:
This scenario would only work in error case if the single splits going to the
Gather step have the same format, because we configured the
Gather step with
Combine XMLs with same format. If an exception occurs in one split, the payload at the time the exception is raised would be routed to the
Gather step. This payload may not have the correct format and so the
Gather would end in an error. To avoid this, the recommended configuration option would be to configure the scenario with an
Exception Subprocess as described in scenario 7.
Further Readings
For more configuration recommendations in respect to Splitter also check out the following blogs: