Here comes another new version 1.0.6 of the Pipeline Concept were we introduced a couple of new features. The package can be accessed from the SAP Business Accelerator Hub. If you have already copied the package to your workspace in your SAP Integration Suite tenant, you can simply run the update of the package to be able to use the latest features.
Upfront, I like to give a warning. We had to introduce an incompatible change of how we define the partner ID to retrieve the Partner Directory parameters, actually this applies to the XSLT for determining the interfaces only. The rest of the Partner Directory definitions are not impacted unless you like to go for the new feature using alternative partner which we would actually recommend.
Before getting into detail for some of the new features, here all new features at a glance:
- Revised partner id approach due to lengths restrictions of the partner id
- Option to use alternative partner in case of sender wildcard scenarios and to overcome partner id restrictions
- Changed default max retry to 5 instead of unlimited retry
- For messages in test mode, failed messages are not put into retry
- For generic integration flows, adapter parameters were externalized
- Option to configure message end event type in case of message stored in dead letter queue, default is Escalated
- Option to bypass receiver determination and interface determination steps in case of Point-to-Point pattern
- Option to configure a pipeline JMS queue prefix which makes configuration of queue names easier if you create another set of generic integration flow
Furthermore, I like to point you to a new github repository were we describe how to setup the Pipeline for Cloud Integration for a couple of sample scenarios.
New partner ID definition incl alternative partner
Due to the length restriction of the partner ID of 60 characters for retrieving string and XSLT parameters from the Partner Directory, there was a risk of exceeding this limit so that the creation of the Partner Directory entries may fail. This especially applies for IDoc messages where we combine message type, IDoc type and extension type as well as for the partner ID for retrieving the interface Determination XSLT which was a combination of sender component, sender interface and receiver component.
For this reason, we had to change the partner ID definition for retrieving the interface determination XSLT. Furthermore, we introduced the support for alternative partners to generally overcome the length restrictions.
With the alternative partner support, you have the option to map the sender component and the sender interface to a partner ID of your choice, the partner ID might be an integration scenario name. There are lengths restrictions for agency (120 chars), scheme (120 chars) and id (255 chars) as well but in practice it’s very unlikely that you will exceed this. If a message enters the pipeline, we first check if an alternative partner exists, in this case we use the mapped partner ID. Otherwise, the old partner ID approach is used which combines sender component and sender interface separated by ~. So, you can stick to the old approach but you need to ensure that it doesn’t exceed 60 characters.
For the interface determination, we used to define the partner ID as combination of sender component, sender interface and receiver component, all separated by ~. The id was defined as constant string interfaceDetermination. With the new version, we now use the same partner ID like for any other Partner Directory parameters, so either the partner ID from the alternative partner or the combined sender component and sender interface. For the id, we now concatenate the constant string interfaceDetermination and the receiver component separated by underscore _.
Here's an example for better understanding:
Assuming, a message of interface MaterialCreate is sent from sender ABC and routed to two receivers XYZ and UVW. Your scenario is called MaterialMasterReplicate.
We define the alternative partner as follows:
- Agency: ABC
- Scheme: SenderInterface (this is a fixed string)
- Id: MaterialCreate
- Pid: MaterialMasterReplicate
For the receiver determination, the key to upload and retrieve the XSLT is as follows:
- Pid: MaterialMasterReplicate
- Id: receiverDetermination
For the interface determination for receiver XYZ, the key to upload and retrieve the XSLT is as follows:
- Pid: MaterialMasterReplicate
- Id: interfaceDetermination_XYZ
For the interface determination for receiver UVW, the key to upload and retrieve the XSLT is as follows:
- Pid: MaterialMasterReplicate
- Id: interfaceDetermination_UVW
The alternative partner approach is not mandatory however we strongly recommend to use this approach for the following reasons:
- In practice, it avoids exceeding the lengths restrictions in any case
- It groups together all your Partner Directory parameters which belong to the same integration scenario using a meaningful scenario name
- It supports sender wildcard scenarios out of the box because it allows to map multiple sender systems to the same partner ID
Before you update your package and re-deploy the generic integration flows, ensure to update your Partner Directory entries accordingly.
See Using the Partner Directory in the Pipeline Concept.
Special case: Point-to-Point scenarios
For Point-to-Point scenarios where we have one sender and one receiver only, we now have the option to bypass the two pipeline steps receiver determination and interface determination which at the end leads to an improved runtime behavior.
In this case, we won't upload the XSLTs to the Partner Directory, instead we need to define two corresponding string parameters receiverDetermination and interfaceDetermination_<receiver name>. Former defines the receiver name, and latter the ProcessDirect endpoint of the scenario-specific outbound processing flow.
For more details, see Special Cases.
Retry handling enhancements
For the retry handling, we have changed the default maximum retry from unlimited to 5. The default applies if you have not maintained a scenario specific maximum retry.
If the maximum retry exceeds, the messages are stored in a dead letter queue. For the standard retry handling you now have the option to configure the message end event type in case of messages stored in the dead letter queue. The default is Escalated. Optionally, you can change the type to Completed.
Furthermore, for messages in test mode, i.e., were the testMode header is true, failed messages are not put into retry, instead they are completed with custom status ErrorInTestMode.
See Standard Retry Handling.
New github repository with sample scenarios
If you like to make yourself familiar with the Pipeline for Cloud Integration and to test the new features that we have recently shipped, check out the Process Integration Pipeline Sample Scenarios github repository.
In the repository, we explain how to setup different sample scenarios using the Pipeline for Cloud Integration. For each sample scenario, an integration package is provided. Furthermore, a Postman collection incl. Postman environment is provided that allows you to setup the Partner Directory entries and to trigger sample messages.
As usually, let me know if you have further ideas, requirements, and feedback, feel free to reach out to me or simply leave a comment.
In the end, I would like to thank my colleagues Mate and Vyacheslav from the Center of Excellence team as well as Daniel from Figaf for their input, a lot of the new features are actually based on their feedback and advise.
Bluesky
