Asynchronous error handling and persist when the target endponit is down for long peirod of time

[nandhini_91]

Hi Experts,

For Asynchronous error handling and message retry in enterprise edition where huge amount of messages are expected to be stored for over 10-20 hours, which method would be the best - using JMS queues or SAP Event Mesh?

Also, in the event mesh scenario, will the queues be stored in the integration suite data store, or does it have another data storage for it.

While both works well for a few messages. What would be the best approach if we the queue has to handle huge message loads for a longer period of time.

[mig1]

Can you define "huge amount of messages" during this 10-20 hours window?

Are you producing more messages than 1.5 GB per day (max queue size in Event Mesh is about that size)?

[Ivan-Mirisola]

Hi nandhini_91,

Both JMS Queue and Event Mesh have the exact same limitations:

https://blogs.sap.com/2017/10/04/cloud-integration-jms-resource-and-size-limits-in-cpi-enterprise-ed...

Remember that JMS Queue is internal to your Cloud Integration and its limit is shared across all integration flows.

With that in mind, what you have to figure out is when will your integration face a bottleneck in JMS Queue. It might be the case where you would have to use both (since you already have Event Mesh licensed anyway).

For robustness, you could pick messages from an Event Mesh queue and send it to a sub-process. If the sub-process fails, you could send it to JMS Queue for later retry. Once the retry count fails altogether, your could send it to a special queue in Event Mesh for later re-processing. You could event break messages into several different instances of Event Mesh based on sender or some other type of granularity that makes sense to you.

So, in short, it would be wise to use Event Mesh to store messages that require a longer period because you can have multiple instances of it on BTP. Whereas messages on the JMS Queue should have a short lifetime so you can optimize its usage across integration processes.

Best regards,
Ivan

[KeshavDesai]

Thanks for the reply Ivan.

If Event Mesh is used to store messages for retry after errors, and if the Integration Suite is on a Messages based Service Plan, wouldn't this approach impact the transaction volume and hence pricing? Would storing these messages in a Data Store be an option? Assuming a utility iFlow could re-process the messages in a Data Store, on a scheduled/ad-hoc execution.

Regards

[Ivan-Mirisola]

Hi c1e681854a8644c2988de87f78656e2b,

Cloud Integration's metric is outbound messages. Whereas Event Mesh's metric is storage for transactions and it is limited the same way as the internal JMS Queue.

The following excerpt corroborates the metric for CPI being outbound messages (from SAP Note 2942344😞

"In the context of Cloud Integration, a capability of SAP Integration Suite, outbound message size via one or more receiver adapters are considered for message metering"

Therefore, If you have enough storage to keep messages for retry on a queue, then they won't be counted until they are successfully delivered to destination.

I wouldn't recommend storing messages on a database for reprocessing (imho: message queues are more effective means).

I'd would only recommend persisting them at database level if the integration flow fails all reties and you want to develop a custom front-end for resubmiting the message.

Best regards,
Ivan