EOIO in SAP CI Using JMS - Exclusive Mode

[vinaymittal]

Introduction

EOIO was a long awaited functionality that every Integration consultant was waiting for in SAP CI. We had this in SAP PO where we would use it to preserve the order of messages. Messages using EOIO were stored in a FIFO queue that ensured the messages that arrive first in the middleware is also the first to leave. The order was ensured even in case of outages on the receiver system and the middleware would continue to retry the failed messages in the context of FIFO.

Solution

SAP has recently released JMS Adapter Adapter version 1.4, that has the Exclusive mode which ensures that the message order is preserved.

Both JMS Sender and Receiver adapters have a new option now - Access type - Exclusive / Non-Exclusive 

For normal scenarios where the Order in which the messages are processed doesn't matter it's highly recommended to use non-exclusive setting in Access Type. Using "Exclusive" as Access type will enforce EOIO and takes away the option of parallel processing which ultimately means that as soon as the receiver system is up messages will get processed one by one and not parallelly. 

Let's dive in to the POC now.

Here we have a integration process that has this message which is split into 5 individual messages and will be sent to a EOIO JMS Queue 

Content Modifier has the following message

<Root>
  <Message>
    <id>1</id>
  </Message>
  <Message>
    <id>2</id>
  </Message>
  <Message>
    <id>3</id>
  </Message>
  <Message>
    <id>4</id>
  </Message>
  <Message>
    <id>5</id>
  </Message>
</Root>

 We are splitting this and sending individual Message nodes to the JMS Queue, Parallel processing is unchecked in the splitter

  

 The Integration process runs and creates 5 messages in the JMS Queue 

Now SAP CI iflow will automatically retry these as the receiver systems URL is a dummy URL which doesn't exist

As expected only the very first message gets Retried and none of the other messages in the queue are picked up.

A word of caution: The way SAP PO XI adapter EOIO worked was that SAP ECC/S4 has EOIO queues as well, the EOIO Quality of service is communicated by the sender system to the middleware. In fact the backbone of EOIO on SAP PO was that the sender system itself was responsible for ensuring that messages are sent out in the exact order they are generated!

AFAIK SAP CI , XI adapter doesn't support EOIO yet! SOAP RM doesn't support it either.

If you are using a SAP CI, XI sender adapter on EO QOS in SAP S4 (as you can't use EOIO - SAP CI wont understand it) , then SAP S4/ECC in case SAP CCI is briefly unavailable will mess up the order in its internal queues! The messages will arrive to SAP CI, XI sender adapter out of order.

If you use SOAP RM Sender adapter then definitely you are not getting EOIO from the source system.

True EOIO can only be achieved if Either SAP CI is pulling the records  from SAP S4 OR the sender system ensures that messages go out in EOIO fashion even in case of an outage on the middleware or network latency.

Where can this be used

This can be used in 

1) SAP to JDBC or JDBC to SAP Scenarios
Integrations consultants use JMS queues for persistence in SAP CI via JMS Queues, If the JDBC receiver is not available then there is a risk that the messages when retried will be retried out of order. This can cause problems on the database side for some scenarios. To mitigate this using EOIO would be a great option. 

2) Integrations where data is transactional and the integration is Asynchronous and the messages are persisted in SAP CI in case of failure for automatic Retries.

These are scenarios where let's say exchange rate data is being passed from SAP to a receiver system asynchronously and the receiver system is not available and due to some reason exchange rates are fluctuating by a lot that day If out of order exchange rates are sent to the receiver system then this may cause problems. To avoid such issues in critical financial integrations EOIO in SAP CI may be an option.

3) In Scenarios where IDOCS are collected in SAP and Sent to SAP CI in bulk - 1000 and

a) Are for example stored in data store for correlation - Banking Payments , and you are worried that hundreds of data store writes in milliseconds will crash the system.
or
b) As soon as one of those IDOCs arrives in SAP CI, SAP CI calls an API to enrich or do a lookup or calls end system and you are worried about overloading the end system. 

That being said, this will never be true EOIO unless supported by SAP S4. 

Still This is a very welcome update from SAP. This will bridge the gap between SAP PO and SAP CI where consultants had to design workarounds to implement EOIO.

c) for low priority high volume Asynchronous interfaces