Batch Request with multiple operations on multiple...

sunny_kapoor · ‎2017 Nov 08

As a member of Customer Success Team, we are responsible for enabling/support customers and internal stakeholders to achieve their integration scenarios using SAP Integration Suite.

During the support, we encounter some tricky scenarios, which are not very specific to their use case but can be common to all.

This blog is an answer to one of such tricky question i.e.

How can I make OData Batch Request with multiple operations on multiple entity sets.

Customer need this capability majorly for the following use cases:

Customer wants to pack several functionally related API requests and send them to the web API service in one HTTP request and receive a single HTTP response with the response to all their requests. For example: Update in one entity set, also requires an upate in another functionally related entity set.

Customer want to pack several API requests and send them to the web API service in one HTTP request and receive a single HTTP response by creating their own OData service on SAP Cloud Integration. This new OData service then call existing OData service with mutiple operations on multiple entity sets using single batch request.This way, the client can optimize calls to the server and improve the scalability of its service.

OData Adapter in SAP Cloud Integration supports batch operation. To know more about the batch operation in OData adapter, kindly read the existing blog on OData Adapter.

As per the OData specification http://www.odata.org/documentation/odata-version-2-0/batch-processing/, a single batch request can contain multiple operations on multiple entity sets but the design time of OData adapter in Cloud Integration supports creation of a batch request with single operation on single entity set.

Nevertheless from a runtime perspective, the runtime of OData adapter in SAP Cloud Integration supports execution of a batch request with multiple operations on multiple entity sets.

Now the question is, if the runtime supports, then how we can achieve the same in Cloud Integration web tooling with its current set of capabilities. By the end of this blog, this question will be answered.

Example Scenario:

Consider a scenario in which we are getting a single payload, with data of multiple entity sets.

Using this single payload we make an OData batch request with multiple operations on multiple entity sets.

For this scenario, we are using the OData Demo V2 service

http://services.odata.org/V2/(S(wqb5nkyghvr4elijgf5vvb1f))/OData/OData.svc

This OData service is having three entity sets i.e. Products, Categories and Suppliers. We will make a single batch call with POST on Categories, PUT on Suppliers and MERGE on Products.

Integration Flow

The above integration flow is explained below:

Configure the Timer to Run Once

In Content Modifier, hardcode the initial payload in the body as given below:

<service_data>

	<post>

		<category>

			<id>3</id>

			<name>NewCategoryName</name>

		</category>

	</post>

	<put>

		<supplier>

			<id>1</id>

			<name>ChangedSupplierName</name>

			<address>

				<street>Whitefield</street>

				<city>Bangalore</city>

				<state>Karnataka</state>

				<zipcode>560066</zipcode>

				<country>INDIA</country>

			</address>

		</supplier>

	</put>

	<merge>

		<product>

			<id>2</id>

			<rating>4</rating>

			<price>24</price>

		</product>

	</merge>

</service_data>

This single payload contains the data for all three entity sets i.e. Products, Categories and Suppliers.

Then use Parallel Multicast(as order doesn't matter in this scenario), with three branches to get the batch request compliant payload for each entity set and operation

In each branch we use Message Mapping with initial payload XSD as the source and XSD of each batch operation as the target to get the batch request compliant payload for each entity set and operation

Initial Payload XSD

<xs:schema attributeFormDefault="unqualified" elementFormDefault="qualified" xmlns:xs="http://www.w3.org/2001/XMLSchema">

  <xs:element name="service_data">

    <xs:complexType>

      <xs:sequence>

        <xs:element name="post">

          <xs:complexType>

            <xs:sequence>

              <xs:element name="category">

                <xs:complexType>

                  <xs:sequence>

                    <xs:element type="xs:byte" name="id"/>

                    <xs:element type="xs:string" name="name"/>

                  </xs:sequence>

                </xs:complexType>

              </xs:element>

            </xs:sequence>

          </xs:complexType>

        </xs:element>

        <xs:element name="put">

          <xs:complexType>

            <xs:sequence>

              <xs:element name="supplier">

                <xs:complexType>

                  <xs:sequence>

                    <xs:element type="xs:byte" name="id"/>

                    <xs:element type="xs:string" name="name"/>

                    <xs:element name="address">

                      <xs:complexType>

                        <xs:sequence>

                          <xs:element type="xs:string" name="street"/>

                          <xs:element type="xs:string" name="city"/>

                          <xs:element type="xs:string" name="state"/>

                          <xs:element type="xs:int" name="zipcode"/>

                          <xs:element type="xs:string" name="country"/>

                        </xs:sequence>

                      </xs:complexType>

                    </xs:element>

                  </xs:sequence>

                </xs:complexType>

              </xs:element>

            </xs:sequence>

          </xs:complexType>

        </xs:element>

        <xs:element name="merge">

          <xs:complexType>

            <xs:sequence>

              <xs:element name="product">

                <xs:complexType>

                  <xs:sequence>

                    <xs:element type="xs:byte" name="id"/>

                    <xs:element type="xs:byte" name="rating"/>

                    <xs:element type="xs:byte" name="price"/>

                  </xs:sequence>

                </xs:complexType>

              </xs:element>

            </xs:sequence>

          </xs:complexType>

        </xs:element>

      </xs:sequence>

    </xs:complexType>

  </xs:element>

</xs:schema>

Hint: To get the XSD of each batch operation along with the entity set and it's properties, configure OData Adapter one by one for:

a. POST on Categories,

b. PUT on Suppliers, and

c. MERGE on Products

This will generate three batch operation compliant XSD files i.e. CategoriesEntityPOST0.xsd, SuppliersEntityPUT0.xsd and ProductsEntityMERGE0.xsd.

Now Join all the three branches followed by the Gather step to get the single batch request payload

This will generate the cumulative batch request payload with single batchChangeSet for all the three operations as given below:

<batchParts>

   <batchChangeSet>

      <batchChangeSetPart>

         <method>POST</method>

         <Categories>

            <Category>

               <ID>3</ID>

               <Name>NewCategoryName</Name>

            </Category>

         </Categories>

      </batchChangeSetPart>

      <batchChangeSetPart>

         <method>PUT</method>

         <headers>

            <header>

               <headerName>If-Match</headerName>

               <headerValue>W/"0"</headerValue>

            </header>

         </headers>

         <Suppliers>

            <Supplier>

               <ID>1</ID>

               <Name>ChangedSupplierName</Name>

               <Address>

                  <Street>Whitefield</Street>

                  <City>Bangalore</City>

                  <State>Karnataka</State>

                  <ZipCode>560066</ZipCode>

                  <Country>INDIA</Country>

               </Address>

            </Supplier>

         </Suppliers>

      </batchChangeSetPart>

      <batchChangeSetPart>

         <method>MERGE</method>

         <Products>

            <Product>

               <ID>2</ID>

               <Rating>4</Rating>

               <Price>24</Price>

            </Product>

         </Products>

      </batchChangeSetPart>

   </batchChangeSet>

</batchParts>

Then call the OData endpoint using OData Adapter using Request-Reply flow step.

Important Note: The OData Adapter can be configured with any batch operation. In design time selecting a particular entity set and batch operation doesn't matter as the call will happen based on the previous cumulative batch request payload.

Finally the OData batch request response is captured as an XML file in SFTP Server

<batchPartResponse>

    <batchChangeSetResponse>

      <batchChangeSetPartResponse>

        <headers>

          <Accept-Language></Accept-Language>

          <DataServiceVersion>1.0;</DataServiceVersion>

          <Accept></Accept>

          <Cache-Control>no-cache</Cache-Control>

        </headers>

        <statusCode>204</statusCode>

        <body/>

        <contentId/>

        <statusInfo>No Content</statusInfo>

      </batchChangeSetPartResponse>

    </batchChangeSetResponse>

  </batchPartResponse>

This way we can make a single batch request with multiple operations on multiple entity sets.

With the learnings of this blog, I recommend you to try this capability in OData Provisoning (ODP) i.e. create a new OData service from existing OData service and internally make a batch request with muiltiple operations on multiple entity sets.

brunoambrozio · ‎2018 Jan 26

Hi sunny.kapoor2,

Thanks for your time preparing this blog. It's very nice and made clear some doubts that I've been looking for.

I have a demand to build an iflow to add and update dependents on SFSF. In order to achieve it, it's necessary to call 5 entities: PerPerson, PerRelationship, PerPersonal, PerNationalId and PerGlobalInfoBRA.

I have created the iflow (below screenshot), it's working fine, but I would like to know what is the better way to control the rollback in case some of these entities fails.

Could you please advise me about how to use the rollback procedure?

Regards,

Bruno

sunny_kapoor · ‎2018 Feb 02

Hi bruno.brigantiniambrzio

If you are asking this question in context of OData batch request, then in a single batch request you can have multipe ChangeSets. A single ChangeSet can contain multiple operation on multiple entity sets.

All operations in a ChangeSet represent a single change unit so the server must successfully process and apply all the requests in the ChangeSet or else apply none of the requests in the ChangeSet. It is up to the service to define rollback semantics to undo any requests within a ChangeSet that may have been applied before another request in that same ChangeSet failed and thereby honor this all-or-nothing requirement.

So in your case you can put all related operation in single ChangeSet, so that if anything goes wrong with any of the opertaion, none of the requests in the ChangeSet should apply.

Hope I am able to answer your question.

Regards,

Sunny

brunoambrozio · ‎2018 Feb 02

Hi Sunny,

Thanks for your reply. Your explanation was very useful.

We have a critical point for these entities: all of them accept only UPSERT and QUERY operations. Following SFSF API, CREATE and UPDATE are not allowed for them and up to my knowledge, OData batch request does not accept USPSERT, right? I've done a test right now and Odata batch response confirmed it.

When setting the SuccessFactors Adapter for UPSERT these entities, there is an option to flag "Enable batch processing". I've also tried it, but the XSD is the same generated when choosing OData Adapter and it also didn't work.

Do you have another idea?

Thanks,

Bruno

former_member183597 · ‎2020 Mar 26

Hi Sunny,

Thank for article, I'm trying to implement the same but sending an OLINGO Batch script via CPI ODATA V2 Receiver Adapter. While doing that I'm getting this error "

Batch Request of mine look something like this. Does the ODATAV2 Adapter in CPI support this kind of batch request.

--batch_34fcd829-64e4-4139-b880-f24aa4ee9235
Content-Type: multipart/mixed; boundary=changeset_a6a10db7-ee1c-4e55-ab6e-4e65a808c683

--changeset_a6a10db7-ee1c-4e55-ab6e-4e65a808c683
Content-Type: application/http
Content-Transfer-Encoding: binary

PUT StringParameters(Pid='XXXXXXXXXXX',Id='TimeZone')?user=Vinay HTTP/1.1
Content-Length: 64
Accept: application/json
Content-Type: application/json

{"Value":"America/Chicago","Pid":"XXXXXXXXXXX","Id":"TimeZone"}
--changeset_a6a10db7-ee1c-4e55-ab6e-4e65a808c683--
--batch_34fcd829-64e4-4139-b880-f24aa4ee9235--

former_member739123 · ‎2021 May 20

Hi Sunny,

If we are using multiple entities upsert through one odata V2 adapter.

What would you keep for below Processing Details tab feilds:

Operation details,fields and other fields in below screenshots differently.

Also, what needs to be mapped at below xml fields in mapping:

xsd part of request batch

Regards

Deepak

sunny_kapoor · ‎2021 May 20

Hi d_jaiswal

In Query Wizard if you select the corresponding entity and the operation you can select what all fields you need to set and then you can automatically get the corresponsing xsds for the same.

Using the XSDs you can then do the mapping and get the single batch payload as depicted in the blog post.

Regards,
Sunny

former_member690710 · ‎2021 Jul 18

I am trying to follow this but I am getting error. My Iflow is as follows.

My content modifier body has payload like below.

<batchParts>
<batchChangeSet1>
<batchChangeSetPart1>
<method>POST</method>
<uri>Categories</uri>
<body>
<Categories>
<Category>
<ID>956</ID>
<Name>Laptops</Name>
</Category>
</Categories>
</body>
</batchChangeSetPart1>
</batchChangeSet1>
</batchParts>

I am using Batch operation

I am not getting what is the issue.But its not working I am getting error as

java.lang.IllegalArgumentException: Invalid content type

Can you help me here

Batch Request with multiple operations on multiple entity sets using SAP Cloud Integration OData Adapter

Example Scenario:

Integration Flow

Get Your SAP HANA Idea Incubator Badge Today!

SCN Mission - SAP HANA Quiz Challenge is now retired

Share your #HANAStory and Win