SAP CPI - Complex integration with with SAP BRM - SFTP

[rhviana]

Hello Folks,

The integration seems to be simple, SFTP -->ASYNC --> SAP CPI --> SYNC --> SAP BRM --> SAP CPI --> ASYNC --> SFTP, so looks simple for the integration interpretate like that, but let's go together for spider web complex integration.

Basically the integration will be better explain but lets for short description to make clear the integration diagram, the SAP CPI is responsible to read a JSON payload from SFTP folder, without pretty printer format with multiple records of data in this payload.

After read the payload SAP CPI is responsible to split the records per line, break it into multiple different parts generating the respective of SAP BRM - json format - for each service for business rules management table translation collect all results and merge the full results from SAP  BRM plus the input payload read from the SFTP in a Hugh multimapping XML file with multiple records and context change from each line from input payload and in the end a groovy script of 200 lines is doing the last step of the transformation from XML to JSON and producing output json expected by the receiver system in another folder in the SFTP service that you be read from the respective receiver system and process the message.

The integration diagram perspective:

Spoiler

Is that clear and comprehensive for you until now ? I hope yes. 😎

What is SAP BTP BRM ?

SAP Business Rules Management (SAP BRM) enables organizations to automate decisions by using business rules. Business users participate in and control rule definition, while business process experts model, validate, deploy, update, and archive business rules through their lifecycle. As such, IT organizations can work with business users to manage business rules that drive process flow and execution.

SAP CPI Value Mapping vs SAP BRM:

The details between those two mechanisms of data translation.

SAP BRM:

  Business rules and user, dynamically change based on business needs or directions direct in SAP BRM tables, changeable all the time.

  Control:

  Business Rules Consultants

  Role:

  Business specialist

  Dynamically Changeable : 

Yes, any time from Business Rules Consultant without interaction with SAP CPI developer.

SAP CPI:

The value mapping is used to translate values between various formats or representations. Value mapping can be used to map language codes, units of measurement, product codes, order kinds, and explanation codes, for instance.

  Control:

  The CPI Developer consultant based in the project request from the business translation requirements

  Role:

  Technical

  Dynamically Changeable :

  No, is state based on the insert parameters made by CPI Developer or doing the configuration direct in the value mapping in SAP CPI Production object, but the control is over technical SAP CPI Developer.

Development suppress details:

Over this part I will explore the development suppressing information's from to avoid any exposition from my company, so the samples of payload and others it's dummy data basically.

The Iflow integration perspective image:

Off course over this part of the blog, It's not make sense provide all logics or steps or why I developed in this way because is not the focus here, my intention is present you the usage of SAP BTP BRM instead of SAP CPI Value mapping, so let's take a look in the description below, where I will add some information's through the Iflow it self.

• First step:

Convert the JSON to XML than use the SPLITER function per each line record.

In this case I developed a groovy script for this conversion because the standard function ( JSON to XML Convertor) it was not working proper, because the sample of files I received it was without clear pattern of root elements in the JSON input payload, sometimes with Documents as start point sometimes with ID, to solve it and don't block progress of the project I decide for that.

• Second step:

After the Splitter per record, the Iflow send the message to local process flow where Content-Modifier saves the original payload and parallel multicast the services to SAP BRM and also some local transformations as you can see below:

Spoiler

I hope until now you understand technically development aspects, let's keep going ! 🤗

SAP BRM - Payload expected:

As I explain before, for each table into SAP BRM, the payload looks very similar but with some elements from the JSON specific to the services that will be consumed, the sample of payload before is a dummy manually created to avoid sensible information's.

I develop a groovy script with 375 lines to generate the expect payload per services ( 7 different services plus two transformations outside need call SAP BRM) as a dummy sample for you below:

{
    "RuleServiceId": " ID UNIC CODE FROM THE SERVICE FROM SAP BRM",
    "Vocabulary": [
        {
            "Service_One": {
                "Dummy_Service_One": " Value from the XML position from inbound record"
            }
        }
    ]
}

The response from SAP BTP BRM as a sample presentation, is like that:

{
    "Result": [
        {
            "Service_One": [
                {
                    "Result_1": "value",
                    "Result_2": "value",
                    "Result_3": "value"
                }
            ]
        }
    ]
}

Sample for language translation:

Problem:

During the test I phase this problem.

As you can see in the picture, there is a PARALLEL Multicast splitting the XML structures and the groovy generating the XML SAP BRM Payload per service, I developed one groovy with multiple functions by name, but when I used the Read ( inputstream ) over the same groovy with multiple functions called at same time.

SAP CPI pop up error, I read the payloads as String, it was the fasts solution to achive the target delived date also I didn’t want develop one groovy per each SAP BRM definition, not practical in my point of vew.

Solving the problem:

groovy.lang.GroovyRuntimeException:

Ambiguous method overloading for method groovy.util.XmlSlurper#parse. – OK clear this error multiples variables def xml.

To solve the problem I need to create read and xml variables with different names for each function in one groovy, I decide go read the body as string instead break in 8 groovies scripts or 8 different variables different that during runtime will not overload the same variables with multiples different payloads, I just took this decision because of timeline of project and taking consideration the size of input payload splited by the content modifier using XPATH was really small.

Integration protocol with SAP BRM:

Basically to integrate with SAP BRM is HTTPs with Token and Basic Authentication over one address only, the identifier of SAP BRM is based on the format of payload based on the service.

• Third step:

After receive and collect all responses from SAP BRM, as you could see the response is JSON, I apply in this case JSON to XML Converter for each response and after that a groovy script ( I could use XSLT ) to suppress the XML header, because after the Gather mechanism there was error in the format of XML because of that ( <?xml version="1.0" encoding="UTF-8"?> )

• Fourth step of development:

I used the function to JOIN all results.

• Fifth step of development:

I used the gather to merge up all results from SAP BRM and local transformations that not need response from BRM

• Sixth step of development:

Content-Modifier generating the full result from the gather, together with original payload from incoming message in the Iflow:

• Seventh step of development:

Another gather after the result from the local process responsible to consume SAP BRM plus local transformation generate a full Multimapping XML with all content and records reader from SFTP per file.

• Eighth step of development:

The complex groovy to read all different contexts from the result from SAP BRM plus the input payload per line into the file, controlling indexes and others, basically around 200 lines to produce the output file, in this case off course, I'm reading the body as inputstream to don't overload the memory in SAP CPI node.

Spoiler

I hope until here you understand and survive, let's go for test ? 🤗

Test phase:

In this part I'm going to present some short cuts of payload, iflow prespective and result

Input payload sample ( Off course detailed information is surpress this is just sample ), not imagine a input payload with 2000 thousand of files with those details below and each of this lines it's different logic, service SAP BRM call, rules, complex business transformations.

Sample of file with one record:

{
    "id": 587,
    "baseId": null,
    "versionNumber": 1,
    "versionHistoryId": 187,
    "releaseDate": "2022-05-16T13:00:04.663801+02:00",
    "lastUpdate": "2024-01-03T19:04:02.014922+01:00",
    "metadata": {
        "2": "",
        "5": "",
        "6": {
            "id": 289
        },
        "8": "",
        "9": "2022-06-21T22:00:00Z",
        "10": {
            "id": 4
        },
        "11": {
            "ids": [
                421
            ]
        },
        "13": {
            "ids": [
                465,
                466,
                475,
                484,
                485,
                486
            ]
        },
        "19": "",
        "24": "",
        "25": ""
    },
    "topicIds": [
        69,
        110
    ],
    "attachmentFileNames": [
        "587_English_Original_Full Version_3669_UN-R_14-09_S2_EN_2022-05-12.pdf"
    ]
}

Original test file:

Input payload folder file:

Processing the message in SAP CPI:

Output folder result transformation:

Result content:

The content for each line from input payload generates and regulations structure for each, in the sample presented is only the first one:

Conclusion:

The intention of this blog is present the integration between SAP CPI and SAP BRM, translation, this is a purely business decision, because of that I didn't explore the chance to use SAP CPI Value Mapping.

I really hope that you enjoy.

Kind regards,

SAP Integration Expert - Viana.

[MustafaBensan]

Hi @rhviana,

In this blog post, I noticed that the term SAP BRM appears to be used interchangeably with SAP BTP BRM.  Since "SAP BTP BRM" isn’t a service on BTP, could you perhaps clarify the terminology to help readers easily understand the context?  Instead of SAP BTP BRM, do you actually mean the classic SAP BRM on NetWeaver?  An update to this blog post might be helpful to avoid any potential confusion.

[rhviana]

Updated for correct terminology.

I mean SAP BRM in the Cloud.

[MustafaBensan]

@rhviana, I am still a little confused by what you mean by "SAP BRM in the Cloud".  SAP does not offer SAP BRM as a cloud product.  Are you just referring to the scenario where the customer is hosting the classic NetWeaver SAP BRM on a hyperscaler?

[rhviana]

@MustafaBensan i believe this blog is more into the complexity of the integration it self and yes as per of  SAP Cloud service,

Please learn over the blog above:

https://community.sap.com/t5/technology-blogs-by-members/setting-up-sap-business-rules-service-in-th...

If your doubts is in relation of this BRM service, I can't help you.

Thank you.