Difference between direct RFC call and calls via RFC receiver adapter in SAP Cloud Integration

[vikas2]

Introduction

RFCs (i.e. Remote Function Calls ) have been used in SAP landscapes for a long time and are one of the easy ways to communicate with SAP system . There are many other options now and they all have their benefits.

In many cases external products have been built utlizing RFC technology by different vendors with one of the most common being SAP JCO.

https://support.sap.com/en/product/connectors/jco.html

From documentation page:

The SAP Java Connector (SAP JCo) is a development library that enables a Java application to communicate with SAP systems via SAP's RFC protocol. The SAP JCo supports both communication directions: inbound Remote Function Calls (Java calls ABAP) as well as outbound Remote Function Calls (ABAP calls Java).

I have previously built Java and shell script based solutions using SAP JCo and it has proven to be an effective workhorse in integration landscapes with very demanding requirements.

Recently,  we onboarded a software from a vendor which has created solution utlizing RFCs . The vendor has so far implemented solution for its clients implementing the RFCs directly. However, we want all access to backend S4 SAP system to come through our middlewares with Cloud Integration being the preferred one.   However, the RFC calls were not returing required data when using RFC receiver adapter in SAP Cloud Integration.

This blog provides my experiences in :

1) Providing deeper understanding in RFC receiver adapter

2) Solution when the standard RFC receiver adapter doesn't work.

Landscape:

It's pretty standard setup with Application calling the RFC in S4 SAP system via SAP Cloud Integration and SAP Cloud Connector.

Sample RFC:

To illustrate the issue for clarity, I'll demonstrate via a simpler RFC and not the actual RFC with which we had the issue.

We have a simple RFC . It has one input and two out parameters.

IM_INPUT : Input Parameter is a string.

Two output parameters:

1) EX_OUTPUT_1: Input Param concatenated by a normal separator ( underscore as an example ).

2) EX_OUTPUT_2: Input Param concatenated by ASCII31 as a separator . ( Reference https://www.ascii-code.com/31 ).

The below execution shows the execution . Even though the RFC standalone execution shows # as the separator, the value is intact ( check in debug ) . It's just the display which shows # - so there is no issue on the SAP side for this RFC.

RFC code for reference - it is concatenating the input parameter and user timezone separated by different delimiters. A "normal" delimiter for underscore ( '_' ) with ASCII code 95 and then using a non-printable character 31.

*ASCII 95 represents underscore and ASCII 31 represents the non-printable unit separator

  ex_output_1 = im_input && cl_abap_conv_in_ce=>uccpi( 95 ) && sy-zonlo.
  ex_output_2 = im_input && cl_abap_conv_in_ce=>uccpi( 31 ) && sy-zonlo.

RFC Integration Flow:

All good so far. Let's create a simple Integration Flow using standard RFC receiver adapter to test out this call . It's a very simple flow, taking the input parameter ,mapping to required RFC format , making the RFC call using the standard RFC receiver adapter and returning the response to the sender application.

This is where it starts to get interesting . On execution RFC response is blank !

<?xml version="1.0" encoding="UTF-8" standalone="no"?><rfc:Z_RFC_TEST.Response xmlns:rfc="urn:sap-com:document:sap:rfc:functions"/>

To rule out error with any config on any of the existing parts, if I change the delimter for both output parameters, it works.

<?xml version="1.0" encoding="UTF-8" standalone="no"?><rfc:Z_RFC_TEST.Response xmlns:rfc="urn:sap-com:document:sap:rfc:functions"><EX_OUTPUT_1>Hello_AUSNSW</EX_OUTPUT_1><EX_OUTPUT_2>Hello_AUSNSW</EX_OUTPUT_2></rfc:Z_RFC_TEST.Response>

Now, I was thinking if RFC adapter has some level of customization which allows us to overcome this issue but there is nothing obvious. I see only two parameters:

Send Confirm Transaction : Required if BAPI_TRANSACTION_COMMIT is needed to implicit commit a transaction. As our RFC is only reading information, this is not needed . I tried and obviously added SAP CC to allow  BAPI_TRANSACTION_COMMIT and BAPI_TRANSACTION_ROLLBACK transactions but it is not needed.

Create New Connection: This is needed if we need to create a new connection ( when using prinicipal propagation ).   I tried various combinations of these two as these are the only parameters which can somewhat influence the behaviour but discarded them as they're of no use.

Cloud connector can have normal settings and the behaviour is still the same.

Can BTP destination settings help ?

The below properties were used to create the RFC destination in BTP Cockpit.

I didn't find any parameter which can used to influence this behaviour.

https://help.sap.com/docs/connectivity/sap-btp-connectivity-cf/target-system-configuration#content-0

RFC call is getting the proper request from the client.

<?xml version="1.0" encoding="UTF-8"?>
<ns0:Z_RFC_TEST xmlns:ns0="urn:sap-com:document:sap:rfc:functions"><IM_INPUT>Hello</IM_INPUT></ns0:Z_RFC_TEST>

This is where it gets very interesting . RFC adapter actually retrieves the proper result from the backend SAP system !

<Z_RFC_TEST><INPUT><IM_INPUT>Hello</IM_INPUT></INPUT><OUTPUT><EX_OUTPUT_1>Hello_AUSNSW</EX_OUTPUT_1><EX_OUTPUT_2>Hello&#31;AUSNSW</EX_OUTPUT_2></OUTPUT></Z_RFC_TEST>

However, the calling application never gets the data back !

<?xml version="1.0" encoding="UTF-8" standalone="no"?><rfc:Z_RFC_TEST.Response xmlns:rfc="urn:sap-com:document:sap:rfc:functions"/>

It seems after getting the data back from SAP S4 system the adapter is:

1) Extracting only the response to be sent to next step in the flow ( which is fair ) 

2) Creating a XML with namespace and doing validations ( which again is fair ) .

Non-Printable Characters in XML Files

Non-printable characters are low-order ASCII characters in the ranges that are described as follows:

#x0 - #x8	ASCII 0 - 8
#xB - #xC	ASCII 11 - 12
#xE - #x1F	ASCII 14 - 31

An example of such a character would be CTRL-Z (ASCII value 26).

The World Wide Web Consortium (W3C) XML language specification dictates that non-printable characters are not legal in an XML file. 

Normally if someone even asks me about non-printable characters in XML, my response is shock and horror. We like to keep XML and non-printable characters in mutually disjoint sets.

Microsoft agrees as well .

https://learn.microsoft.com/en-us/previous-versions/troubleshoot/msxml/error-when-xml-contains-low-o... 

However, what is  fair seems to trip the XML parser - which is fair but doesn't help us.

Now, before moving forward someone is thinking, why don't we just replace the separator. Good idea - however, it's an addon provided by a company and as it's a product, it has to get into backlog before the delimter can get changed.

Digression : I heard an interesting discussion in a talk where functional programming was described in a way that the functions should be idempotent ( is the property of certain operations in mathematics and computer science whereby they can be applied multiple times without changing the result beyond the initial application ) . However, those are mathematical functions and software developers are paid for creating side effects - updating a record, creating a file etc . The user doesn't care if our programs are functional or not. They pay for the side effects . 

Similarly, carrying a non-printable character in a XML may not be a great idea but software developers live in the real world where the query is - when we call the SAP RFC directly, it works. You're adding integration layer in between and don't want external systems to call SAP directly, can you just make it work ?

Software development is often characterized by inelegant solutions to wicked problems. With that thought, I move on and started thinking of solutions.

Levels of Abstraction:

Now, all developers work at some level of abstraction. Cloud Integration RFC receiver adapter makes our life easy but doing everything and give us a nice XML in the integration pipeline. However, in this case it's at a higher level of abstraction and we need to do what RFC adapter is doing under the hood.

Solution:

We create a simple iflow with using groovy script and the script makes the RFC call .

Add property for RFC dest, RFC name and map the input parameter.

Things are a bit different compared to SAP PO where we could extract the source code ( get the JAR -> decompile -> get an idea on inner workings ) . So we have to fall back on using some kind of manual programming to call the SAP RFC.  We create a Groovy Script.

Groovy Script:

    import com.sap.gateway.ip.core.customdev.util.Message
    import com.sap.conn.jco.*

    def Message processRFCCall(Message message) {
        def destinationName = message.getProperty("RFC_DEST")
        def rfcName = message.getProperty("FM_NAME")



        try {

            JCoDestination jcoDestination = JCoDestinationManager.getDestination(destinationName)
            JCoFunction function = jcoDestination.getRepository().getFunction(rfcName)

            if (function != null) {
                def parameterList = function.getImportParameterList()

                message.getProperties().each { key, value ->
                    if (key.startsWith("IM_INPUT")) {
                        def paramName = key
                        try {
                            parameterList.setValue(paramName, value)
                        } catch (Exception e) {
                            throw new Exception("Error setting parameter '${paramName}': ${e.getMessage()}")
                        }
                    }
                }






                function.execute(jcoDestination)

                def result = function.toXML()
                message.setBody(result)
            } else {
                throw new Exception("ERROR: RFC function '${rfcName}' not found")
            }

        } catch (JCoException e) {
            throw new Exception("ERROR executing RFC function: ${e.getMessage()}")
        } catch (Exception e) {
            throw new Exception("Unexpected error: ${e.getMessage()}")
        }

        return message
    }

I have done last SAP JCO development with bash shell scripts and  Java programs around 6 years back for use by thousands of adapters. 

This code in CPI is much smaller due to:

1) Using the RFC destination created in BTP Cockpit and configurable in the flow. I like the separation of all the destination / JCO configuration being in BTP Cockpit instead of it being part of code.

2) Groovy being less verbose.

Let's test the new flow.

Groovy Script is being supplied similar parameters.

<?xml version="1.0" encoding="UTF-8"?>
<ns0:Z_RFC_TEST xmlns:ns0="urn:sap-com:document:sap:rfc:functions"><IM_INPUT>Hello</IM_INPUT></ns0:Z_RFC_TEST>

After script execution , RFC comes back with the whole RFC ( with input and output parameters ) . If required, we can do futher processing to remove request structure, add namespace etc.

<Z_RFC_TEST><INPUT><IM_INPUT>Hello</IM_INPUT></INPUT><OUTPUT><EX_OUTPUT_1>Hello_AUSNSW</EX_OUTPUT_1><EX_OUTPUT_2>Hello&#31;AUSNSW</EX_OUTPUT_2></OUTPUT></Z_RFC_TEST>

This demonstrates that the output result is extracted even if it has non-printable ASCII character.

Conclusion:

Normally, we stay at the level of abstraction provided by standard Cloud Integration Adapter. This blog demonstrates that in RFC context, we can go under the hood and make things work for us in Cloud Integration.