#CoE#MKI How to send Keep-Alive to SAP BTP

[k-fuji2458]

Dear SAP,

The other day, in an OSS incident, we received an answer that the session of HTTP requests sent to BTP expires in 10 minutes, so we need to send Keep-Alive packets periodically to maintain the session.

We have tried to send Keep-Alive packets by modifying the registry of the Windows Server we are using, but it is not working and we are still experiencing a timeout after 10 minutes.

I was wondering if you could tell me what is the best way to send a Keep-Alive in order to solve this problem.

Best Regards, Kento

[Ivan-Mirisola · Accepted solution]

Hi kento_fujita,

Keep-Alive is not your answer. It has to do with web page loading performance. Please read:https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Keep-Alive
https://www.tunetheweb.com/performance/http-performance-headers/keep-alive/
What you are probably looking for is a way to avoid the application session from expiring. You could achieve this by make calls to a ping service in the server so it maintains your session valid.

function pingSAP() {
    var req= new XMLHttpRequest();
    req.open('GET', "/sap/bc/ping"); //Some server resource that does nothing other than return 200 to the browser
    req.send(null);
}

setInterval(pingSAP, 540000);  //Since BTP session expires after 10min, you could make a request every 9min

Hope this helps.Best regards,
Ivan

[k-fuji2458 · Accepted solution]

Hi ivan.mirisola ,

Thank you for contacting us.

Sorry for the lack of specific description, but my question is not about the communication from SAP BTP to S/4HANA Cloud, but about the connection from external system (we are using Asteria Worp) to Java application deployed in BTP.

The code you gave looks like it sends a ping to S/4HANA Cloud from a browser or something, is that correct?

Also, since the requirement is to keep waiting for the HTTP request sent to the Java Servlet in BTP, I feel that I have to send Keep-Alive packets instead of sending a new HTTP request.

Best Regards, Kento

[Ivan-Mirisola · Accepted solution]

Hi kento_fujita,

Sorry, I was assuming you were serving pages on BTP via a Java Application. Apparently, you are serving an API via Java Application whereas Asteria Warp will be your data integrator/message broker.

Now, from your statement related to wait time for the Java Servlet in BTP I take it that you are trying to achieve something else in your integration scenario. It seems to me that you expect your Java Servlet to keep on getting data indefinitely or for a very long time - just like Apache Kafka receiving data streams.

BTW: did you know that SAP Data Intelligence does exactly what you need takes the same development approach used by your message broker. Check this out:

https://blogs.sap.com/2020/04/30/sap-data-intelligence-3.0-walk-through/

https://blogs.sap.com/2021/09/20/sap-data-intelligence-cloud-pipeline-debugging/

As said before, keep-alive doesn't do what you think it does. It is more related to performance of web pages. Here is a brief explanation: You have multiple resources to load on a page (css, html, jpg, etc.) and they exceed the maximum amount of tcp open connections at the OS level (browser). Here the keep-alive will keep the tcp connection opened while your server is able to send multiple resources to your browser by reusing them. Hence, this will increase performance on your browser - which would have to wait for a tcp socket to be released in order to get more web page resources. The link I shared with you explains that in detail and how it increased performance overall.

Furthermore, HTTP protocol was never meant for that type of integration scenario. It was meant to serve resources over the internet the fastest possible. Which means that what you are trying to achieve goes against the protocol's intentions.

I believe that what you are looking for is WebSockets. With this protocol you can develop your Java application to establish a bidirectional communication with the calling application (Asteria Warp) so that it can keep on sending data through a tcp connection until either end decides to close it. The communication starts via HTTP protocol (so you can perform other actions such as login with the server) and then it proceeds with a simple tcp connection between peer and server. Most common scenarios nowadays is web chat applications - or even web development tools that implement a system terminal window such as SAP BAS to interact with the OS.

I'm not sure which framework you've used to develop your Java Application. If you are using a pure Java implementation, may I suggest to you Spring Boot. It is a very well known framework and it will ease the development task with such complex application.

The following blog show you how to use WebSockets with Spring Boot. It just the server piece plus a html client - not a message broker such as Asteria. But this is just to understand how the whole thing works in a simple scenario:

https://www.baeldung.com/websockets-spring

Now, here is another blog that will show you how to consume your java api using postman. Here postman can be any client for this java api - I am assuming WebSockets is supported on Asteria just the same:

https://www.baeldung.com/postman-websocket-apis

Hope this helps.

Best regards,
Ivan

[k-fuji2458 · Accepted solution]

Hi ivan.mirisola,

Thank you very much for your detailed instructions.

I understand that it is against the intent of the protocol to communicate using the HTTP method in the first place when you need to wait for a response from a program that takes a long time to process.

I also understand that I was wrong about the purpose Keep-Alive packets should serve.

Now I need to think about the specific steps I need to take to fix my program. Am I right that I should use WebSocket instead of the HTTP method to execute long processes that take more than 10 minutes?

[Ivan-Mirisola · Accepted solution]

Hi kento_fujita,

A long running process in that should return something via HTTP could of of the approaches below:

1) You trigger the data processing via a single request asynchronously. Then you either keep pooling the api to check it it has done the process or preferably the application makes a call-back to the integrator to continue its processing once it is done.

2) Check what is causing the bottleneck at the Java process. A 10min delay for a response should not occur in most cases. You either have too much processing at the application layer or your database server is not optimized to perform the query it needs. Database queries should be very quick to start producing some data. However, you may be missing some indexes or some pre-processing of data so your Java application doesn't have to chew on the data instead.

3) You trigger the data processing via a single request asynchronously. Your Java application starts outputting the data into a flat-file. Then, your message broker could retrieve it via FTP protocol and process record by record onto the destination system.

4) Your broker subscribes to a queue that is going to be fed by your Java Application. You Java Application will keep on processing data from time to time and generating data events that would be sent to the message queue. You message broker will keep on listening for incoming messages and forwarding them to the destination system.

As a rule of thumb, your database should be giving all the data to your Java application without any actual data transformation (processing) or as little as possible. Even though Java can be processing data in-memory, it is bound to be less efficient when you are querying the database in between. Most database systems can pre-process the data for you instead of having an overhead in Java. Therefore, take the time to figure out where the bottleneck is. In most cases you will find a bad data model or some tweaking needs to be done in term of indexes or in the case of Hana using some of its features: like calculation view, table functions and so on. Sometimes you will have to pre-processed data using other techniques to make data ready to be consumed by your Java application.

WebSockets could be used on cases where your data is retrieved in times where you do not have control over it. Let's say you have an IoT scenario where devices are sending data to your Java application. However, you are only interested on data that exceeds some specific threshold. On such cases your Java api will only issue this type of data eventually - filtering out a lot of stuff that is not necessary. And you want to analyze these data "events" on your destination system integrated by the message broker. On such cases you ought to use WebSockets. Simply because you don't really know when your Java application will send out any of the interesting data. A chat application is another case where this be clarly understood. You don't really know when the other side will send the message - but when it does, you want it on the other side right away.

As you can see, it all depends on your concrete scenario.

Best regards,
Ivan

[k-fuji2458 · Accepted solution]

Hi ivan.mirisola ,

It means that we should identify and fix the bottleneck for each function, not simply HTTP requests or WebSockets.

I will try to identify the problem areas of each function and deal with them individually.

Thank you very much for your detailed explanation with many easy to understand examples.

Best Regards, Kento