Technology Blogs by Members
Explore a vibrant mix of technical expertise, industry insights, and tech buzz in member blogs covering SAP products, technology, and events. Get in the mix!
cancel
Showing results for 
Search instead for 
Did you mean: 
Arundathi
Participant
Introduction:

With the increasing level of interface complexity, it’s very important to log payloads at different stages of build. Payloads can be logged using groovy scripts in message processing logs (MPL), data store and persist message. SAP has added a circuit breaker to stop MPL attachments if 1 GB data is stored in 24hrs. Currently SAP doesn’t provide any UI to download Persisted messages. Logging payloads using data store is a better option, but with the increase in payload logging the memory consumption will also be high. So, data store entries should be cleared regularly, especially in production tenant. SAP has provided standard API to extract data store entries. In this blog I will explain how to delete data store payloads/entries using SAP API, groovy script and delete data store operation.

Scenario:

Retrieve the data store entries from SAP API, filter the entries to be deleted using groovy script and delete the data store entries using delete operation.

Integration Artifact Details:

Step 1:

Log/Store payloads using write data store operation in any required integration process. I will not cover many details on this topic. In the current scenario I am storing the payloads in data store: DS_Payloads:



StorePayload


Step 2: Create an iFlow:


Main iFlow




  1. Start Timer: Schedule the interface to recur daily.

  2. Set Header: Configure the header details as below

  3. Request Reply SAP API call:
    https://tenantID-tmn.xxxxx.ondemand.com/api/v1/DataStores(DataStoreName='DS_Payloads',IntegrationFlow='',Type='')/Entries

  4. Format API Response: Refer the below xslt code to format API response
    <?xml version="1.0"?>
    <xsl:stylesheet version="1.0" xmlns:xsl="http://www.w3.org/1999/XSL/Transform">

    <xsl:output indent="yes" method="xml" encoding="utf-8" omit-xml-declaration="yes"/>

    <!-- Stylesheet to remove all namespaces from a document -->
    <!-- template to copy elements -->
    <xsl:template match="*">
    <xsl:element name="{local-name()}">
    <xsl:apply-templates select="@* | node()"/>
    </xsl:element>
    </xsl:template>

    <!-- template to copy attributes -->
    <xsl:template match="@*">
    <xsl:attribute name="{local-name()}">
    <xsl:value-of select="."/>
    </xsl:attribute>
    </xsl:template>

    <!-- template to copy the rest of the nodes -->
    <xsl:template match="comment() | text() | processing-instruction()">
    <xsl:copy/>
    </xsl:template>
    </xsl:stylesheet>​


  5. Filter Entries ToBeDeleted: Refer the below groovy script to filter data store entries to be deleted and create a deletion entries list - EntryID. Here I am deleting the payloads which were saved 3 days before the current time. Payload’s retention period can be changed based on the business requirement.
    import com.sap.gateway.ip.core.customdev.util.Message
    import java.util.HashMap
    import com.sap.it.api.ITApiFactory
    import com.sap.it.api.mapping.ValueMappingApi
    import groovy.xml.MarkupBuilder
    import groovy.time.*;

    def Message processData(Message message)
    {
    def a = ITApiFactory.getApi(ValueMappingApi.class, null);
    def body = message.getBody(java.lang.String) as String;
    def Test = new XmlParser().parseText(body);
    def writer = new StringWriter();
    def builder = new MarkupBuilder(writer);

    def ToBeDeleted = []
    def ID = []
    def TimeDuration = []
    String Create = '';

    builder.Target
    {
    'Record'
    {

    Test.entry.each {


    it.properties.each{

    def mappedValue = "${it.Id[0].text()}";


    if ( mappedValue.equals("") || mappedValue ==null )
    {

    ToBeDeleted.push("No")
    }
    else
    {

    Create = "${it.CreatedAt[0].text()}"
    def CreatedAt = Create.substring(0,19)
    CreatedAt = CreatedAt.replace("T", " ")
    def Newdate = Date.parse("yyyy-MM-dd HH:mm:ss",CreatedAt)

    def Currentdate = new Date()

    TimeDuration duration = TimeCategory.minus(Currentdate, Newdate);
    TimeDuration.push(duration)
    if(duration.getDays() > 3){
    ToBeDeleted.push("Yes")
    ID.push(mappedValue)
    }

    }
    }
    }
    }
    'Id'(ID)
    }
    // Generate output
    message.setBody(writer.toString())
    message.setProperty("ToBeDeleted", ToBeDeleted)
    message.setProperty("EntryID", ID)
    message.setProperty("TimeDuration", TimeDuration)

    return message;

    }​


  6. Looping Process Call:  Process call to delete messages. This process repeats for each entry in the deletion list.

    Delete Message Logs: Sub process to delete data store entries



  7. Set DelEntryID: Refer the below groovy script to set delete EntryID and set indicator if any valid entries are present to delete.
    import com.sap.gateway.ip.core.customdev.util.Message;
    import java.util.HashMap;
    import com.google.gson.Gson;
    import com.google.gson.GsonBuilder;

    import groovy.json.*;

    def Message processData(Message message) {

    def body = message.getBody(java.lang.String) as String

    def DelEntryID = message.getProperties().get("EntryID")[0]
    def ToBeDeleted

    if ( DelEntryID.equals("") || DelEntryID ==null )
    {

    ToBeDeleted= 'No'
    }
    else{
    ToBeDeleted= 'Yes'
    }

    message.setProperty("DelEntryID", DelEntryID)
    message.setProperty("ToBeDeleted", ToBeDeleted)

    return message;
    }​


  8. Router: Decides if there is any valid entry to delete. If not ends the call.

  9. Delete: Deletes data store entry:

  10. Remove Deleted Entry Id: Refer the below groovy script to remove the deleted EntryID from the deletion list.
    import com.sap.gateway.ip.core.customdev.util.Message;
    import java.util.HashMap;
    import com.google.gson.Gson;
    import com.google.gson.GsonBuilder;

    import groovy.json.*;

    def Message processData(Message message) {

    def body = message.getBody(java.lang.String) as String

    //Remove EntryID from lookup list
    def packages = message.getProperties().get("EntryID").reverse() as Stack
    packages.pop()
    packages = packages.reverse() as Stack
    message.setProperty("EntryID", packages)

    return message;
    }​



Step 3:
Configure the above iflow, schedule it as per business requirement and deploy it.

Conclusion:
We just saw how to configure the interface to automatically delete the logged payload in data store. The interface schedule can be altered based on payload retention period as per the business requirement.

References:

https://blogs.sap.com/2020/01/29/sap_cpi-demonstrating-the-datastoreoperations-feature/

https://help.sap.com/viewer/368c481cd6954bdfa5d0435479fd4eaf/Cloud/en-US/8c35f3fa3b9c42c5b810332eccb...

 
5 Comments
Labels in this area