Cloud Integration with Commerce Azure Blob Storage using REST API - Part 2

[PriyankaChak]

Introduction:

This blog post is the continuation of my previous blog post: Part 1

Scenario:

Get files from Commerce Azure Blob storage.

Prerequisite Setup:

1. Set up an Integration suite trial. Help Link: Setup.


2. Azure Storage Explorer. Help Link: Azure Storage Explorer.

Design Solution in Cloud Integration:

Integration Flow

Step 1:

Configure timer to pick up files at a regular interval.

Step 2:

Use Content Modifier to set properties and header for version, as shown below.

Set Version Header

Set Properties

Step 3:

Use Groovy Script to set up the headers for calling REST API to get the name of files starting with 'order'.

import com.sap.gateway.ip.core.customdev.util.Message

import com.sap.it.api.ITApiFactory

import com.sap.it.api.securestore.SecureStoreService

import com.sap.it.api.securestore.UserCredential

import com.sap.it.api.securestore.exception.SecureStoreException

import javax.crypto.Mac

import javax.crypto.spec.SecretKeySpec

import java.security.InvalidKeyException

def Message processData(Message message) 

{

    //Set Current Time

    TimeZone.setDefault(TimeZone.getTimeZone('GMT'))

    def now = new Date().format("EEE, dd MMM yyyy HH:mm:ss") + " GMT"

    //Set header for datetime

    message.setHeader("x-ms-date", now)

    //Get container name

    String container = message.getProperties().get("container")

    //Get folder path

    String folderPath = message.getProperties().get("folderPath")

    //Get File Name Prefix

    String filenamePrefix = message.getProperties().get("filenamePrefix")

    //Set Prefix name

    String prefix = folderPath + '/' + filenamePrefix + '_'

    message.setProperty("prefix", prefix)

    //Get Account Name

    String account = message.getProperties().get("accountName")

    // Set canonicalized Resource

    String canonicalizedResource = '/'+ account + '/'+ container + '\n'+ 'comp:list' + '\n' + 'prefix:' + prefix + '\n'+ 'restype:container'

    // set verb as requested method

    String verb = 'GET'

    //Get version

    String version = message.getHeaders().get("x-ms-version")

    //Set Signature String

    String StringToSign = verb +'\n'+'\n'+'\n'+ '\n' +'\n'+ '\n' + '\n' +'\n'+'\n'+'\n'+'\n'+'\n'+'x-ms-date:'+ now +'\n' +'x-ms-version:' + version + '\n'+ canonicalizedResource

    //Get Account Key from Secure Parameter

    String accountKeyAlias = message.getProperties().get("accountKeyAlias")

    def accountKey = getAccountKey(accountKeyAlias)

    // Decode Account Key

    def decodedKey = accountKey.decodeBase64()

    //Get Hash Value

    String hash = hmac_sha256(decodedKey, StringToSign)

    //Set Authorization header

    String auth = 'SharedKey'+ ' ' + account + ':' + hash

    message.setHeader("Authorization", auth)

    return message

}



String getAccountKey(String accountKeyAlias)

{

   def secureStorageService =  ITApiFactory.getService(SecureStoreService.class, null)

    try

    {

        def secureParameter = secureStorageService.getUserCredential(accountKeyAlias)

        return secureParameter.getPassword().toString()

    } 

    catch(Exception e)

    {

        throw new SecureStoreException("Secure Parameter not available")

    }

}



String hmac_sha256(byte[] secretKey, String data) 

{

    try 

    {

        Mac sha256_HMAC = Mac.getInstance("HmacSHA256")

        SecretKeySpec secret_key = new SecretKeySpec(secretKey, "HmacSHA256")

        sha256_HMAC.init(secret_key)

        byte[] digest = sha256_HMAC.doFinal(data.getBytes())

        return digest.encodeBase64()



    } catch (InvalidKeyException e) 

    {

        throw new RuntimeException("Invalid key exception while converting to HMac SHA256")

    }

}

Step 4:

Use Request-Reply step to get the list of files. Configure HTTP receiver channel as below.

HTTP Receiver Channel

Step 5:

Use XSLT mapping to sort the list of filenames in ascending order based on last modified time.

<?xml version="1.0" encoding="UTF-8"?>

<xsl:stylesheet version="3.0" xmlns:xsl="http://www.w3.org/1999/XSL/Transform">

<xsl:output method="xml" indent="yes"/>

  <xsl:template match="EnumerationResults/Blobs">

    <xsl:copy>

      <xsl:apply-templates select="Blob">

        <xsl:sort select="Properties/Last-Modified"/>

      </xsl:apply-templates>

    </xsl:copy>

  </xsl:template>

  <xsl:template match="@* | node()">

    <xsl:copy>

      <xsl:apply-templates select="@* | node()"/>

    </xsl:copy>

  </xsl:template>

</xsl:stylesheet>

Step 6:

Use General Splitter step to retrieve each file entry as shown below.

General Splitter

 

Step 7:

Use Content Modifier step to save last modified time and filename as properties.

Content Modifier - Properties

Step 8:

Use Groovy Script to check whether the file is new based on the comparison between last poll time and last modified time.

import com.sap.gateway.ip.core.customdev.util.Message

import java.text.SimpleDateFormat

def Message processData(Message message) {

    SimpleDateFormat customFormat = new SimpleDateFormat("E, dd MMM yyyy HH:mm:ss z");

    def lastmodified = message.getProperties().get("lastmodified")

    def lastpoll = message.getProperties().get("lastpoll")

    def dateTime1 = customFormat.parse(lastmodified)

    def dateTime2 = customFormat.parse(lastpoll)

    def getFile = "No"



   if (dateTime1 > dateTime2)

     {

         getFile = "Yes"

     }

    message.setProperty("getFile", getFile)

    return message

    }

 

Step 9:

Use Router step as shown below.

Router

Step 10:

Use Groovy Script to set up the headers for calling REST API to get the specific file.

import com.sap.gateway.ip.core.customdev.util.Message

import com.sap.it.api.ITApiFactory

import com.sap.it.api.securestore.SecureStoreService

import com.sap.it.api.securestore.UserCredential

import com.sap.it.api.securestore.exception.SecureStoreException

import javax.crypto.Mac

import javax.crypto.spec.SecretKeySpec

import java.security.InvalidKeyException

def Message processData(Message message) 

{

    //Set Current Time

    TimeZone.setDefault(TimeZone.getTimeZone('GMT'))

    def now = new Date().format("EEE, dd MMM yyyy HH:mm:ss") + " GMT"

    //Set header for datetime

    message.setHeader("x-ms-date", now)

    // Get file name

    String filename = message.getProperties().get("fileName")

    //Get container name

    String container = message.getProperties().get("container")

    //Get Account Name

    String account = message.getProperties().get("accountName")

    // Set canonicalized Resource

    String canonicalizedResource = '/'+ account + '/'+ container +'/' + filename

    // set verb as requested method

    String verb = 'GET'

    //Get version

    String version = message.getHeaders().get("x-ms-version")

    //Set Signature String

    String StringToSign = verb +'\n'+'\n'+'\n'+ '\n' +'\n'+ '\n' + '\n' +'\n'+'\n'+'\n'+'\n'+'\n'+'x-ms-date:'+ now +'\n' +'x-ms-version:' + version + '\n'+ canonicalizedResource

    //Get Account Key from Secure Parameter

    String accountKeyAlias = message.getProperties().get("accountKeyAlias")

    def accountKey = getAccountKey(accountKeyAlias)

    // Decode Account Key

    def decodedKey = accountKey.decodeBase64()

    //Get Hash Value

    String hash = hmac_sha256(decodedKey, StringToSign)

    //Set Authorization header

    String auth = 'SharedKey'+ ' ' + account + ':' + hash

    message.setHeader("Authorization", auth)

    return message

}



String getAccountKey(String accountKeyAlias)

{

   def secureStorageService =  ITApiFactory.getService(SecureStoreService.class, null)

    try

    {

        def secureParameter = secureStorageService.getUserCredential(accountKeyAlias)

        return secureParameter.getPassword().toString()

    } 

    catch(Exception e)

    {

        throw new SecureStoreException("Secure Parameter not available")

    }

}



String hmac_sha256(byte[] secretKey, String data) 

{

    try 

    {

        Mac sha256_HMAC = Mac.getInstance("HmacSHA256")

        SecretKeySpec secret_key = new SecretKeySpec(secretKey, "HmacSHA256")

        sha256_HMAC.init(secret_key)

        byte[] digest = sha256_HMAC.doFinal(data.getBytes())

        return digest.encodeBase64()



    } catch (InvalidKeyException e) 

    {

        throw new RuntimeException("Invalid key exception while converting to HMac SHA256")

    }

}

Step 11:

Use Request-Reply step to get the file. Configure HTTP receiver channel as below.

HTTP Receiver Channel

Step 12:

Use Write Variables to save the last poll time.

Write Variables

 

Conclusion:

Using the above explained steps, the new files can be retrieved based on last modified datetime. Please note, to avoid getting a large list of filenames during 1st call, an archiving strategy can be devised at Azure Blob storage end to move the old files to archive directory after N days.

Thank you for reading this blog post. Please feel free to share your feedback or thoughts or ask questions in the Q&A tag below.

QA link

Reference Links:

• Get a list of files: List Containers


• Get file: Get Blob


• Script Logic: Authorize with shared Key

 

Regards,

Priyanka Chakraborti

 

   Previous - Part 1