SAP Datasphere: Replicating Custom CDS Views from S4 HANA to Azure Data Lake with CDC

[ansharm71]

Introduction

In this article I will share work experiences of using a custom CDS view on Joining BKPF and BSEG table in S4HANA and consuming it in SAP Datasphere for delta enabled replication (Change Data Capture) to Azure Data Lake Storage Gen2.

Create a Custom CDS View: ADT on Eclipse

Go to ABAP Repository Object and Click on the drop down for Core Data Services and then click on Data Definitions:

Provide the CDS view name and description. Note that we have not selected a referenced object, as we will be using two tables

Fill change request and click next , now select a Template view : Define View

Once you click on Finish , CDS view code is generated

Now modify the CDS view code and mention the tables BSEG and BKPF , add the join conditions and mention the columns in select clause.

Below is the code after modifying

Define CDS annotations 

Put in below annotations for "dataExtraction" and "changeDataCapture"

:{
    dataCategory: #CUBE,
    dataExtraction: {
        enabled: true,
        delta.changeDataCapture: {
            mapping:
            [ { table: 'BSEG', 
                role: #MAIN,
                viewElement: ['CompanyCode','Fiscalyear','Documentnumber','Lineitem'],
                tableElement: ['bukrs','gjahr','belnr','buzei']
              },
              { table: 'BKPF', 
                role: #LEFT_OUTER_TO_ONE_JOIN,
                viewElement: ['CompanyCode','Fiscalyear','Documentnumber'],
                tableElement: ['bukrs','gjahr','belnr']
              }
            ]
        }
    }
}

Now below is the completed CDS View Code with Data extraction and CDC enable

Save & Activate the View.

Replicate the CDS view in Datasphere (Initial and Delta Mode)

In SAP datasphere go to the Data Builder & Click on new Replication flow.

Ensure Source and Target connection are created and working in Datasphere, Here we have already created the connections.

Now create the replication flow by selecting Custom CDS view from S4 HANA System Connection.

Select CDS from source container and then add the source object i.e. Custom CDS View

Similarly select the Target connection and root folder in Azure Data Lake to place the target files.
Now select the Load Type as Initial and Delta and mention the Delta Load Interval as every 1 hr.

Click on Save and then Deploy replication flow.
Once deployment is complete, start the execution.
When we initiate the replication flow, the process begins with an initial load, followed by subsequent runs based on the delta load frequency to transfer the incremental changes

Monitor the Run :

Here we can see its showing the Initial load operations i.e. 1451 and followed by Delta load operations 110.
Please note replication flow was running for more than a day and automatic Delta records are captured and loaded into ADLS.

Validate the records in Azure Data Lake Storage

We can see in target data lake both Initial and Delta files are generate separately. We can see below initial load records with Operation_Type as L

Below are new delta records populated in separate file with Operation type as ‘A’.

Conclusion

This article is an attempt to showcase how some of the Custom S4 CDS views on multiple tables (extraction and CDC enabled) can be consumed in SAP Datasphere.