SAP MDG Consolidation data import: The ETL way

[mageshwaran_subramanian]

The objective of this blog is to explain how the SAP MDG Consolidation source tables can be populated using SAP Data Services.

SAP MDG, consolidation and mass processing enable you to consolidate your master data and it helps you to run mass processes to update more than one master data records at a time. The order of the process steps, as well as the behavior of each individual process step type, can be adapted for your necessities.

 



Data load is the first step in master data consolidation of the source records. The process flow diagram above shows the data import from the file. You can fill the data in a .csv file or .xlsx file in a predefined format that the consolidation system accepts. SAP note: 2233142 – Example: How to fill Source Tables uses ABAP code to fill the consolidation tables.



The data to be consolidated can be loaded from SAP and non-SAP-systems using ETL tools. In Create Data Import Fiori app, there's no option to select the ETL method.  Now you might wonder how to load the data in the MDC system using an ETL tool. The answer is to load the data directly in the MDC tables. At the time of this writing, MDG consolidation is available for business partners, Material master, and custom objects.

Master data consolidation source tables follow the naming convention <table_name>_SRC. For example, BUT000_SRC, LFA1_SRC, etc.,





Refer to this page for more information on BP/Customer/Vendor tables.

Now let's see how the MDC tables can be populated using SAP Data Services.

The SAP MDG system in this example uses the HANA database to store the consolidation source tables. So as a first step create a data store connection pointing to the HANA database in Data Services Designer. Make sure that the DB user has read and write access to the _SRC tables.

 

Note that the consolidation tables typically available in SAPABAP1 schema.

Once the data store connection is created, import the table you want to populate. Let us import the table BUT000_SRC

The next step is to build a job to populate the consolidation table(s). Create a data flow and use the imported MDC table as a target. For the sake of simplicity, I'm using a Row_generation transform to populate one test record to the table BUT000_SRC. You can replace the Row_generation transform with the actual source table or file as a source.

Pay attention to the table columns and data types. The source ID has to be a unique identifier for a specific source system. If you upload data using the vendor or customer data model, you have to make sure that the ID remains unique. I defaulted CLIENT as 100, SOURCE_SYSTEM as DS_Source, SOURCE_ID as DSSOURCEID01 and SOURCE_FILTER as DS Source Filter in order to uniquely identify the record to be imported.

Execute the DS job and make sure it runs without an error and the table is populated.

Log on to the MDG consolidation system and open Manage Data Source under the Consolidation of Mass Processing for Business Partner Data.

Search for DS_Source to make sure it indicates the successful data import.

The records imported can now be used for the next steps in the consolidation process.

[former_member650989]

Hi Magesh,

 

Thank you for the post, but i have a question related to some other tile in Consolidation and Mass Processing of business partner. Hope it is okay to ask about that.

Could you please let me know how we can achieve the Create Consolidation Business Partner.

I am actually following the below thread from SAP

https://help.sap.com/saphelp_mdg90/helpdata/en/86/9b1154aeb08a24e10000000a4450e5/frameset.htm

But it would be really helpful if you write a blog for the same. 🙂

Thank you.

[bpawanchand]

Hello Mangesh,

 

I have a question regarding the access to SAPABAP1 ~ SAPHANDB Schema is that fine to give write access to this most important schema where it has all the tables of SAP S4

 

Thanks

Pavan Bhamidipati

[former_member356892]

Hi Pavan,

You raise a valid point here.  Typically we would not recommend providing DB access to the transactional schema within S/4HANA.  Even though Data Services would be using what we consider a "technical user" to connect to the HANA DB it still would allow all DS developers who have access to that datastore to potentially be able to read/write to tables in the SAPABAP1.  This potentially validates certain license requirements as well.

We would recommend only populating the MDG Consolidation Source tables through the Application layer in SAP.  This could be done via the Web Service SOAP call or by creating a custom ABAP function which would populate those tables.

Either way ---- not going against the DB layer directly.

-Brendan