This article is intended for database admins, consultants, customers & partners to know more about SAP Commissions - SDIArchitecture Flow to prepare your various available data integration process.
First thing First
You need to Install DP Agent to connect to clients/customer source system to bring data into business need and transform into SAP Commissions staging tables.
SAP Commissions - SDI Architecture Flow
Direct Approach using DP Agent (Most Recommended)
SDI is packaged as part of HANA Enterprise which comprises the following components:
HANA Server (SDI Data Provisioning Server) hosted by SAP/Callidus Datacenter
Data Provisioning Agent(DP Agent) hosted on-premise by customers
The DP agent is installed in the customer’s premise and is set up to connect with SDI and the HANA DB Server. The DP Agent connects to the data source systems (SQL Server, Hadoop,Flat Files, Odata, RDBMS, or others) and passes information between the source systems and the HANA Server using built-in adapters that are packaged with the DP Agent. Data is transmitted over HTTPS to and from the HANA Server.
Every SAP Commission tenant is provisioned with a separate HANA tenant database. Web IDE facilitates access to the HANA database and SDI components. A development space is provisioned in Web IDE XSA (Extended Application Service) to enable application developers to manage content integration development. The Web IDE DB Explorer plugin allows users to connect to the tenant database and access the schemas.
SDI component which allows connectivity to external sources.
Temporary data storage area in Commissions where data from SDI is placed during export. Data is validated in the staging area before transferring into the Commissions tables and executing the pipeline. See Commissions Administrator online help and Data Dictionaries for more details.
The Data Provisioning Agent hosts all SDI Adapters and acts as the communication interface between Hana and the Adapter.
Flat file allows you to specify data attributes, such as columns and data types table by table, and stores the data in plain text format.
A remote source can be used to create virtual tables or to use the linked database feature.
A graphical user interface to develop data integration mapping and transformations.
EXT Schema in HANA database is a temporary database that facilitates data validation, transformation, aggregation, and cleaning for large volumes of data. It allows creating custom tables and stored procedures to process bulk data.
SAP HANA database which is used by Commissions for data storage and processing.
Pipeline is a compensation computation process initiated from the Pipeline workspace in the Job Queue view or from the command-line utility. The pipeline produces compensation and pay results for payees assigned to variable compensation plans. See Commissions online help for more details.
Designated area in Commissions where related compensation objects are grouped together so that a user can perform related tasks from the same place.
Protocol for building and consuming REST APIs.
A HANA component, which allows read and write of data from external sources.
SAP Web IDE is a browser-based integrated development environment (IDE), comprised of web-based UIs, business logic, and extensive SAP HANA data models, that are leveraged by SDI.
Web IDE facilitates access to HANA database and SDI components. It is also a Web-based development environment for SAP Fiori, SAPUI5, and full-stack business apps.
Alternative approach using Commissions Data Loader (CDL) without DP Agent
In alternative approach, when client/customer doesn't allow their database network to connect via DP Agent...
Customer can push the raw data to SAP Commissions sFTP inbound folder in secured way, as it goes directly into Commissions Data Loader (CDL) as custom inbound file processing and that gets loaded into temporary table for further pre-processing the data through flow-graph and loads into Commissions Staging table. Virtual Table will trigger automatically when the data is ready for Validate&Transfer job to process.