In our recent release of SAP Data Hub 2.7, we introduced a set of new functionalities. Let’s review the new 2.7 functionality in this blog. We will focus on 4 major areas and go over each area in detail.
We focus here on two major sub-areas in ABAP integration:
Our goal is to provide one model to consolidate all interaction scenarios between SAP Data Hub and an ABAP-based SAP system both one directional and bi-directional, as shown below. In an ABAP-based SAP system, we include SAP NW-based Business Suite, Business Warehouse, and SAP S/4HANA and SAP S/4HANA Cloud. The goals are to extract metadata from the ABAP systems to the SAP Data Hub metadata catalog and to ingest data from ABAP systems into a pipeline for processing and/or subsequent transfer to targets.
ABAP Integration is performed via out of box ABAP CDS READER and SLT CONNECTOR operators as well as it has the capability to build CUSTOM ABAP OPERATOR. Functionalities of each operator and restrictions for each operator is listed below.
ABAP Integration is used for the direct communication from SAP S/4HANA 1909 to SAP Data Hub 2.7 and onward. SLT replication is leveraged to connect to older releases as shown below.
Please note, SAP LT Replication Server (SLT) can be used free of charge for the use case to move into SAP Data Hub.
One of the use cases of ABAP Integration is to replicate ABAP CDS VIEW, which is a semantically rich data model in SAP S/4HANA that allows the consistent representation of a business object like a Business Partner. CDS view can be replicated to cloud storage such as AWS S3, Azure Data Lake, Google Cloud Storage, HDFS, etc. It not only supports initial load but also it allows delta mode.
SAP Data Hub provides a broad spectrum of connectivity with a strong focus on heterogenous landscapes, including Hadoop, Cloud Storages, Messaging Technologies, third-party databases, enterprise applications and so on. SAP LT Replication Server replicates data via SAP Data Hub into additional target systems that aren’t natively supported by SAP LT Replication Server.
The SLT CONNECTOR operator establishes a communication to the remote ABAP system and consumes data (initial load and delta) via SAP LT Replication Server technology. This allows the consumption of business data directly within an SAP Data Hub pipeline to leverage a tight integration between business data and big data. The following diagram shows how SLT CONNECTOR allows us to replicate ABAP tables to various cloud storage for both initial and delta mode.
In addition, CUSTOM ABAP OPERATORS can be used to enhance the scope of a data-driven application. For example, it is necessary to execute a function module within a pipeline to read data into SAP Data Hub, post information into an ABAP-based SAP system, or trigger an execution in the remote system, as shown below.
In the above diagram, we need DMIS 2011 SP17 or DMIS 2018 SP2. Please check out a series of blogs listed at the end of this blog for additional information.
In metadata and data excellence, we highlight into four sub-areas:
The rule dashboard helps to visualize rule validation results. The capabilities of rule dashboards are:
A hierarchical tagging system helps you to organize and manage your tags. Goals of hierarchical tagging include:
Capabilities
Allow business users to upload files and manage folders on supported connections
Providing metadata of ABAP-based SAP systems in SAP Data Hub Metadata Explorer and publish them to the catalog
New connectivity operators
SAP Data Hub 2.7 provides connectivity to Azure SQL Database with new Azure SQL DB SQL Consumer, Azure SQL DB Table Consumer operators and connectivity to SAP IQ with SAP IQ SQL Consumer, SQP IQ Table Consumer operators.
SAP Data Hub extends Copy File, Move File, Read File, Remove File and Write File operators for Alibaba Cloud Object Storage Service (OSS).
Please explore the new functionality described in this blog, and provide feedback using them with existing functionality.
- ABAP Integration
- Metadata and Data Excellence
- Connectivity and Processing
- Deployment and Operations
ABAP Integration
We focus here on two major sub-areas in ABAP integration:
- ABAP data provisioning to access business data in SAP S/4HANA and consume ABAP CDS views directly in a pipeline.
- Ability to create your own operators that reference the corresponding ABAP functionality to execute ABAP code like function modules.
ABAP Integration Overview
Our goal is to provide one model to consolidate all interaction scenarios between SAP Data Hub and an ABAP-based SAP system both one directional and bi-directional, as shown below. In an ABAP-based SAP system, we include SAP NW-based Business Suite, Business Warehouse, and SAP S/4HANA and SAP S/4HANA Cloud. The goals are to extract metadata from the ABAP systems to the SAP Data Hub metadata catalog and to ingest data from ABAP systems into a pipeline for processing and/or subsequent transfer to targets.
ABAP Integration is performed via out of box ABAP CDS READER and SLT CONNECTOR operators as well as it has the capability to build CUSTOM ABAP OPERATOR. Functionalities of each operator and restrictions for each operator is listed below.
ABAP Integration is used for the direct communication from SAP S/4HANA 1909 to SAP Data Hub 2.7 and onward. SLT replication is leveraged to connect to older releases as shown below.
Please note, SAP LT Replication Server (SLT) can be used free of charge for the use case to move into SAP Data Hub.
One of the use cases of ABAP Integration is to replicate ABAP CDS VIEW, which is a semantically rich data model in SAP S/4HANA that allows the consistent representation of a business object like a Business Partner. CDS view can be replicated to cloud storage such as AWS S3, Azure Data Lake, Google Cloud Storage, HDFS, etc. It not only supports initial load but also it allows delta mode.
SAP Data Hub provides a broad spectrum of connectivity with a strong focus on heterogenous landscapes, including Hadoop, Cloud Storages, Messaging Technologies, third-party databases, enterprise applications and so on. SAP LT Replication Server replicates data via SAP Data Hub into additional target systems that aren’t natively supported by SAP LT Replication Server.
The SLT CONNECTOR operator establishes a communication to the remote ABAP system and consumes data (initial load and delta) via SAP LT Replication Server technology. This allows the consumption of business data directly within an SAP Data Hub pipeline to leverage a tight integration between business data and big data. The following diagram shows how SLT CONNECTOR allows us to replicate ABAP tables to various cloud storage for both initial and delta mode.
In addition, CUSTOM ABAP OPERATORS can be used to enhance the scope of a data-driven application. For example, it is necessary to execute a function module within a pipeline to read data into SAP Data Hub, post information into an ABAP-based SAP system, or trigger an execution in the remote system, as shown below.
In the above diagram, we need DMIS 2011 SP17 or DMIS 2018 SP2. Please check out a series of blogs listed at the end of this blog for additional information.
Metadata and Data Excellence
In metadata and data excellence, we highlight into four sub-areas:
- Rule dashboard and scorecard to visualize rule results
- Hierarchical tagging to datasets and columns to efficiently search datasets
- Ability to upload data to standard file-based storage
- Enabling metadata from ABAP-based systems into Metadata Explorer
The rule dashboard helps to visualize rule validation results. The capabilities of rule dashboards are:
- A wizard to set up a scorecard based on the object. The result shows either a line graph, bar chart, donut chart, or trend score.
- It allows you to create a group to organize your scorecards. For example, you could create group that contains all the trend scores from your datasets, categories, and rulebook.
A hierarchical tagging system helps you to organize and manage your tags. Goals of hierarchical tagging include:
- To manage a large catalog and organize them in a hierarchy
- To allow search dataset and columns by tags
Capabilities
- Add tags to a dataset or at the column level
- Edit or delete tags that are no longer relevant
- View the tags in the Catalog, Lineage, and Fact Sheet pages
- Filter and search by tags
Allow business users to upload files and manage folders on supported connections
- In Metadata Explorer, it allows business user to upload files and manage folders on Amazon S3, Google Cloud Storage (GCS), Hadoop Distributed File System (HDFS), Microsoft Azure Data Lake (ADL) and Microsoft Windows Azure Storage Blob (WASB)
- In data preparation, it allows to import data from a file to create a new preparation
Providing metadata of ABAP-based SAP systems in SAP Data Hub Metadata Explorer and publish them to the catalog
- Receive metadata information of tables, Views, CDS Views, and custom CDS views (depending on the source system)
- Covering of standard metadata functions such as browse, preview, publish, and catalogue relevant metadata information
Connectivity and Processing
- New connectivity to Azure SQL Database, SAP IQ Server
- Support for new cloud storage, Alibaba Cloud Object Storage Service (OSS)
New connectivity operators
SAP Data Hub 2.7 provides connectivity to Azure SQL Database with new Azure SQL DB SQL Consumer, Azure SQL DB Table Consumer operators and connectivity to SAP IQ with SAP IQ SQL Consumer, SQP IQ Table Consumer operators.
SAP Data Hub extends Copy File, Move File, Read File, Remove File and Write File operators for Alibaba Cloud Object Storage Service (OSS).
Deployment and Operations
- A new supported deployment on AliCloud with OSS as a checkpoint storage type
- Support of external Identity providers (IdPs).
- SAP Data Hub System Management Command-Line Client (vctl) tool to manage SAP Data Hub System Management functionalities, including life cycle management.
Please explore the new functionality described in this blog, and provide feedback using them with existing functionality.
Additional Resources
SAP Community - ABAP Integration Blog posts
- ABAP Integration for SAP Data Hub and SAP Data Intelligence – Overview Blog
- ABAP Integration – Software Delivery & Installation Aspects
- ABAP Integration – Calling a ABAP function module within a SAP Data Hub pipeline
- ABAP Integration – Replicate ABAP CDS views via SAP Data Hub
- ABAP Integration – Replicating tables into SAP Data Hub via SAP LT Replication Server
- SAP Managed Tags:
1 Comment
