SAP Data Intelligence – What’s New in DI:2023/02

[eduardo_haussen]

SAP Data Intelligence, cloud edition DI:2023/02 is now available.

Within this blog post, you will find updates on the latest enhancements in DI:2023/02. We want to share and describe the new functions and features of SAP Data Intelligence for the Q1 2023 release.

If you would like to review what was made available in the previous release, please have a look at this blog post.

Overview

This section will give you a quick preview about the main developments in each topic area. All details will be described in the following sections for each individual topic area.

SAP Data Intelligence 2023/02

 

Connectivity & Integration

This topic area focuses mainly on all kinds of connection and integration capabilities which are used across the product – for example: in the Metadata Explorer or on operator level in the Pipeline Modeler.

Support non-basic authentication for Microsoft Exchange e-mail servers in IMAP connections

USE CASE DESCRIPTION:

• Support of OAuth2 authentication for Microsoft Exchange e-mail servers in IMAP connections via the authorization method ms_client_credentials


• Supported OAuth2 authentication methods
  
  
  
  • Client ID + Client Secret
  
  
  • X.509 Certificates
  
  
  
  

 

 

BUSINESS VALUE – BENEFITS:

• Usage of non-basic authentication in IMAP connections

 

 

Support HANA Cloud as source in Replication Flows

USE CASE DESCRIPTION:

• Allow customers to leverage mass data replication scenarios via Replication Flows using HANA Cloud as a source system


• Data from HANA Cloud can be loaded into any target supported by Replication Flows


• Supporting standard functionalities offered by Replication Flows such as parallelization and resilience

 

 

BUSINESS VALUE – BENEFITS:

• Customers can realize mass data ingestion from HANA Cloud in a simplified way compared to pipelines


• Lower TCO & TCD for data replication scenarios from HANA Cloud as a source

 

Provide additional runtime metrics for ODP in Generation 2

USE CASE DESCRIPTION:

• Provide additional important information for end-users in the process logs when extracting data via ODP interface by showing the number of replicated records from the ODP source object for the generation 2 “Read Data from SAP System” operator

BUSINESS VALUE – BENEFITS:

• Seamless user experience when using the Read Data from SAP System operator by showing similar metrics for all use cases (CDS View, table via SLT and ODP extraction)

 

Pipeline Modelling

This topic area covers new operators or enhancements of existing operators. Improvements or new functionalities of the Pipeline Modeler and the development of pipelines.

Possibility to connect own Docker registries with SAP Data Intelligence Cloud

USE CASE DESCRIPTION:

• SAP Data Intelligence Cloud is able use external Docker registries that are configured and owned by customers to pull images that are used as graph base Docker images


• It is not possible to push Docker images that were created in DI Cloud to external registries


• Supported hyperscaler Docker registries:
  
  
  
  • AWS Elastic Container Registry (authentication via token)
  
  
  • Microsoft Azure Container Registry (authentication via service principals)
  
  
  • GCP Container Registry (authentication via json-keys)
  
  
  • GCP Artifact Registry (authentication via json-keys)
  
  
  
  

 

 

BUSINESS VALUE – BENEFITS:

• Customers can create their own Docker-based-images for DI Cloud from scratch and can use them in several DI Cloud tenants (e.g. Dev, QA, Prod)


• Make migration from DI on-premise to DI Cloud more seamless

 

 

API extension to improve Pipeline Scheduling

USE CASE DESCRIPTION:

• Possibility to set the timeout for graph scheduling via the POST /v1/runtime/graphs API.
  The Graph turns dead in case it cannot be scheduled within the timeframe that is specified with this new property:
  
  
  
  • New property scheduleTimeoutSeconds (minimum value: 240)
  
  
  
  


• Possibility to retrieve a more detailed context in a new Reason property that is retrieved when executing the GET /v1/runtime/graphs/’handle’ API in case a graph could not be scheduled/started properly (e.g., Unschedulable,…)


• See also SAP API Business Hub for a full specification of the Pipeline Engine API for SAP Data Intelligence Cloud

 

 

BUSINESS VALUE – BENEFITS:

• More flexibility for automatic pipeline scheduling via Pipeline Engine API

 

 

Support GBQ in Generation 1 and Generation 2 Flowagent SQL Executor Operator

USE CASE DESCRIPTION:

• Allow Execution of SQL queries against Google Big Query in Generation 1 and Generation 2 Pipelines via the Flowagent SQL Executor Operator


• Generation 1: GCP_BIGQUERY connection Type selectable in Database type dropdown of the Flowagent SQL Executor Operator configuration


• Generation 2: Connections of type GCP_BIGQUERY are browsable in the menu of the Connection ID configuration of the Flowagent SQL Executor Operator

 

 

BUSINESS VALUE – BENEFITS:

• Improve integration of Google Big Query with SAP Data Intelligence Cloud

 

 

These are the new functions, features and enhancements in SAP Data Intelligence, cloud edition DI:2023/02 release.

We hope you like them and, by reading the above descriptions, have already identified some areas you would like to try out.

If you are interested, please refer to What’s New in Data Intelligence – Central Blog Post.

For more updates, follow the tag SAP Data Intelligence.

We recommend visiting our SAP Data Intelligence community topic page to check helpful resources, links and what other community members post. If you have a question, feel free to check out the Q&A area and ask a question here.

Thank you & Best Regards,

Eduardo and the SAP Data Intelligence PM team