Technology Blogs by SAP
Learn how to extend and personalize SAP applications. Follow the SAP technology blog for insights into SAP BTP, ABAP, SAP Analytics Cloud, SAP HANA, and more.
cancel
Showing results for 
Search instead for 
Did you mean: 
fabioferrari
Advisor
Advisor
5,519
Disclaimer: the paragraph "SAP Data Intelligence pipeline" has been updated on 2022, September 29th.

 

Welcome to the third episode of the series: Integrating SAP Signavio Process Intelligence and SAP Data Intelligence.

In case you missed the two previous blog posts:

After the enhancements that were released in the product in June 2022, with the general availability of Ingestion API (in the Appendix you can find the link to the documentation), we can now overcome that preliminary approach and actually move to a direct connectivity between the products.

In this third post, I’ll therefore describe a concrete example that shows how you can use an SAP Data Intelligence pipeline to feed data directly into SAP Signavio Process Intelligence, via Ingestion API, bypassing the need for a staging area.


Ingestion API allows you to create an external data pipeline and ingest data into SAP Signavio Process Intelligence via API with an authentication token. This facilitates the integration with SAP Data Intelligence Cloud, where a data pipeline can be created with an operator at the end pushing the data into SAP Signavio Process Intelligence.

 

The example implemented in this blog shows how to read data from a customer survey from a Qualtrics system and how to then feed it into SAP Signavio Process Intelligence.

What is interesting is that such approach can be applied to any of the numerous sources supported by SAP Data Intelligence Cloud leveraging on its data processing and transformation capabilities, but it can also be applied to any kind of SAP and non-SAP data pipeline, external to SAP Signavio Process Intelligence.

 

 

Configuring an SAP Data Intelligence Cloud pipeline to move data from Qualtrics to SAP Signavio Process Intelligence via Ingestion API


Let’s first introduce some context.

Some of you already know about journey to process analytics, an innovative process management practice that connects experience and business operations data, aiming at understanding, improving and transforming your customer, supplier and employee experience. If you haven’t heard of it yet, I would suggest you to read this 5 minute blog post by aida.centelles.

Let’s now imagine that we want to analyze an Incident-to-Resolution process. From a traditional process mining perspective, the approach is quite straightforward: we configure a connection to ServiceNow, extract the data, create an event log, and apply process mining techniques on it.


But what if we could collect customers feedback, at the end of their journey, in a Qualtrics survey?

This experience data could be combined to the operational data to bring the customer experience perspective into play, enriching the data model and leading to a broad variety of new and valuable insights.

However, currently there is no native connector to Qualtrics in SAP Signavio Process Intelligence. Hence, we can create a data pipeline in SAP Data Intelligence to extract the survey data from Qualtrics and push it to SAP Signavio Process Intelligence.

 

Let’s see how we can quickly set this up.

 

Ingestion API


First of all, let’s create a new Ingestion API data source in SAP Signavio Process Intelligence.


You can either create a data source or an integration. Either way, the other one is then automatically created with the same title and with the same type (Ingestion API), and they will be interconnected with each other.



The data source provides two parameters:

  • The API endpoint is the target URL to make a request to ingest external data into SAP Signavio Process Intelligence

  • The Token is the authentication token required for a successful request pushing data into this specific pair of data source and integration


 

 

Qualtrics


Second, let’s connect to Qualtrics to retrieve the information we need.

Login into your Qualtrics system and go to Account Settings and then to Qualtrics IDs


Here you can retrieve the following parameters

  • Datacenter ID

  • API authentication parameters

    • Token

    • User ID



  • Survey ID


 

 

SAP Data Intelligence pipeline


Last, let’s create a data pipeline in SAP Data Intelligence Cloud.

In this GitHub repository you can find a ready-to-use SAP Data Intelligence graph, with a connector to Qualtrics and a connector to SAP Signavio Process Intelligence.

You can download and archive the content of the 3 folders (that you can find in the repository) into 3 zip files, so that they can be imported as solutions into your SAP Data Intelligence tenant. Do not archive the folders, but for each folder select the content and “add to archive”.

In order to import solutions in SAP Data Intelligence, you can go to System Management, go to Files, click on the '+' symbol, select "Import Solution" and choose your zip file.

 

In SAP Data Intelligence, you can create an OPENAPI connection to Qualtrics

  1. Host: qualtrics.com

  2. Authentication Type: Basic

  3. Username: the Qualtrics data center ID (can be found under your Qualtrics User, Account Settings, Qualtrics IDs)

  4. Password: the Qualtrics token (can be found under your Qualtrics User, Account Settings, Qualtrics IDs)



 

In SAP Data Intelligence, you can create an OPENAPI connection to SAP Signavio

  1. Host: spi-etl-ingestion.eu-prod-cloud-os-eu.suite-saas-prod.signav.io (note: the API endpoint url of Ingestion API will change soon)

  2. Authentication Type: Basic

  3. Username: it's not relevant, you can type "signavio"

  4. Password: the Ingestion API token (can be found in the Ingestion API data source)



 

In SAP Data Intelligence, you can open the Modeler and add the imported graph ("Qualtrics to SAP Signavio").



 

You can configure the graph.

  1. In the Qualtrics operator, add (1) the connection, and (2) the Qualtrics Survey ID

  2. In the SAP Signavio operator, add (1) the connection, and (2) the name of the target table to be pushed in SAP Signavio Process Intelligence


 

You can now save and run the pipeline.

Once its status is “completed” you can go to the SAP Signavio Process Intelligence Ingestion API integration and check the logs.


A new execution log will appear, with “Running” status. After a few seconds the status will become “Done” and you can check in the Tables tab that a new table has indeed been uploaded, with the name provided in the second python operator of the DI pipeline.


 

The data is now ready to be used for process mining!

 

 

Additional step: Add Experience Data to your Data Model!


As we stated above, the whole idea started with the business requirement to add Qualtrics data to the Incident to Resolution process, to enrich operational data with the customer experience perspective.

To achieve that, you can create a new data model selecting ServiceNow as source system.


Select the available Incident-to-Resolution transformation template, as an accelerator to transform raw data into an event log with a series of preconfigured SQL scripts.


Select New Integration and connect to an existing ServiceNow data source.


 

With the new Process Data Management (ETL 2.0) interface which was released in June 2022 you can now see the whole data pipeline, spanning connection, integration, and transformation. You can add the target Process Intelligence process.


Click on “+ Data source and integration” and select your Ingestion API data source. It will be added to the data pipeline.


 

Now you can start changing the existing SQL scripts to combine operational data with experience data coming from Qualtrics. For testing purposes, you can edit the Transformation block by adding a new “Qualtrics” business object.


At this point you can add a new event collector with a simple SQL script to preview the survey data that has been extracted from Qualtrics and pushed to the Ingestion API integration through the SAP Data Intelligence data pipeline.


 

Qualtrics survey data is now ready to be combined with ServiceNow data to create new valuable insights!

 

 

 

Further steps


As a summary, in this blog post I have covered:

  • How to create and use an Ingestion API data source to push external data into SAP Signavio Process Intelligence

  • Which information you need to gather from your Qualtrics system in order to extract data from a survey, and how you can retrieve it

  • How to connect SAP Data Intelligence Cloud to Qualtrics and then create a data pipeline with two operators, that respectively:

    • Connect to Qualtrics and extract survey data

    • Push the data to SAP Signavio Process Intelligence via Ingestion API




 

Follow the SAP Signavio Community and the SAP Data Intelligence Community for similar content, peer-to-peer networking and knowledge sharing.

 

 

 

Appendix


If you’re interested to know more about Ingestion API, here are some additional resources that you can check:

In this GitHub repository you can find the data pipeline and the two python operators to be downloaded and imported into your SAP Data Intelligence tenant.