On Tuesday 26th of November, we presented the latest Customer Data Cloud Webinar with a focus on how to activate your customer data to build trusted relationships. The presenters of the session were:
Ratul Shah – Senior Product Marketing Manager, SAP Customer Data Cloud
Stephen Purvis – Product Expert, SAP Customer Data Cloud
Julien Goulley – Technical Architect, SAP Customer Data Cloud
In the webinar, we discussed the importance of building a trusted relationship with customers while ensuring compliance with data protection rules, before going on to showcase how to implement synchronous and asynchronous integration in a working demo.
If you were not able to attend the live session, you can find a link to the recording here.
This blog post explains the steps required to build real-time, near real-time and batch integrations on top of the existing Webinar demo site used in previous Webinars in September. The functionality features:
How to ensure a high level of data quality by using Extensions
How to build a batch integration using Identity Sync
How to support near real-time integration processes with Webhooks
Like client-side integration, Extensions allow real-time integration with 3rd party APIs. However, unlike client-side integration, Extensions endpoints centrally manage customisation, allowing customers to reduce implementation costs and simplify maintenance and implementation of changes.
In the example below, we'll imagine we are working with a customer who is managing food catering for SAP employees. Although the customer website allows self-registration, only SAP employees should be allowed to register and login to the website.
When building an Extension endpoint, you first need to understand the format of the request and the response.
The request body contains a JWS (Json Web Signature) token composed of 3 parts separated by dots. The first part is the header, which includes the algorithm and key ID used to sign the token. The second part is the payload and includes:
the target api key
the extension point (register, login or update)
the data submitted
the current data stored in Customer Data Cloud (not applicable to registration)
/** FUNCTIONS **/
// Build HTTP Response
function json_response($message = null, $code = 200)
// clear the old headers
// set the actual code
// set the header to make sure cache is forced
// header("Cache-Control: no-transform,public,max-age=300,s-maxage=900");
// treat this as json
header('Content-Type: application/json; charset=utf-8');
$status = array(
200 => '200 OK',
400 => '400 Bad Request',
422 => 'Unprocessable Entity',
500 => '500 Internal Server Error'
// ok, validation error, or failure
// return the encoded json
// Base64 to URIBase64
$str = str_replace(['-','_'], ['+','/'], $str);
Once the endpoint is up and running, the configuration of the Extension in Customer Data Cloud is done from the “Settings” area of the parent site. Within the “Extensions" section, click “Add”.
The Extension configuration form will be displayed and that's where the details of the endpoint are defined:
Pick a name
Enter a quick description
Enter the endpoint URL
Define the timeout (max 5 seconds)
Define the fallback policy; if FailOnAnyError is selected then any timeout would fail the registration (or the API selected earlier)
From that point on, anyone trying to register with an email address outside the SAP organisation would receive an error message.
Identity Sync is an ETL (Exchange, Transform and Load) platform which allows orchestration and management of batch import/export. In Identity Sync, you have the ability to create dataflows which are composed of blocks or components. The list of components breaks down into 4 categories:
Data source: read, write and delete
File: format/parse and compress
Record: custom field transform
To build a batch integration you must first create a dataflow. New dataflows are usually created from predefined templates, but there is also the option to duplicate an existing flow. Firstly, you need to navigate to the Dataflows section under Settings > IdentitySync and click Create Dataflow.
At that point, using a mix of clicks and drag&drops, you can remove components which are not needed (by clicking the red bin icon), add new components and link them to each other.
For exports, the final step of the flow is a 'writer' which is used to push the data out of Customer Data Cloud. The data can be transferred to a repository (SFTP, S3 etc.) or pushed directly to the external systems using standard connectors or the generic api component.
Once the dataflow is complete, it can be scheduled to run once or on a regular basis. During scheduling, you have the option to:
define the frequency
define the execution type (Delta vs. Full)
define the log level (Debug, Info, Trace)
define the distribution list for the success and failure notifications
Schedules can be added or edited by navigating to the Scheduler menu as follows.
Finally, you have the option to monitor previous runs of your dataflow by navigating to the Status menu. The logs are categorised under 3 sections:
Trace section shows the logged statements
Step metrics section shows the duration and counts for each dataflow step
Errors section shows details about the records that failed
Some business processes require user profiles to be provisioned in 3rd party systems as quickly as possible after the user creation in Customer Data Cloud. A typical scenario is that a consumer registers online to report an incident with their product and calls the support centre immediately afterwards. At this point, the support engineer should be able to find the consumer in the appropriate support system. Webhooks is a near-real time event-based mechanism which enables notifications to be sent to an external endpoint. This endpoint can then process the notifications and synchronise the data accordingly.
The first step when setting up Webhooks is to build the endpoint. Any technology can be used to host the endpoint, as long as the below can be achieved:
Support HTTPS requests
Verify the HMAC-SHA1 hash
Queue events for asynchronous processing
Make requests to Customer Data Cloud API (accounts.setAccountInfo)
Make requests to 3rd party APIs
For the demo, I hosted my endpoint in SAP Cloud Platform Integration. Below is a screenshot of the iFlow.
Once the endpoint is up and running, the configuration of the Webhooks in Customer Data Cloud is done from the “Settings” area of the parent site. Within the “Webhooks" section, you’ll have to click on the “Create Webhook” button. At that point, you can complete the configuration:
Define the endpoint URL
Define the signing secret (partner secret or application secret)
Select the triggering events
Define custom headers (optional)
Once the configuration is saved, notifications start flowing to the endpoint. The below screenshot shows an example of payload and hash.
This ends our blog post which discussed how we can use the SAP Customer Data Cloud platform to build a trusted relationship with customers while ensuring compliance with data protection rules. It has show-cased the tools to build secure integration and how easily they can be used.
To learn more about SAP Customer Data Cloud, please sign up for our next webinar.