Cloud Integration Pipeline - Datastore Extension UI

[lisarizzo]

⚠️  Disclaimer: The UI described in this blog post is not an official product by SAP. If you are interested in using the UI, you can find the published code and set up instructions on the official SAP-samples GitHub here:   https://github.com/SAP-samples/cloud-integration-pipeline-datastore-extension-ui

Purpose and background

The pipeline in Cloud Integration is a well-established framework for asynchronous integrations. A blog post to explain the starting points for the pipeline can be found here.

In addition to the pipeline, we offer the datastore extension, which simplifies error handling via automatic retry jobs and APIs for resending and cancelling messages. For details on the datastore extension, please refer to this blog post. The datastore extension package provides automatic restart capabilities with Restart Jobs that are based on pre-delivered IFlows. In addition, it provides on-demand operations that require the use of API client tools such as Postman or Bruno. This is technical and error-prone to handle from a user perspective.

The UI introduced in this blog post is designed to provide a simplified and user-friendly approach to access these on-demand APIs.

On a technical level, it is relevant to note that the UI does not call Cloud Integration APIs directly. It calls IFlows of the Datastore Extension package, which are on-demand API Operations (Manage Retries and Manage Datastore Entries). It triggers a corresponding IFlow and operation and then displays the response.

This blog article introduces and explains the functionality of the Cloud Integration Pipeline Datastore Extension UI.

Overview of key features

The current version of the UI provides these capabilities, which are shown again in more detail in the “How to use the UI” section.

• Visual overview of all available datastores relevant for the pipeline reprocessing, their messages and overdue messages
• Search and filter capabilities to quickly locate specific datastores
• Visual overview of all messages in each datastore
• Search and filter capabilities to quickly locate specific messages
• Action to restart messages from the UI
• Action to move messages to another datastore or a no-retry datastore when located in a retry datastore
• Action to delete/cancel messages 
• Detailed view of message content (error, headers, payload)
• Possibility to perform batch actions to restart, move, cancel several messages at once

How to use the UI

Datastore overview

The first screen of the UI shows a table with all the datastores, the number of messages, and the number of overdue messages. You can also search and filter the table for specific information. 

Datastore detail view and overview of messages

The second screen shows details of the selected datastore and contains a table with all the messages. 

One or more messages can be selected in the table and moved to another datastore or to a no-retry datastore. The messages can also be restarted or deleted from here.  

There are sorting capabilities available for the datastore entries table. They can be found behind the gear icon ⚙️. For example, the table can be sorted by the number of restarts and the timestamp of error, ascending or descending. 

There is also a filter hidden behind the same gear icon ⚙️.

Message detail view

When clicking on one of the rows in the datastore entries table the user is directed to a third screen with details about the error, headers and payload. If, after inspecting, the user decided what to do with the message it is also possible to perform the same move, delete and restart actions from this screen with the buttons in the top right corner.

Outlook & how to implement the UI

This project is available as an open-source project on the SAP-samples GitHub organization: https://github.com/SAP-samples/cloud-integration-pipeline-datastore-extension-ui . Here you can find the repository and a detailed implementation guide. If you notice any issues or have further ideas feel free to contribute to the project. You can also reach out to @lisarizzo or @mike_sibler  if you have any further questions.