Govern Data Traffic with SAP Data Hub

[molson]

Earlier in the fall, SAP announced a new data management solution to the marketplace known as SAP Data Hub. This solution was designed to address a problem that can sometimes arise with the Big Data Conundrum: gathering massive amounts of information only to have to figure out how to operationalize it to make sense, then in doing so throw all that raw data into many targets (creating a Data Lake) and then have to figure out how to map it to find what you need for your operations.

 

In order to understand what SAP Data Hub is there is a great introductory article written by Marc Hartz, Product Manager for SAP Data Hub that you can read here. However in this article I will explore the use of SAP Data Hub for ISV companies that create products or services that deal with a barrage of information and how it can help simplify tasks related to dealing with the Big Data Conundrum.

 

For example, think of when you travel for business to a customer's location there are many different processes (booking a flight, hotel, rental car), workflows (leaving home to the airport in a taxi/ride hail) and tasks (checking bags, getting something to eat, etc.) that are done directly by you or indirectly by your travel. Now think of all the data that needs to be gathered to make the necessary decisions to complete your travel. The different execution paths that need to occur for the many tasks related to your travel and what happens when some of that data and tasks become unreliable. How does all of that affect the outcome of you arriving at your customer business meeting on-time?

 

SAP Data Hub was designed with this type of challenge in mind by focusing on the following key areas*:

• Data Integration. Provide a robust and extensible way to interface with many different types of data sources both SAP and non-SAP in the cloud, on-premise or in a hybrid setup.


• Data Orchestration/Pipelining. Manage workflows of information and execute custom processes across a distributed environment for multiple tasks.


• Data Governance. Ensure the data is reliable, traceable and is compliant in accordance with your business.

*See the introductory blog by Marc Hartz for additional areas of focus for SAP Data Hub.

 

So how does this apply in the world of ISV organizations that have to deal with managing the data traffic in our business travel example?

1. Let's apply the Internet of Things here since more systems and devices are being connected each day to help simplify our travel experience. As more devices become interconnected the number of data streams seems to grow exponentially increasing the challenge of data integration and ingestion for an ISV that needs to manage the velocity and volume. SAP Data Hub is designed to connect to information at the source and not move it to a central location or hub (although the name implies that). Whether the information is kept in local data stores, customer owned data stores, cloud services, or remote streaming devices SAP Data Hub has a simplified UI to connect, discover and profile source data.


2. Once the data comes in how do you know where it's going and how do you implement controlled processes that are triggered based on variable data traffic? The Data Orchestration capability in SAP Data Hub enables the creation and monitoring of workflows and the data that flows through it across a distributed landscape. The ability to create data pipelines for flow based applications consisting of reusable and configurable operations (i.e. data preparation, ETL processes, code execution, functional operations, etc.) enables the execution of custom processes that are automated reducing or eliminating manual intervention. This can significantly reduce the operational costs for ISVs when supporting a SaaS environment.


3. Now that data is connected and flowing through the correct pipelines in the system how do you enforce governance around it so that it is compliant with your business policies? With Data Governance in SAP Data Hub an ISV can take advantage of a metadata repository to store information about the connected landscape, enforce access policies to dynamically secure data from source to destination, and mask and anonymize data to protect sensitive sources. For example, companies who do business in the EU will need to comply with GDPR the European General Data Protection Regulation that comes into effect on May 25, 2018. Having a system that can enforce policies pertaining to the proper handling of personal identification of EU citizens will be mandatory as per EU law.

 

To learn more about the capabilities of SAP Data Hub, please visit our channel on YouTube.

 

While the above is meant to serve as a simple example of data traffic control; organizations of all sizes will need to address how to handle the proliferation of data flowing through their systems as information will continue to grow in accordance with our desire to connect everything in our world.

 

If you are interested in understanding more about how SAP Data Hub can be utilized in your product or service as an ISV, please contact us to find out more on how you can embed and OEM some of SAP's technology into your own applications.