In his blog series on Data Mesh Wolfgang Epting gives a comprehensive and detailed introduction to Data Mesh and which components in the SAP Business Technology Platform (SAP BTP) can be leveraged as foundation to implement Data Mesh in an organization.
In this blog I wanted to add an additional angle to the discussion. While the phrase “The winner takes is all” might be true in many areas of life, this mustn’t be the case in the context of a Data Mesh project. The question which technology to leverage to implement a Data Mesh is not an either-or question. SAP BTP is well suited to be leveraged as part of a ‘bigger’ platform supporting a Data Mesh implementation.
I use term BTP based data products for data products hosted on SAP Business Technology Platform (as described in Wolfgang’s blog series).
Data Mesh leveraging SAP and Non-SAP Technology
Looking at your data from a (SAP-biased) bird’s view there are domains mainly residing in SAP Systems and others in non-SAP Systems. When starting your Data Mesh project with data products in the non-SAP area, non-SAP technology could be the best choice to build your initial data products. Your Data Mesh initiative is initially based on non-SAP technology.
As soon as you start to include data products and domains residing in SAP systems (S/4HANA or ECC or ERP, and SAP Data Warehouses), you should consider leveraging SAP-Technology. Here is why:
Not only do SAP systems contain high value data, but they also contain domains and domain knowledge. I.e, SAP S/4HANA contains business and entity data models for customer data in a company.
Out-of-the box data products are delivered as content packages (in SAP Analytic Cloud and SAP Datawarehouse Cloud see Wolfgang’s blog series), which can be tailored to your needs.
SAP BTP offers supreme out-of-the-box integrations to SAP systems, enabling easy re-use of the semantic models and domain data models as well as virtual real-time access to the actual data.
Your Data Mesh will be built on SAP and non-SAP technology. The combination of both technologies will serve your needs best.
[Remark: As explained in Wolfgang’s blog series, SAP BTP can be leveraged for non-SAP data as well, especially to build data products containing SAP and non-SAP-Data. If most of your data products rely on data in SAP Systems, leveraging only SAP BTP for your Data Mesh initiative is an option, you should consider.]
Here is how SAP BTP fits into a Data Mesh platform combining SAP BTP and non-SAP technology. Let me go through the 4 pillars of Data Mesh to explain how the pieces fit together.
Domain-oriented decentralized data ownership and architecture
Domain Ownership is a key principle for a Data Mesh. To view SAP S/4HANA or ECC or ERP and SAP Data Warehouse systems only as data provider is too short sighted. These systems contain domains and domain knowledge. The users of these systems are domain experts, both on the IT as well as on the business side. SAP systems serve as trusted source of information on customers, products, equipment and many more domains.
Therefore, all the domain knowledge within your SAP eco system (IT and LOB) in your company can be leveraged by including SAP experts in your domain teams. Technically the well-established SAP data and semantic models can - and should be - reused for building data products.
Figure 1: SAP Business Technology Platform as part of a 'bigger' Data Mesh: Domain Ownership and Data as a Product
Data as a Product
SAP S/4HANA or ECC or ERP and SAP Business Warehouse systems are a great source to build source-oriented data products. Data products for data in SAP Systems can be best build (see above) based on the SAP BTP.
SAP Data Catalog entries can be published/integrated via APIs to/with an Enterprise Data Catalog leveraged in a Data Mesh. BTP based data products can be easily consumed by standard APIs and within a Data Mesh.
Self-serve data infrastructure as a platform
Provisioning of underlying the platform based on a self-serve data infrastructure is another key pillar for a Data Mesh initiative. The SAP Business Technology Platform does not offer Infrastructure as a Service, but all services can be provisioned either via automation based on scripts or in a self-serve manner. The infrastructure and resources needed for the services are provisioned alongside with the service.
Scripts for provisioning the SAP BTP services can be included in an overarching approach for providing a Self-Services Data Infrastructure. I.e., domain teams can leverage the BTP Cockpit for instantiating the SAP BTP service needed to build and deliver data products.
Figure 2: SAP Business Technology Platform as part of a 'bigger' Data Mesh: Infrastructure and Federated Governance
Federated computational governance
To enable the interoperability of self-sufficient data products, there is a need for federated computational governance. Rules defined for the Data Mesh can be federated to the BTP based data products. Computational execution of the federated rules is supported by SAP BTP services.
SAP BTP fits well as part of a platform supporting a Data Mesh. As always detailed questions must be clarified based on your specific requirements. The integration options are available.
When not already done so, you can start with Wolfgang’s blog series on Data Mesh to understand SAP’s technology which can be leveraged as basis for your Data Mesh initiative.
Especially, when data products based on data from SAP Systems are on the roadmap, you should consider SAP Business Technology Platform build and operationalize these data products.
I am very interested in your comments, please leave these below.