Best Practices Recommendations for SAP Datasphere

[alexandergregor_scivos]

10 Best Practice Recommendations for working with SAP Datasphere

SAP Datasphere, SAP’s cloud-based data warehouse and data fabric solution has become a cornerstone for modern data architectures and gained visibility during the past months, especially as it is the core of SAP Business Data Cloud. SAP Datasphere  enables organizations to integrate, model, and consume data across hybrid landscapes. To unlock its full potential, companies should follow a set of proven best practices.

Here is a “best 10” list:

1. Establish a clear data architecture. Define layers, domains, ownership, and naming conventions upfront to avoid inconsistencies later. Naming conventions should indicate stable properties of your objects, such as:

• the level of harmonization,
• if it combines several sources or which source system its data is derived,
• if a table is delta capture enabled.

2. Use Spaces. Use as many spaces as needed to effectively separate business units, projects, or environments, ensuring proper governance and isolation. It’s a good idea to have different spaces, for example, for specific consumption areas. But don’t create too many spaces as they need to be maintained in scoped roles, more objects need to be shared, and resources need to be allocated if you use spaces highly. 

3. Adopt a data product mindset. Build reusable, well-documented datasets with clear business meaning. Use the built-in options for documenting the artifact’s business purpose.

4. Indicate Semantic Usage. Use relational dataset only for low-level objects. On high level, use fact for data that contains aggregated measures, and dimension for pure language-independent attributes based on a specific key value. Use text for language-dependent descriptions. Note that changing the semantic usage is possible but may require additional specifications, e. g. a language field.

5. Leverage the Analytic Model for business-specific KPIs and field names, but ensure consistent definitions for base measures, and dimensions in the underlying views.

6. Minimize data replication. Whenever possible, use virtualization to reduce redundancy. Of course, there are cases when remote access of mass data repeatedly for several queries is too slow or puts too much stress on the source system. In such cases, implement a replication flow and transformation flow with delta capture for a performance-optimized ELT process.

7. Design models with performance in mind. Reduce the data set early. Apply filters directly on original fields, push filters down to the source system. Prefer associations over joins, optimize joins, avoid unnecessary calculations, and monitor query execution. A key best practice is to adopt a tiered data architecture. Define different tables or partitions for different time slices. Frequently accessed, business-critical data (hot data) should remain in Datasphere’s “hot” core storage (in memory) for optimal performance. Less frequently requested data can be kept in “warm” storage (on disc), or even in a “cold”, file-based storage space to reduce costs (capacity units).

8. Implement strong security practices. Define roles for space-level and activity-based authorization control with as few overlap as possible. Use data access controls (DAC) to control data visibility mainly on one level, for example on the highest view before the Analytic model. Implement auditing and export audit log entries before they are deleted. Enable a password policy. Use encrypted communication for network communication channels.

9. Establish lifecycle management. Define CI/CD processes to handle changes reliably across environments (tenants or spaces). Pre-define topic-related packages to combine what needs to be transported in the same way, but keep packages small enough to handle the data imports. If you redefine the space during import, develop and follow a clear guideline for space mapping.

10. Finally, integrate SAP Datasphere within SAP Business Data Cloud. Enrich the possibilities with data science analytics tools like SAP Databricks or Snowflake. Use SAP Analytics Cloud to deliver business value through intelligent applications, and use seamless planning to combine the possibilities of both tools.

By combining governance, performance optimization, and user-centric design, organizations can build a scalable and future-proof data foundation with SAP Datasphere.

Final remark

Many aspects of best practice are specific to your needs and skills. For example, if you are familiar with SQL, you may be able to optimize the data access in a complex case by a smart SQL script in an SQL view. However, as the system will stick to your sequence of steps, if your program design does not match your data patterns, especially if patterns in your data vary, a graph view may be the better choice. (The system follows general performance optimization strategies for graph views which can make them faster even if they are badly designed.)

Moreover, check out our product home page SAP Datasphere | SAP Help Portal | SAP Help Portal especially the SAP Datasphere Security Recommendations | SAP Help Portal.

Feel free to ask your questions and present your own recommendations here in the community. For discussing your specific questions directly with a subject matter expert, visit our instructor-led training DSP01 – Introduction to SAP Datasphere:

DSP01 - Introduction to SAP Datasphere | SAP Training

[TomasSolil]

Hello, 

Most of these points are clear and I dont think anyone would disagree with them. 
However there are two points that, from my experience on multiple Datasphere implementation projects, are in reality much more nuenced and I think needs more context.

Specifically point on Spaces and Data replications. 

Spaces:
While overall I do agree on the point you are making I would rather put it as "Use as few spaces as possible to still effectively separate..."
I have seen too many cases where high amount of spaces caused a lot of overhead work, namely with package transports between tenants while not providing enough benefits to justify it. I understand its sometimes needed if each part of business have their own IT department to support them, but that usually not the case.

So my usuall recommendation is rather to have space specific for Modelling (Developers) and Reporting (Where key users can model their AM, on top of shared views and models) and, where it make sense, have a space for authorization setup. 
And then only have separate space for highly confidential data like have HR data for example.

Data replication:
Yes, its important to get away from BW mind set where data where replicated after each step and physically stored. However, it needs to be concidered that Memory usage is way more pricey then physical space in Datasphere.
So its, in my opinion, unless there is a strong business reason, to replicate (persist) data in "final" view which is used in Analytical models. Even if it means data Duplication when there are multiple level of such views for different analytical models. It support performance and is more cost effective. 

Similar way of thinking should go for data replication across systems. 

Best Regards,
Tomáš Solil

[alexandergregor_scivos]

Hi Tomáš 

thank you so much for sharing your experience. What you say about designing spaces is exactly what I wanted to get across: "use as few spaces as possible to still effectively separate authorization - just as many as needed to do so."  If there is a use case for separating authorizations, like for securing HR topic or FI topic, a dedicated space should be set up, but the overall message should be: Try to keep the number of spaces small, to avoid an overhead of administrative tasks and shared entities. Of course, it is like with any authorization concept: Try to keep it simple, but allow the required limits and flexibility.

However, there is one further aspect that directs us towards a smaller number of spaces.  Within a space with many objects, you can group models into folders and sub-folders to separate topics and modelling layers. In contrast, if you create separate spaces, you need to share many objects, and these shared objects appear on the main level of the target space and can't easily be organized within the target space.

Regarding Virtualization versus Replication, it is really a matter of balancing the pro and con. Yes, Datasphere offers cheaper ways to store data sets, but so does BW if you use Near-line storage or Data Tiering Optimization. The main questions to decide this question are: 

* Do you need totally up-to date values (then go for virtual models) or is some latency acceptable (then replicate the data)?

* How often (and with which performance expectations) are the dataset analyzed repeatedly? (Replication is most important for frequently needed data.)

* How large is the replicated data volume? (Accessing large data sets across systems is rather slow, so the need to optimize performance is higher, but the storage will be more expensive, too.)

* Should we rather replicate values to reduce the stress on the source system? (Yes, try to extract only once instead of frequently accessing the data. But you don't want to replicate a data segment that is hardly ever needed.)

Persisting views (on final level before the Analytic Model) is a good idea to improve performance for a specific purpose, but it generates additional tasks or confusion when the view is changed. So, I recommend it mainly for short-term storage or for stable models.

I am looking forward to more and other opinions, experience, and discussion.

Thanks again for sharing your thoughts.

 Alexander

[FernandoPerezMerino]

I fully agree with separating authorisations into a single space, and one suggestion SAP should take into account is that, in the intelligent apps it is proposing to customers, there is a Central Permissions table that is used in the ingestion space rather than being placed in a separate space. Why am I mentioning this? Because when you want to customise the harmonisation and analytics spaces, you have to make a copy of them, and in the copy, you may not want to have the same level of authorisations as you have in the standard intelligent app, so authorisation maintenance is duplicated. And if, on top of that, you want a separate space to build all your customisation and keep it separate from the copy, you have to set up another level of authorisations in that space, meaning maintenance is tripled. What would be the solution? In SAP Intelligent Applications, generate the DACS and the Central Permissions table in a separate authorisation space, so that these authorisations can be shared across the different usage spaces. Wouldn’t that be more optimal?