Data volume problems during an SAP S/4HANA Conversion

[YvonneNozukoMaboyi]

An SAP S/4HANA conversion refers to migration from a legacy SAP ERP system, such as SAP ECC, to the modern SAP S/4HANA business suite. This process is essentially a platform upgrade that includes adapting data system configurations, and custom developments to ensure compatibility with S/4HANA. Click here to read more about the features of S/4HANA.

There are three approaches available when embarking on this journey (read a more in-depth discussion about them here)

1. Greenfield approach - means starting from a clean slate.
2. Brownfield approach - is more like a system upgrade.
3. Hybrid approach - lets you select the best parts of Greenfield and Brownfield implementations.

Many customers embarking on this transformation journey opt for the brownfield conversion approach. The brownfield approach retains your historical data which can lead to the technical challenge we will discuss and solve in this blog.

SAP provides a process for the conversion to SAP S/4HANA, this is illustrated in the figure below which gives an overview of the tools, the phases, and the activities involved in the process. The activity that we will be focusing on is t5 (Software Update Manager (SUM)), specifically the “Data Conversion” task:

For further reading:

• Conversion phases and related activities are documented in the  “Conversion Guide for SAP S/4HANA and SAP S/4HANA Cloud Private Edition 2023”, which can be found under the Implement section of the SAP S/4HANA | SAP Help Portal.
• For information specific to accounting components please refer to the following SAP note 2332030 - Conversion of accounting to SAP S/4HANA - SAP for Me.

The “Data Conversion” task is the conversion of your data into the new data structure used by SAP S/4HANA.

At this point you could be wondering “What new data structure used by SAP S/4HANA?”, well I’m glad you asked, let me explain.

The data model used in SAP S/4HANA has been simplified drastically when comparing it with the data model used in SAP ERP. If you look at the Finance section of the below diagram you will see that many of the legacy tables have been consolidated into a simplified data model focused on the ACDOCA table.

Furthermore, the effects of Accounting Data Model Changes and how they impact the conversion are described as follows in an extract from the Converting Your Accounting Components to SAP S/4HANA guide:

To use Finance in SAP S/4HANA you must migrate the existing user data from the General Ledger, Asset Accounting, Controlling and Material Ledger. The data migration is necessary because Finance in SAP S/ 4HANA rests on a uniform data model for all accounting areas. The comprehensive ACDOCA data table contains all line-item documents. After the migration, all postings of the named applications are written into the ACDOCA table.

The following tables are replaced by views using the same technical names:

• The line item, totals tables and application index tables of the General Ledger (GLT0, BSIS, BSAS and FAGLFLEXA, FAGLFLEXT, FAGLBSIS, FAGLBSAS).
• The totals tables and application index tables of Accounts Receivable and Accounts Payable (KNC1, KNC3, LFC1, LFC3, BSID, BSIK, BSAD, BSAK).
• The line item and totals tables of Controlling (COEP for certain value types, COSP and COSS).
• The material ledger tables for parallel valuations (MLIT, MLPP, MLPPF, MLCR, MLCD, CKMI1, BSIM).
• The Asset Accounting tables (ANEK, ANEP, ANEA, ANLP, ANLC).

This is the crux of our challenge: data that was previously distributed across multiple tables in the legacy system is now consolidated into a single table in the S/4HANA environment, that is ACDOCA.

Although this may not appear to be an issue under normal circumstances, in high-volume scenarios, technical limitations of the HANA database platform can arise when a single table (or partition) approaches a very large number of records. Specifically, each table or partition in HANA is limited to a maximum of 2 billion entries.

The solution: table partitioning!

Let’s do a deep dive into partitioning and its role during conversions by exploring a few questions.  

Question 1: What is table partitioning?

According to  SAP Help Portal: The partitioning feature of the SAP HANA database splits column-store tables horizontally into disjunctive sub-tables or partitions. In this way, large tables can be broken down into smaller, more manageable parts. Partitioning is typically used in multiple-host systems, but it may also be beneficial in single-host systems.

Question 2: Why do we need to do partitioning?

As mentioned earlier, there is a limitation of 2 billion records per table (partition) on the HANA database. This is described in more detail in SAP Note 2154870 - How-To: Understanding and defining SAP HANA Limitations - SAP for Me.

If the limit is reached the following errors could be generated:

• 2048: column store error: [17] Docid overflow in current delta.
• 2055: maximum number of rows per table or partition reached: '<schema>.<table>'
• Maximum number of rows per partition reached.
• Max row count exceeded.
• exception 1: no.3100025: Allowed rowcount exceeded.

Below is an example of a short dump with the error “maximum number of rows per table or partition reached” that can occur in step ID MUJ “Data Migration into Unified Journal: Line Items”:

Question 3: How do we know that we need to do partitioning for the conversion?

The Readiness Check which forms part of the Preparation Phase can guide the need for partitioning. It details the table records in the existing tables in ECC (classic General Ledger or New General Ledger), and also gives an estimate of the number of records expected in ACDOCA. See the table below as an example:

The calculation is not exact, but it already gives you an idea that the number of records will exceed 1 billion for ACDOCA, in this case, partitioning is required.

Further Reading: Great community blog - SAP S/4HANA Simplification Item Check - How to do it right.

Note: Ensure that all relevant notes are implemented to get thorough analysis and results.

Question 4: How should we partition the tables?

Luckily there is quite a bit of guidance on partitioning in general, as well as specific guidance for Finance Tables, please review the following:

1. SAP Note 2044468 - FAQ: SAP HANA Partitioning
2. SAP Note 2289491 - Best Practices for Partitioning of Finance Tables - SAP for Me
3. For scale out scenarios read SAP Note 3325787 - Partition-Distribution of Financial Data in SAP S/4HANA
4. Guidance provided in SAP Customizing Implementation Guide:

When designing a partitioning strategy, it is important to analyze how the data is distributed over Fiscal Periods and Years, and to anticipate future data growth.

Note: Pay attention to the following guidance from SAP Note 2289491 - Best Practices for Partitioning of Finance Tables when creating future partitions à Too many empty partitions can affect the HANA optimizer and cause poor performance for queries which don't apply partition pruning.

Question 5: When should partitioning be done during the conversion process?

The partitioning of the ACDOCA table, in the target system, can be done once the SUM tool has reached the “business downtime” phase (this phase is known as DOWNCONF_DTTRANS) on the source system (in SUM uptime phase).

Note 1: Other tables are partitioned automatically by the SUM tool.

Note 2: If partitioning isn’t performed before the data migration it is recommended to reset the migration rather than trying to apply partitioning with an already populated ACDOCA. Once records exist in ACDOCA, applying partitioning becomes significantly slower. It's recommended to reset the migration (Step ID MUJ is where ACDOCA begins to be populated) - this clears the records in ACDOCA — then proceed with partitioning.

Before we wrap up, we would like to mention that it is very important to focus on Data Volume Management before conversion.

Benefits are:

1. Reduced data footprint which results in a shorter conversion duration.
2. Reduced memory requirement for the new SAP S/4HANA system.

The following guide and notes contain useful reading on the topic:

• 2818267 - Data Volume Management for FI/CO during migration to SAP S/4HANA
• Data Volume Management during an SAP S/4HANA Conversion

Hope you enjoyed this blog! Please feel free to leave suggestions and comments.

This blog was co-authored by @DotEiserman.