Introduction to SAP Data Archiving

[sanjay_prasad5]

Being SAP FICO consultant I have got the opportunity to implement archiving solution as a part of ‘Data volume management’ for reducing the data footprint of ‘business complete’* data and moving forward with only online open items and business critical data for business continuity.

(*Business-Completed Data: Data that has no further processing like an Invoice that has been settled and cleared by a payment)

Since these days every organization is looking forward to move their SAP applications from traditional On-Premise to modern Cloud Database so it becomes essential to reduce the data volume at the online or active database as more data means more downtime during Cloud or HANA Migration

As a part of series of blog posts will share with you the data archiving prerequisites, activities to be performed by SAP functional & technical consultant, ‘Open Text’ consultant and Business Users.

Introduction to SAP Data Archiving

Before going into the details of archiving, let’s understand first the lifecycle of data and at what stage or under which conditions we can consider data for archiving.

Fig.1 (Source: Self)

Here Fig.1 shows the flow of information system data throughout its lifecycle: from creation and initial online storage to archive storage to the time when it becomes obsolete and is deleted. Newer data and data that must be accessed more frequently is stored on faster but more expensive storage media in SAP while less critical and less frequently accessed data is stored on less expensive media outside SAP.

Let’s understand what it means by Archiving in SAP

SAP Data Archiving is the method supported by SAP to remove business-complete application data from the database in a consistent, secure, and comprehensive manner and to store it in such a way that it can be accessed in the future

“Archiving” refers to the process associated with copying data and supporting documents from an active system (SAP) to an external source (Open Text) for the purpose of deletion and/or storage for later retrieval.

There are basically 2 approaches suggested by SAP to archive data

• Classical Archiving


• ILM (Information Lifecycle Management)

Here in my blog post will be covering only 'Classical Archiving' at present

Summarized Benefits and Business Impact 

Data Archiving provides the mechanism to move static, business-complete data records from the active SAP Database to Non-Sap archive storage.

Data archiving provides several advantages:

- Improved system performance and response times: Large data volumes in application tables result in long run-times for transactions and reports. A smaller amount of data in the active database improves performance for dialog transactions and reports, especially in areas where all records in a table have to be processed. Removing data from the online database will allow transactions and queries to be completed in a shorter time. End users will have to sort through less data to find the required information.

- Lower hardware and administration costs: Data Archiving will help avoid additional hardware costs and reduce administration costs. Archived data gets compressed during the SAP archiving process. The compression factor is dependent on the actual data which gets archived. On average, a ratio of 5:1 is a conservative estimate for the actual compression ratio generated by archiving. Archived data can be stored on less expensive disk systems than active database records, since no updates are performed on archived data.

- Greater system availability due to reduced down-time during release upgrades, faster back-up and recovery, and reduced runtime for database reorganization: Reducing the amount of data stored in the active database reduces the execution time for administrative tasks like backup/restore, database reorganization, system upgrades, etc. By reducing the data volume in the database, Data Archiving helps to minimize the time a system is unavailable.

- Compliance with any regulatory or audit requirements: General accounting rules and industry-specific regulations frequently require retention of relevant information in a format which protects data from modification and premature deletion, or mandate implementation of controls and procedures to guarantee availability and authenticity of stored information. Since Data Archiving stores information in a SAP proprietary binary format, archived data is automatically protected from updates.

- Reduced liability risk: In many organizations, historic data is considered as a potential liability risk as soon as the retention period defined by records management is expired. To avoid exposure to costly discovery processes, records should be destroyed as soon as possible after their retention time. Data Archiving also builds the first step for data destruction, since no other SAP approved mechanism is available to remove business records from the active database. Final data destruction is accomplished by deleting the SAP archive files from the storage system and by removing the corresponding SAP meta-data.

- Reduced online residency requirements with transparent retrieval: End-user retrieval/reporting will be provided by ‘Open Text’ Content repository. For data retrieval, ‘Open Text’ enhances standard SAP transactions to provide transparent access to archived and online data. This eliminates the need for any knowledge about where information resides (online or archived). The ‘Open Text’ data access transactions are exact copies of the corresponding SAP transactions. ‘Open Text’ Content repository are software solutions, completely implemented with SAP standard tools. To support access to archived data, a small index for each archive file is built in the file system. No additional hardware is required. In addition, ‘Open Text’ Content repository comes with tools to support the construction and removal of ‘Open Text’ index data for SAP archive files. All tools are completely implemented within SAP.

Archiving Process Flow - Accessing of Archived Data

Data Archiving Process Flow (Source: https://help.sap.com/)

1. ‘Online’ Storage: SAP Database stores any data which is created or changed and is available for frequent and real time access.


2. ‘Nearline’ Storage: During the process of data archiving as the 2^nd step of the process the data is selected from SAP online database and stored as Cache or Temporarily stored in File system (AL11) folder before relocating the data on external storage (Open Text)


3. ‘Offline’ Storage (Opentext): Provide external storage for Archived files


4. Business users can access the archived data from the Online SAP application and can use standard SAP transactions and reports to access and view archived data from ‘Open Text’

Now as SAP Consultant let’s understand some of the key details to proceed with Data Archiving

Archiving Object

The "Archiving Objects" consist of a predefined set of database tables consisting of your application data

It explains which database tables need to be grouped in order to obtain a completed business scenario that can be interpreted at the time of archiving.

The archiving objects can be categorized into following 2 types

Predefined Standard Archiving Objects:

SAP has provided set of predefined archiving objects which are further fragmented into archiving objects with functional tables and with technical tables.

Here will share with you in the upcoming blogs the details of following archiving objects on which have worked

Functional Archiving Objects:

Archiving Object	Description
FI_DOCUMNT	Financial Accounting Documents
FI_SL_DATA	FI Special Ledger: Totals and Line Items
EC_PCA_ITM	Profit Center Acctg: Actual and Plan Line Items
EC_PCA_SUM	Profit Center Accounting: Totals Records
CO_ITEM	CO Line Items
CO_TOTAL	CO Totals Records
SCMG	Case Management: Case Archiving
MM_ACCTIT	MM- Accounting interface posting data
MM_MATBEL	Materials management: Material documents
MM_EBAN	Purchase Requisitions
MM_EKKO	Purchasing Documents
MM_INVBEL	Materials management: Inventory documents
SD_VBAK	Sales Documents
SD_VBRK	Billing Documents
RV_LIKP	Deliveries

Technical Archiving Objects:

Archiving Object	Description
WORKITEM	Work Items from Workflow System
CHANGEDOCU	Change Documents
BC_SBAL	Archiving Object for Application Log
IDOC	IDoc - Intermediate Document

Custom Archiving Objects:

During the course of GAP identification during implementations or enhancements we tend to develop multiple custom transaction tables which holds lot of data and for reporting requirements we need to retain the data for certain duration. So we will have to configure custom archiving objects and capture the custom tables under these archiving objects for data to be archived.

Archiving Object	Description
ZFI_XXXX	Custom Finance Tables

Technical Table Data Deletion:

There are few technical tables which hold a lot of data wherein the historical data becomes irrelevant and shall be deleted either during housekeeping done by AM team or before we migrate to HANA or Cloud. These data cannot be archived and shall be deleted ans also SAP has also not predefined these table under any Archiving Objects.

SAP Table	Description
ARTCDATA ARFCSSTATE TRFCQOUT	QRFC Tables (queued Remote Function Call)
SOFFCONT1	Table for Document Contents (import/export)
TST03	Spool data
DOKCLU	Doc. Lines
APQD	Data Definition Queue Data

Archiving Roadmap

For better high level understanding of end to end archiving activities we need to follow the following road map (Source:Self)

Data Archiving Roadmap

Conclusion

Now am going to conclude this blog post by sharing the process flow of designing "Strategy and  Approach" for data archiving (Source: Self) to have holistic view and please follow me and my blog posts to get notified for many such upcoming articles

[former_member467152]

Hi Sanjay,

Such a clear blog providing broader picture. However, can you please explain in detail about each steps in Archiving Roadmap and who is owner of that task. Really Appreciate your input.

Thanks

[ronald_buchanan]

Really good write up on classical archiving.  Do you have anything on the ILM approach or know of any resources you could point me to?

[abprad]

Nice blog to understand archiving.

 

I have a question, We want to archive deliveries and all associated documents along with it . We have checked individual objects and all are working. Is there a process where we can run all the archiving objects at once ?

Similar for Sales order , we want to run a single program or tcode and archiving everything for the sales order, deliveries, shipment, GI , Invoices  . Is there something in standard for the same .

 

Regards,

 

Abhilash

[kollabathini]

Very nice blog, Can you please provide Step by step process for SAP BASIS activities.

[RolandKramer]

Hello

did you also has a look to this Information - SAP ILM based on SAP IQ Database

Best Regards Roland

[scott_kelson]

Hello Sanjay,

Excellent blog.  Thank you very much.

For the last slide, "Strategy and Approach", have you also done Cost Estimating on this?  I am working with a customer that would like to know how much all of this would cost them.

Thank you again.

 

[mahesh-rm]

-Very nice

[chirdeeprastogi]

Very Nice Blog