Delta Handling in #SAP #BW

[MGrob]

Introduction

Organizing data in a standard datastore object (DSO) based on a technical key enables supplying the datafields during a dataload with changed values. Not only this but also to successful track and protocol those changes. Those changes are stored with the common valid delta procedure ODS.This document tries to explain the various delta concepts.

Delta Concepts

With ODS delta concept it's possible to harmonize the data which are extracted through the various delta processes. Those can then be used with all BW datatargets afterwards. An InfoCube for example cannot process all delta concepts and is therefore relying on the delta procedure ODS.

Even Full loads can be turned into delta with this concept. The delta information holds in this case all changed records. All deleted records which are not delivered with the full load will not be reflected.

The delta mode in which data records are written into a DSO is based on the ODS delta process which hands over the InfoObject 0RECORDMODE.

If this InfoObject is not deliverd in the transformation the DSO always assumes an After-Image. 0RECORDMODE can be modified within the transformation.

Depending on the delta mode the datasource is capable of, summarizing keyfigures makes more sense than overwriting them.  The transformation contains a seperate rule group which handles 0RECORDMODE. This can also be overruled and if this is the case all aggregation modes are made available in the standard rule group. The delta mode can be overruled in either the technical rule group, the start-, end- and expert routine.

Delta process delivered by the standard DSO  Delta process delivered by the datasource

ABR
Forming deltas with after, before and reverse images updated directly in the delta queue. This show the status either, after a change or before a change (with neg. values) Adding and overwriting is permitted

ADD/ODS

The extractor delivers additive deltas that are serialized by request. This is necessary as the extractor within a request delivers every key ones. Changes on non keyfields wouldn't be copied correctly otherwise. (used by LIS)

AIMD/AIM

Forming deltas with after image, which are updated directly in the delta queue. As the packages are serialized the same key can be copied more than once in a request. This does not allow a direct update to an InfoCube. An DSO is required.

FIL0 FIL1

Delta with (After-)Images

Overwriting data fields is only possible if the used delta mode of the datasource supports before and after  images. If this is not the case datafields cannot be overwritten. DataSources with delta mode of the old LIS or the current LO extractors can only be processed by the DSO if 0RECORDMODE is handled properly in the transformation.

To be able to derive delta information the new data needs to be compared with the old data. It is of importance in what sequence the delta records arrive and be added to the DSO. For this reason the data is not immediately posted into the DSO because if requests get parallelised (within the package or multiple requests) the sequence cannot be guaranteed.

For that reason all new data is stored in the "new table" and in a second step matched to the active data by posting the new records into the "active table" in the right sequence through the "activation queue".  This is called data gets activated, meaning  data moves to active table and changelog from the activation queue. The after image of the activation queue overwrites the existing records in the "active table". Images from changelog would not overwrite the "active table".

The "change log" also contains Request ID, Datarecord and Datapackage. This allows for multiple requests and datapackage to be written in parallel into the activation queue respecting the right order of the records.

The change log table is generated automatically with each DSO together with the export datasource.

ODS process

The technical names of the delta processes have not changed even though the new DataStore Objects are called DSO. The ODS delta concept still exists.