Full vs. Delta Load

[Former Member]

Hi BI-Experts!

This is a rather general question.

Is there a recommendation when to use Full or Delta Load?

I know the functional differences but do SAP or some technical reasons say that each of them may only be used in special cases?

For example Delta Load can only be used with SAP source systems?

Best regards,

Philipp

[Former Member · Accepted solution]

Hi Philipp,

This is a pretty open-ended question. I will give you a few examples of when I use each ....

Delta's:

1.) If the record set is large and changes a lot daily. If you have a large initial load, let's say 2Million records and you get about 200k records per month, then you want to use a delta mechanism. There are just too many records to reload daily.

2.) If the record set is wide (many fields instead of a few), it will take less load time to run a delta than a full load.

3.) Datasource restrictions....some business content datasources don't have full loads and vice versa (some don't have deltas).

4.) Do my loads take too long? If so, I will convert full loads to delta's to shave off time here and there.

Full

1.) I gauge this mainly based on load time. If it takes 5 minutes to load 100k records, then it's going to take longer to figure out what delta's were missed (if there was a problem) than it would to just reload the dataset.

2.) Do my requirements dictate that I need to drop and reload the dataset? The best example of this would be the current year's forecast. Some companies revise the forecast and don't want to keep any previous revisions.

3.) Some data loads load a snapshot of a specific date. So, if I wanted to take a snapshot of inventory in a DSO/ODS on Monday, I would Datamart it into a Cube doing a full load for Monday's date.

Hope it helps!

Thanks,

Brian

[Former Member]

Hi Landmann,

I just need some more information on this. What will happen if we do load twice?. I mean for the first time we do load and later on we do delta load for get the updated and changed data. if, we do full load one again instead of delta load what will happen in target system. In DSO and/ or Cube. Whether it ll replace old data or add with old data?

[RamanKorrapati]

if your understand about full and delta concepts you may not be raised your doubt.

Full load - is one time activity.

Delta load - always loads newly create or changed records from the last update.

if you load full load twice to info cube, it leads duplication. in case  of DSO it overwrites if your key fig property was over write other wise key fig values will be doubled. leads in consistency.

[Former Member]

Hi Philipp,

There are lots of parameters based on which you decide whether to go for a full/delta load:

1. Type of datasource - If the datasource supports delta updates then preference is delta load as it consumes less time and system resources.

2. Reporting Needs

3. Type of source system - For flat files it is typically always full load.

4. Application area - There are certain application areas which have reports based on time-stamp. Example: At this point of time how does the data look. In such cases a full load is always preferred.

5. Type of data target updated

6. Frequency of data load - More frequent loads typically are delta enabled whereas loads which are done once in a month are mostly full loads.

Bye

Dinesh

[SreekanthSurampally]

Hi Philip,

Full Load brings all the records exist in The data source. in The business When you are loading a Data target every day. You get huge data in the cube so you need to delete the previous data every data to load the full load. It degrades the performance.

So you prefer Delta which gives you only changed records since the last update. you can improve the performance.

Some times we may the requirement to load only full loads to the target and delete the previous data if the datasource doesn't support delta.

[former_member453731]

Type of data load update is dependent on the datasource.Delta is always preferred as it will only bring the newly added or changed records.But if your datasource does not support delta, then your options are limited. you have to use full load.