importing methods in models - suggest correct method

[KeerthiG16]

Hi,

I have created a data model, the data is imported using Odata API, I have scheduled the import using the import method as an update. but it is creating duplicates in the data, I don't understand why?

There are this four methods available in import method

1. Update

2. Append

3. clean and replace selected version data

4. clean and replace a subset of data

The import happens once every day, I want the newly added rows in the database to be imported.

I have another data set which is having 2 million rows, I am afraid of which import method to use for this data set. As we know the maximum import limit is 800,000. please suggest an import method that doesn't create duplicates and import only deltas.

[N1kh1l · Accepted solution]

keerthi117

A very detailed explanation of various import methods has been explained below. Which method to use depends on what target import behavior you intent to achieve. Also you can filter the records for initial load behavior check before doing the complete load. I don't think import method

I am not sure if your data source has incremental load facility. If it has you can configure that along with import method "Append". Refer to below for details. Import method only decided the load behavior but it does not have any say on whether the data source will return delta records only.

https://help.sap.com/docs/SAP_ANALYTICS_CLOUD/00f68c2e08b941f081002fd3691d86a7/a8435da6970041d2beb32...

Import Methods:

• Update: The target model’s measure values for the dimension member combinations specified by the source data are updated by the corresponding measure values in the source data. If the particular dimension member combination has no measure values in the model prior the import, new value is inserted.
• Append: The target model’s measure values for the dimension member combinations specified by the source data are added by the corresponding measure values in the source data (summed together). If the particular dimension member combination has no measure values in the model prior the import, new value is inserted. For a more refined scope, use either the Clean and replace selected version data or Clean and replace subset of data update options.
• Clean and replace selected version data: Deletes the existing data and adds new entries to the target model, only for the versions that you specify in the import. You can choose to use either the existing version or specify a new version under Version. If you specify to import data for the "actual" version, only the data in the "actual" version is cleaned and replaced. Other versions, for example "planning", are not affected.
• Clean and replace subset of data: Replaces existing data and adds new entries to the target model for a defined subset of the data based on a scope of selected versions using either the Existing Version or New Version buttons. You can also limit the scope to specific dimensions. To define a scope based on a combination of dimensions select + Add Scope and use the Select a dimension field to specify a dimension.

  When a Date dimension is defined in the scope, the time range in the source data (determined by the minimum time value and maximum time value of the source data, as well as the granularity of the date dimension) combined with other dimensions in the scope, will determine what existing data is cleaned and replaced.

  If for example, Date and Region dimensions are defined as part of a scope, only entries that fall within the time range and match Region from the source data will be replaced in the target model. Existing data that does not match the scope will be kept as is. Other dimensions that are not part of the scope will be cleaned and replaced regardless of whether the dimension members are in the source data or not.

https://help.sap.com/docs/SAP_ANALYTICS_CLOUD/00f68c2e08b941f081002fd3691d86a7/cf82b68af0644fe799fb4...

Example to understand:

Blog with detail examples:

https://blogs.sap.com/2020/07/09/data-import-methods-in-import-data-connections-sap-analytics-cloud/

Hope this helps !!

Nikhil

[UmaArjunan · Accepted solution]

Hi Nikhil ,

Clean and replace subset of data: for this option, if NO dimensions are mentioned in the scope, it takes version as first dimension to clear the data irrespective of entity. Does it check the authorization of user who has access to specific entities to clear the data from the system ? or it just cleans the data without checking the authorization. Could you please explain on this ?

Regards,

Uma

[former_member805109]

I have had a similar issue. See this ticket below, we resolved it using the KEY in the CDS view.

https://userapps.support.sap.com/sap/support/knowledge/en/2816755