Operational Data Provisioning (ODP) FAQ

[rudolf_hennecke]

Hello All,

Do you know … What is ODP? Or How does ODP work? Or how you can use it? What are the prerequisites? What impact will it have…

Well… working as Development Manager in the SAP BW/4HANA Data Warehousing Development, I have received many questions on that topic in the last years. Please let me therefore share some important information with all of you:

Operational Data Provisioning provides a technical infrastructure that you can use to support two different application scenarios. The first of these is Operational Analytics for decision making in operative business processes (see Operational Data Provisioning for more information).

The other very prominent application scenario is data extraction and replication: Operational Data Provisioning supports extraction and replication scenarios for various target applications and supports delta mechanisms in these scenarios. In case of a delta procedure, the data from a source (the so called ODP Provider) is automatically written to a delta queue (the Operational Delta Queue – ODQ) using an update process or passed to the delta queue using an extractor interface. The target applications (referred to as ODQ 'subscribers' or more generally “ODP Consumers”) retrieve the data from the delta queue and continue processing the data.

With SAP BW/4HANA, Operational Data Provisioning (ODP) now becomes the central infrastructure for data extraction and replication from SAP (ABAP) applications to a SAP BW/4HANA Data Warehouse. Already with SAP BW we recommend customers to use ODP for the implementation of new extraction and replication scenarios from SAP (ABAP) applications.

For the many more detailed questions on architecture, scenarios, implementation, prerequisites and availability we have compiled an FAQ document that we want to share with you now. Please find the full document here:

Operational Data Provisioning – FAQ

Feel free to comment on this FAQ list below. We are happy to update the FAQ list, as it is a living document.

[Abhishek_Hazra]

Good info Rudolf, may be you could give some insight about if an existing datasource can be converted to an ODP datasource? We are currently in analysis phase of feasibility study of moving to ODP datasources, & here I am talking with respective to HANA source system.

Best Regards,
Abhishek

[Former Member]

Hello,

Assuming we have 2 BW systems.  1 BW Classic and the other BW/4 HANA.   And we want to use the same Busines Content extractor for both but 1 is ODP and the standard?  Is this possible?

Thanks in advance.

Best Regards,

Rose

[rudolf_hennecke]

Hi Abhishek,

There is no need to convert a Service API DataSource (aka "Extractor"). It only needs to be  released for ODP (see SAP Note in FAQ Document). You can then connect that DataSource to any BW using the ODP-SAPI Source System in BW.

In case you want to convert from a running SAP Source System delta load (including delta queue) to the same delta load based on an ODP-SAPI Source System you can use the most recent version of our BW/4HANA conversion tools.

Best regards,

Rudolf

[rudolf_hennecke]

Hi Rose,

Yes, it is possible, but it will multiply the data. ODP is a new source system for BW and would add the DataSource in a new context to the system.

Please also note that there is a few Business Content DataSources (Extractors) that do not support multiple delta initialization for several (target) BW or BW/4HANA system. For more information see SAP Note 1932459 - BW DataSource: Loading of data into multiple BW target systems is not supported.

Best regards,
Rudolf

[former_member282254]

Hello Rudolf,

 

I have two small questions for you :

• If we have enhanced a standard SAP datasource with an append structure and via user exit function module or BAdI, will it still work using ODP ?


•  How does the Operational Delta Queue handle the case of deleted records for SAP standard datasources and for generic datasources ?

Thanks in advance

Kind regards

Philippe

[bryn_evans6]

 

Hi Rudolf,

I have not been able to find a document explaining master data delta using ODP. For example 0WBS_ELEMT_ATTR is currently delta enabled however when using ODP it only allows FULL loads. Is there a way to enable delta for master data?

Thanks

Bryn

 

[Former Member]

Hi,

Can you also include details about which ODP providers are delta enabled and how to find it?

Thanks

Anup

[former_member307387]

Hello,

We have an ETL tool which uses the RFC NetWeaver SDK 7.21 for data extraction from SAP System.

However we need to support the SAP BW Standard Extractors ( SAP ODP Extractors ) for data extraction from our Application.

We would like to use RFC function modules from SAP ODP API's 2.0.

Below are the function modules which can be used for data extraction from ODP Extractors.

1. RODPS_REPL_ODP_GET_SUBSCR


2. RODPS_REPL_ODP_GET_DETAIL


3. RODPS_REPL_ODP_READ_DIRECT_XML


4. RODPS_REPL_ODP_OPEN


5. RODPS_REPL_ODP_FETCH


6. RODPS_REPL_ODP_CLOSE

We might need to use some more functions as well.

From the SAP Note 1931427 - ODP Data Replication API 2.0. Those RFC function modules can be used for RFC Client applications.

 

However from one of the SAP ODP FAQ documents says that SAP ODP API's is restricted to SAP Applications and not used by 3rd party party ETL Tools.

Can we use those RFC Function modules for RFC Client application ( ETL Tool) ? If this is not way the correct way then which approach we should use for data extraction from SAP ODP Extractors ?

 

Thank you in advance.

 

Regards,

Sachin

 

 

 

[rudolf_hennecke]

Hi Bryn,

The delta procedure that a Business Content Extractor supports is defined in its metadata in the Service API Framework (table ROOSOURCE-DELTA). This delta method should be available independently of whether the Business Content Extractor is extracted via an SAP Source System or via an ODP-SAPI Source System. Do you have a reproducible example? What is the source system release? What is the (BW) target system?

Best regards,
Rudolf

[rudolf_hennecke]

Hi Anup,

Upon replication of DataSources for a given ODP Provider Context to a target system (e.g. BW or BW/4HANA) you will see the delta capability per individual DataSource.

In general the following applies per ODP Provider context:

• ODP-SAPI: The delta method (if available) is defined in the extractor metadata. See standard documentation or table ROOSOURCE-DELTA in the source system


• ODP-CDS: The respective DataSource is delta enabled in case the underlying ABAP CDS View is annotated for delta extraction (see the How to Guide referenced in the ODP FAQ document)


• ODP-SLT: Is typically per default delta enabled as SLT is about realtime (delta) replication to the Operational Delta Queue (ODQ)

Even in case where a DataSource is only full enabled you can define a "Generic Delta" procedure in the DataSource Maintenance on BW or BW/4HANA side in case the DataSource contains a field such as date, timestamp or counter that can be used for delta derivation.

Best regards,

Rudolf

[rudolf_hennecke]

Hi Sachin,

The document is correct: the ODP API is an SAP internal interface that is only available for SAP Products such as SAP DataServices or SAP HANA Smart Data Integraton. The ODP API can't be used by 3rd party ETL tools. Overall, there is no public interface for 3rd party ETL tools to directly connect to Business Content Extractors.

Best regards,

Rudolf

[Jan_van_Ansem]

Hello Rudolf,

I notice you mention an ODP API for SAP Data Services and for HANA Smart Data Integration - but I can't see these interfaces. There is a 'ODP - other Contexts' node in RSA1. Should I go down that route to connect SDI / Data Services?
I can't find any documentation on connecting either SDI or Data Services directly to BW/4HANA so if you have any information on this that would be very welcome.

My current understanding is that neither product can load into BW/4HANA directly - you would have to load into HANA tables first and then integrate the data in the BW/4HANA context. Obviously this has license implications....

Thanks in advance for your help on this.

Jan.

[former_member307387]

Dear Rudolf,

Thank you very much for the information that you have provided.

As per the update we cannot use the RFC Function modules to extract the data from SAP ODP Extractors.

Is there other way to extract the data from the SAP ODP extractors to other external applications. I saw using OData we can extract the data of ODP extractor to other non ABAP Applications

https://help.sap.com/viewer/ccc9cdbdc6cd4eceaf1e5485b1bf8f4b/7.5.9/en-US/11853413cf124dde91925284133...

Please can you suggest which will be the best way to extract the data from SAP ODP Extractors ?

 

Thanks again for your help.

 

Regards,

Sachin

 

[ClausEriksen]

Hi Rudolf,

Is there any way to see any new delta records (not yet extracted to the subscriber) in ODQMON as we can in RSA7? Or just a way to know if new records exist? In ODQMON I can only find the delta records that was already extracted.

If ODQMON was only designed to monitor the Delta Queue Requests, How then do you monitor the actual delta queue?

Example:

LO Cockpit DataSouce 2LIS_05_QE1 in update mode Queue Delta. On every run of the update job, new delta records get collected to the delta queue. In RSA7 we can see these new records. In addition the column Total (LUWs) gives us an indication of whether new records are sitting in the queue. Two consecutive delta extractions will reset the counter.

Thanks,

Claus

[rudolf_hennecke]

Hi Jan,

Yes, you are right. The ODP API that I mention (and that is only available for SAP DataServices and SAP HANA Smart Data Integration (ABAP Adapter)) extracts from ODP (Providers) such as S-API DataSources (Extractors), BW InfoProviders or ABAP CDS Views.

It is not meant to load data into BW/4HANA. Your current understanding from above is also correct. At this time, SAP Data Services (or other tools) can write to a HANA Table from which BW/4HANA can retrieve the data again. The former BAPI Source System in SAP BW is not available in SAP BW/4HANA. We are planning to offer a direct load into BW/4HANA with our next major release in 2019 (current planning - subject to change).

Best regards,

Rudolf

[rudolf_hennecke]

Dear Sachin,

Yes, there is ODATA support for retrieving data from ODP Providers such S-API DataSources (Extractors).

But as said above, there is however no public (ABAP) interface for 3rd party ETL tools to directly connect to Business Content Extractors. There has never been an interface in the former Service API Context (-> related to the SAP Source System in SAP BW). By interfacing the Service API Extractors with ODP we offer now only that internal ODP API that SAP Data Services and HANA Smart Data integration can use.

So in order to read from Business Content Extractors either SAP BW, SAP BW/4HANA, SAP DataServices or SAP HANA Smart Data Integration has to be used.

Best regards,

Rudolf

 

 

[rudolf_hennecke]

Hi Claus,

Yes, you are able to see this in ODQMON. Using Provider = BW Data Source, Subscriber Type = SAP Business Warehouse (in case your target is SAP BW what I assume) you will get a list of all your queue.

Then, please click on the queue you are interested in and you will get a list of subscribers (in case of a target SAP BW this is all the DTPs that load in active delta mode from this queue).

Clicking on one of the subscribers you will then get a list of requests. In this list you can see whether a request has been extracted successfully to SAP BW (status: confirmed) or whether the request has not been transferred (confirmed) yet.

Best regards,

Rudolf

[former_member222726]

Hi Rudolph, thanks for the given information.
So does it means that ODP could not be used as a source of data for an Oracle database for example?
Which should be the best approach to send data from BW4 to Oracle in a daily basis?

Thanks in advance,

Luis Loebel

[ClausEriksen]

Hi Rudolf

Thanks for explaining. I get the idea now, but on our system it do not behave this way. I only see the confirmed requests, never requests that are "unconfirmed". When I run delta update from BW, a new confirmed request is added to the existing list of confirmed requests. This do not allow me to see if data is sitting in the queue.

Before delta update:



After delta update:



 

[amigo]

Hi Rudolf. Thanks a lot for sharing this info on ODP extraction. I have a specific question related to custom a extractor. I have developed a custom extractor which is feeding up a RSA7 delta queue automatically (through BTE or any in other update task on ERP) via a FM RSC1_TRFC_QUEUE_WRITE. Would this still work once we decide to release it for ODP? In other words, is there a way we can trigger ODP queue for custom extractor within an ERP update task e.g. at Purchae Order creation/change or goods receipt as I am doing it now? If there is a way of doing it, could you please share some details?

Thanks a lot and greetings,

Peter -

[anilreddyk]

Hi Rudolf,

I have a Question regarding "Exchanging Data Between BW Systems Using the ODP Source System".

I am trying to use BW to BW ODP replication Scenario. And in this scenario, I am facing some issues and not getting enough information on how can resolve them.

I have 3 BW systems in total (first one is BW 7.5 SPS 4 - let's call this BW1 hereafter,
second one is BW 7.31 SPS 12 - let's call this BW2 hereafter and
third one is BW 7.5 SPS 05 - let's call this BW3 hereafter), in which I want to extract data from BW1 & BW2  systems to 3rd BW3 System.

In this scenario I have connected both BW1 & BW2 to BW3 as ODP BW Source Systems.

Here in first case from BW1 to BW3, all the Info Providers in BW1 are 7.3 Objects. Here I am unable to see few Hierarchy info providers as data sources in BW3 and

in second case from BW2 to BW3, all the Info Providers in BW2 are 7.3 Objects. Here I am able to see only very few info providers as data sources in BW3.

Could you please suggest on what are the steps/settings I should follow in order to see all the Info Providers as Data Sources from BW1 & BW2 in BW3.

Thanks in advance and greetings.
Anil

[shasan]

Hey cpt.itelligence 

You can check Units tab, next to Requests tab whenever you clear your LBWQ and data sits in your delta queue.

[ClausEriksen]

Thanks a lot Shadab, I get it now. If you double-click a request you only see the units for that request, but if you just click Units, you see all units - incl. the new ones.

[shasan]

True that.

 

Cheers,

Shadab

 

[samnanda]

Hi Rudolf,

 

We have extended a CDS view xxx which has a underlying ODP with custom view yyy. However, the field added in yyy is not reflecting in the ODP of xxx. Result of which we are getting the error "Could not resolve field zzz (ODP xxx)" in tcode RSRT->Generate Query->Debug mode.

Please note that we have provided the analytical query details in RSRT which is built on top of CDS view xxx.

Can you please help us out as it is a last-minute requirement.

Regards,

Samay.

[former_member596579]

Hello Rudolf,

I activated the extractor 0PLANT_ATTR and 0PLANT TEXT, only the extractor 0PLANT TEXT works with ODP, is it normal?
I do not see in BW the extractor 0PLANT_ATTR when I replicate the DATASOURCE

best regard

[AndreasTenholte]

Hi Farchid,

I believe the step you have missed is to follow SAP Note

2232584 - Release of SAP extractors for ODP replication (ODP-SAPI)

Make sure that the latest version of this SAP Note is implemented in your source systems (e.g. SAP ERP, SAP S/4HANA, SAP CRM, ...) and you execute the report BS_ANLY_DS_RELEASE_ODP in the source system.

Best regards,

Andreas

[former_member634125]

Hello all. on a different topic do you know if PI can utilize the ODP Frame work in BW?

Many thanks for your Help

 

Tomas

[rphss]

Hi Rudolf,

 

I am looking for some technical background on which table is covering what for ODP S/4 to BW.

Specifally I want to understand how a new init can be enforced.

as far as I understand as long as we don't have any data / requests from ODP data source in first ADSO layer the dtp collects first the setup tables then the queues?

after that and having at least one request in first adso only queue is collected.

Now i want to delete teh queue data , but even if I am cleaning up by using 0 days it allows me only to clen up requests send to BW. So how to clean completely without request check?

Also if I wnat to clean the queues is there a chance to select the client as well. As we had to enhance the extract structure and it will not work if all clients have been cleansed. So going into each client is time consuming.

So summing up , here the short questions:

1. technical doicumention tables /functions etc.


2. clean up all queus independent on forwarded requests


3. enforce / reset init

If someone has any information would be very appreciated.

Ralftechnical

[sushil_290459]

Hello Andreas,

We are facing some ODP Related issue in ECC 6.0 Quality System.

ECC on Oracle and BW 7.5 on Oracle.

We wanted to make use of ODP functionality, so we enabled some data source in D System and Replicated same in BW-Dev system accordingly.

We moved the Active Data source from Dev to Quality System of ECC, our TR moved successfully but Data Source not ODP enabled.

I have gone through Note : 2232584 - Release of SAP extractors for ODP replication (ODP SAPI) - Here they are insistng run the Prograrn : BS_ANLY_DS_RELEASE_ODP - to make ODP enable.

If we run this Program, could you please let us know, what will be impact on Existing Data source which are running as Standard Extractor to BW System.

if we have to run this program means, in Production also we have to execute the same or how it is.

[former_member685361]

Hello Rudolf,

we came across one scenario..

ABAP CDS view data restriction through SAP BW4 Hana query Variable.. Possible..?

My Data Flow:

S4 Hana System(CDS VIew)--> Can be parameterized or Non Parameterized

BW4 Hana System--> Open ODS VIew(Based on CDS view)--> Composite provider--> BW Query with variable for field company code. So this is 100% virtual hybrid modelling scenario.

now, user inserts company code 'IN10'(In Pop Up) while executing query and he wants to execute/Process CDS view only for 'IN10' at the back end. although there are many company code data available in the S4 system.

can you please suggest how to implement this.

[former_member206066]

Hello Rudolf,

 

We have 2 system one is in SAP BW 7.3 and another is in BW/4 HANA.

If I want to upgrade a flow(Delta load for LO extractor) from 7.3 to BW/4 HANA, what are the pre Upgrade activities I should follow and how can I upgrade the delta LO extractor

 

Thanks,

Ipsita

[aditya_palekar]

Hello Rudolf,

I am doing an advisory role for a customer where the implementation team is working on a BW/4HANA 2.0 implementation. They have deactivated the ODP BAdI and used the classic BADI RSU5_SAPI_BADI for data extraction from S/4HANA 1909.

From a long term viewpoint are there any consequences if the customer continues to use the classic BAdI purely for S/4HANA data extraction?

1. Is SAP planning to phase out classic BAdIs in later S/4HANA releases?


2. Do you recommend that moving to ODP BAdI sooner than later is better from a future viewpoint even if its for ERP to BW data extraction? I see that performance wise it is better and I am concerned that after go-live if someone activates an ODP BAdI with same filters as the classic BAdI scenario then the classic BAdI will suddenly stop working. So one needs to be more aware when both the ODP and Classic BAdIs are jointly used.

Please let me know your comments.

Thank you,

Best Regards,

Aditya

[rudolf_hennecke]

Hi Aditya,

To your questions:

1. No, there is no plan at this time to phase out the classic BAdIs (subject to change).


2. Your concern about introducing ODP BAdI at a later time is valid as you can see from SAP Note 2684463. Also, the ODP BAdI additionally works for direct access scenarios whereas the classic BAdIs are only used in extraction cases. Besides these considerations, customers are however not forced to move away from the classic Exits or BAdIs.

Hope that helps and best regards,

Rudolf

[former_member112966]

​I have accidentally exposed 1 LO datasource as ODP replication instead of API standard replication because of which the delta is not working as originally the datasource was done using API replication technique.

Is there any way to replicate it in a normal way again, as the delta is not working after exposing it as ODP datasource in a standard SAP BW SQL database. How to replicate it back or unexpose the datasource as a standard datasource again. Like unexpose the ODP replication. Please advise.

[GordonW]

Hi Manu,

first of all the question is why is the LO DS you are using not delta enabled? Is the LO Datasource you are using mentioned in SAP Note https://launchpad.support.sap.com/#/notes/2232584/E?

Secondly, you can delete the ODP DataSource in your BW system and replicate the Datasource via SAPI again. 

 

BR
Gordon.

[former_member729347]

Hi Rudolf,

I have the exact scenario

The source system is  ECC6( EHP1 Netweaver 7.0)  SP17.

We have 2 target systems i.e. BW/4 HANA and BW 7.0. BW 7.0 system is up and running since few years and BW/4 HANA system will be new.

Suppose both the system wants to exact data from from one data source say 2lis_11_vaitm, will it work as BW/4 HANA only supports ODP data source.

 

By applying the note 2232584 I can convert the data source 2lis_11_vaitm to ODP enabled. By applying this note will it affect the delta queue of the BW 7.0 data flow ??

 

I am confused how the delta will behave as one support RSA7( delta queue ) and other supports ODQMON.

 

Please let me know.

Regards,

T

 

 

[GordonW]

Hi T,

the data source is part of the source system - in this example your source system is  ECC6( EHP1 Netweaver 7.0)  SP17 .

SAP BW/4HANA can only handle ODP enabled DataSources - SAP Note https://launchpad.support.sap.com/#/notes/2232584/E explains which DataSources SAP has released for ODP replication. This is for the SAP BW/4HANA data loads.

For the load into BW 7.0. you can still use the classic SAPI interface. There is no need to change this interface for feeding the data from your source system ECC6 -> BW 7.0 but you can setup a additional data load to your SAP BW/4HANA system based on ODP.

Hope this helps.

BR
Gordon.

[former_member720151]

Hi All,

We have one Generic data-source in S/4 1909 which is based on ODP along with few custom fields written Badi- BADI_RODPS_DATASOURCE_EXT. (As per logic)

We have created Badi as per steps given by SAP in below link but still it is not calling data-source given Badi.

https://help.sap.com/viewer/6522d0462aeb4909a79c3462b090ec51/1709%20000/en-US/8a4121d239094ffebac228...

Do we need to do anything else? We have disable code in CMOD as well and given data-source filter in badi as well.

 

[Roman_Broich]

Hi all,

the ODP FAQ states: "The ODP data replication API is restricted to SAP applications and not open to 3rdparty ETL tools."

While SAP BW/4 documentation explicitly refers ODP & OData extraction for "External Applications" and does not mention the restriction to SAP applications as consumers:
ODP-Based Data Extraction via OData - SAP Help Portal

Is BW/4 and ODP & OData based data distribution an exceptional case which supports non-sap consumers?  Does this ruling as well apply to S/4 and ODP & OData based extraction?

Thanks!
Roman

[GordonW]

Hi Roman,

it is possible to use the OData Interface externally for ETL Tools if you develop a OData consumer. This is independent from underlying DOP technology.

BR

Gordon.

[former_member215470]

Hi Gordon,

Greetings!

Thanks for your answer. It clears the doubt which the user had.

However, I just have one follow up query on this.

You have said that the data can be loaded to both the BW/4HANA (using the ODP framework) and the 7.0 (using the SAP interface), from the same ECC6 system, so I am curious to ask - if running the two frameworks - SAPI and ODP in parallel in the same source system would have any drawback there? This means we will be running the Delta Queue (RSA7) as well as the ODQMON there. So basically, this would mean that the same data is being collected in two separate queues for the two frameworks? Am I correct in saying this? And if yes, what repercussions/effects would this be having on the source system?

Many thanks in advance for your answer,
Sayan Dey

[jadhvaryu1065]

Hi Gordon, Rudolf,

 

We have a peculiar use case.

At initial full load & delta loads from ECC6, we need to do complex data aggregation with other non-SAP systems and then load it to SAP BW.

How can i get ODP to send data out to non-SAP staging area first? We are using JCO functions with RODPS_REPL_ODP_PREFETCH & RODPS_REPL_ODP_FETCH_XML. For small data packages, it works fine. But for larger data sets (>10M records), we are getting duplicate data.

We know ways to load to SAP BW.

Any guidance will help.

Thanks in advance,
IA

[GordonW]

Hi Jaideep,

for this kind of data manipulation SAP is positioning SAP Data Intelligence to send data from SAO system to 3rd party environments.

BR
Gordon.

[jvennervald]

Hi Philippe

 

Did you ever get an answer to your question about how deleted records are handled in ODQs?

 

Kind regards,

Jacob

[M_Abdelaziz]

Same situation here with this BADI (BADI_RODPS_DATASOURCE_EXT) no other information other than the link provided  ! did you be able to find why it's not getting called or triggered in RSA3  ?

[Jean_pierre_06]

Hello

The link (faq) provided at the end of the blog post doesn't seem to work anymore.

Thx. BR.

[rudolf_hennecke]

Hi Jean-Pierre,

Thanks for raising this issue with the link to the ODP FAQ. This is fixed in the meantime and the ODP FAQ is avaiable again.

Best regards,

Rudolf

[sachinjakkaria]

Hi,

Link to Introduction to Operational Data Provisioning  is not working. Kindly update the same.

[Gajanan_S]

,