OPD Odata extraction using CDC and filter on initial load

[ozan_eroglu]

Hello!

As you might know, the closing of ODP-RFC connections for 3rd party application has forced us to look at alternatives to our data ingestion into Azure ADF and/or databricks. 

Datasphere is the alternative, but we're also considering other options, one of which is OData ODP extraction.

At this point in time, we have quite a few big tables, with over 1 billion records in the tables.

According to many documentations, OData and ODP supports $filter, however, we noticed a behavior we're not sure how to avoid.

We are using CDS views with annotations @analytics: {dataExtraction: {enabled: true,delta.changeDataCapture: {automatic: true} so utilizing the native S4HANA 2023 on-prem functionality. 

Upon initializing the delta via OData with odata:track-changes header and $filter to avoid loading 1b records that we already have in target system, a new subscription is created in ODQMON with initial delta with filter. Using the provided deltatoken for next delta request, it actually results in yet another initial delta with filter being the exact opposite of the first filter. So we end up in the first request asking for 20 records, and second request gives us the remaining 999.9999.980 mio that we try to avoid.

How would you go about delta initializing with CDC without needed to fetch the whole tables?

[ozan_eroglu]

Did you solve your problem? Did you open any incident to SAP ?