Technology Blogs by SAP
Learn how to extend and personalize SAP applications. Follow the SAP technology blog for insights into SAP BTP, ABAP, SAP Analytics Cloud, SAP HANA, and more.
cancel
Showing results for 
Search instead for 
Did you mean: 
andrefischer
Product and Topic Expert
Product and Topic Expert

Updates



  • 04.03.2014     --> Added a link to document that describes how to consume a service that desribes the OData Delta Query mechanism

  • 20.06.2023     --> Added a link to the SAP Gateway OData V4 framework documentation.


Objectives







Hint:
The delta query is supported with the OData V4 framework (but only in on premise or private cloud systems)











In this how-to guide, you will learn how to implement Delta Token capabilities with SAP NetWeaver Gateway to enable offline functionality. This how-to guide assumes that you have already worked your way through the following how-to guides:



  1. “How to Develop a Gateway Service using Code based Implementation”

  2. "How to Develop Query Options for an OData Service Using Code-Based Implementation"


 
These guides describe the basic implementation steps of the service that we are now going to enhance. The sample service uses demo data from the Enterprise Procurement Model, which is part of every SAP NetWeaver ABAP server as of 7.02.


After completing these steps, you will be able to:

• Enhance the GET_ENTITYSET and GET_ENTITYSET_DELTA methods of an entity set to support the delta query functionality.



Business Example



You want to build an application that shows a list of products. The application shall be able to retrieve only those products that have been newly created, changed or deleted after having retrieved a complete list of products as an initial load.


 

Short Introduction to Delta Token Functionality



OData exposes collections of records as EntitySets. These EntitySets support various options like sorting, filtering or paging to minimize the amount of data that is sent accross the wire.


Paging is a very powerful option to reduce the size of the payload and improve performance. To use paging on client side, the OData request needs to specify the $skip and $top tokens. For server-driven paging you have to use the $skiptoken.


Some applications, however, prefer to use client side caching rather than paging. In such cases, SAP NetWeaver Gateway provides Delta Tokens that help to reduce resource consumption on the server and the wire when the same request is sent several times from the same client. The OData Delta Token enables clients to use caching when retrieving entity sets. Consequently, the client gets only the changed records after an initial retrieval of all records. Therefore, the initial request will return all records and a Delta Token. The subsequent request (with exactly the same query options) sends the Delta Token from the first request. The response will then contain only the changed records. If no records have been changed since the first request then the response will not contain any records at all.



You can implement Delta Token in several ways and these options can be grouped into two main approaches:




  1. The first approach, which is based on Syclo Exchange framework, calculates deltas at modification time: The ABAP system tracks relevant changes when they occur. At request time, the deltas are already prepared and thus available. On the one hand this approach requires more development. On the other hand it’s more scalable and has an optimized overall performance.

  2. The second approach is based on delta determination at request time where the system compares old and new state to find out which records have been changed/deleted. The implementation effort is rather small but it does not optimize the performance of the backend. That means, the more records you have in the full collection, the longer the response time of the request.



With both approaches, the payload of the response is reduced but as described only the first approach is also able to optimize the performance in the backend.


In this guide we will look at the basic implementation, using the Delta Request Log Component. The Delta Request Log Component is based on the second approach. Therefore the implementation effort is rather small but the performance of the backend is not optimized.



Prerequisites



You have implemented an OData service as described in the two how-to guides mentioned above.

If you are using NetWeaver 7.02, then you need at least software component IW_BEP Support Package 7 and software component IW_FND Support Package 7.

If you are on NetWeaver 7.40, then you need at least software component SAP_GWFND Support Package 4.

You have to implement following SAP notes: 1926946, 1932247 and 1970229.


Implementation Overview



In this how-to guide, you will create a new Entity Set Products_DQ that shall support delta queries in the existing sample service that you have created earlier. You will redefine the methods GET_ENTITYSET and GET_ENTITYSET_DELTA to support the delta query functionality by calling method /IWBEP/CL_QUERY_RESULT_LOG-> CREATE_UPDATE_LOG_ENTRY_HASH.


Task 1: Create an additional Entity Set Products_DQ



We will enhance the service you created earlier to support delta queries for product data. For this we create a new entity set called Products_DQ where we implement the delta query specific coding while leaving the original Products Entity Set unchanged.


Task 2: Enhance method ZCL_GW_PRODUCT_DPC_EXT->PRODUCTS_DQ_GET_ENTITYSET to support the delta query functionality



In this task you will enhance the existing GET_ENTITYSET method of the newly created entity set Products_DQ. Like the existing GET_ENTITYSET method of the Products entity set it should retrieve the application data containing product entries.


First you need to create a local data provider facade which will be needed later by the Delta Token functionality.


In the next step you will need to call method /IWBEP/CL_QUERY_RESULT_LOG-> CREATE_UPDATE_LOG_ENTRY_HASH for the delta query functionality by passing the local table containing the product entries and the local data provider facade. The method is the central method for the Delta Token functionality. It calculates a hash value for each product and stores the hash values in some application tables, so the entries can be later checked for changes. The method will also return a delta token which you need to export.


Step 1: Get the data provider facade (you need to pass the facade later to the Delta Token method)



* get the data provider facade
TRY.
lo_dp_facade
= /iwbep/if_mgw_conv_srv_runtime~get_dp_facade( ).
CATCH /iwbep/cx_mgw_tech_exception.
RAISE EXCEPTION TYPE /iwbep/cx_mgw_tech_exception.
ENDTRY.





Step 2: Call method /IWBEP/CL_QUERY_RESULT_LOG-> CREATE_UPDATE_LOG_ENTRY_HASH for the delta query functionality by passing local table et_entityset containing the product entries



* call the delta token functionality

  TRY.
CALL METHOD /iwbep/cl_query_result_log=>create_update_log_entry_hash
EXPORTING
io_tech_request_context 
= io_tech_request_context
io_dp_facade            
= lo_dp_facade
ir_service_document_name
= mr_service_document_name
ir_service_version      
= mr_service_version
it_entityset            
= et_entityset
CHANGING
ev_delta_token          
= lv_delta_token.
CATCH /iwbep/cx_qrl_locked.
RAISE EXCEPTION TYPE /iwbep/cx_qrl_locked.
CATCH /iwbep/cx_qrl_delta_unavailabl.
RAISE EXCEPTION TYPE /iwbep/cx_qrl_delta_unavailabl.
ENDTRY.





Step 3: Export the delta token which was calculated by the method in step (2)




es_response_context-deltatoken = lv_delta_token.



Task 2: Re-define method ZCL_GW_PRODUCT_DPC_EXT-> /IWBEP/IF_MGW_APPL_SRV_RUNTIME~GET_ENTITYSET_DELTA to support the delta query functionality



In this task you will re-define method GET_ENTITYSET_DELTA. First, you need to retrieve the application data containing product entries, just like in GET_ENTITYSET method.

Then you need to create a local data provider facade which will be needed later by the Delta Token functionality.

In the next step you will need to call method /IWBEP/CL_QUERY_RESULT_LOG-> CREATE_UPDATE_LOG_ENTRY_HASH for the delta query functionality by passing the local table containing the product entries and the local data provider facade.

It calculates a hash value for each product. Those hash values are then compared with the hash values of the prior call (related to the same delta token ID) to determine which values have changed in the meantime. The method will return a new delta token which you need to export. The method will also return any entries that were changed in the meantime (please store these in local table lt_product) and any entries that ware deleted (please store these in local table lt_deleted_product).



Step 1: Re-define method /IWBEP/IF_MGW_APPL_SRV_RUNTIME~GET_ENTITYSET_DELTA



Step 2: Retrieve all relevant entries for this entity set (like in GET_ENTITYSET above)



Step 3: Get the data provider facade (you need to pass the facade later to the Delta Token method)




* get the data provider facade
TRY.
lo_dp_facade
= /iwbep/if_mgw_conv_srv_runtime~get_dp_facade( ).
CATCH /iwbep/cx_mgw_tech_exception.
RAISE EXCEPTION TYPE /iwbep/cx_mgw_tech_exception.
ENDTRY.





Step 4: Call method /IWBEP/CL_QUERY_RESULT_LOG-> CREATE_UPDATE_LOG_ENTRY_HASH



Call method /IWBEP/CL_QUERY_RESULT_LOG-> CREATE_UPDATE_LOG_ENTRY_HASH for the delta query functionality by passing local table et_entityset containing the product entries – just like in task 1, step 2 but this time take over the changed objects (and store them in local table lt_product) as well as the deleted objects (and store them in local table lt_deleted_product):



call the delta token functionality

TRY.
CALL METHOD /iwbep/cl_query_result_log=>create_update_log_entry_hash
EXPORTING
io_tech_request_context 
= io_tech_request_context
io_dp_facade            
= lo_dp_facade
ir_service_document_name
= mr_service_document_name
ir_service_version      
= mr_service_version
it_entityset            
= lt_product
IMPORTING
et_deleted_entityset    
= lt_deleted_product
et_entityset            
= lt_product
CHANGING
ev_delta_token          
= lv_delta_token.
CATCH /iwbep/cx_qrl_locked.
RAISE EXCEPTION TYPE /iwbep/cx_qrl_locked.
CATCH /iwbep/cx_qrl_delta_unavailabl.
RAISE EXCEPTION TYPE /iwbep/cx_qrl_delta_unavailabl.

ENDTRY.



Step 5: Export the delta token which was calculated by the method in step (4)


es_response_context-deltatoken = lv_delta_token.



Step 6:  Export the deleted entity set



Export the deleted entity set - add deleted records (from table lt_deleted_product) to the deleted entries list er_deleted_entityset


*   export the deleted entity set
copy_data_to_ref
( EXPORTING
is_data
= lt_deleted_product
CHANGING
cr_data
= er_deleted_entityset ).



Step 7: Export the changed entity set - add changed records from lt_product to er_entityset



*   export the changed entity set
copy_data_to_ref
( EXPORTING
is_data
= lt_product
CHANGING
cr_data
= er_entityset ).



Implementation



ZCL_GW_PRODUCT_DPC_EXT->PRODUCTS_DQ_GET_ENTITYSET



The coding of the method:


DATA:
lt_return         
TYPE TABLE OF bapiret2,
lv_delta_token    
TYPE string,
lo_dp_facade      
TYPE REF TO /iwbep/if_mgw_dp_facade.



 
* retrieve all relevant entries for this entityset
CALL FUNCTION 'BAPI_EPM_PRODUCT_GET_LIST'
TABLES
headerdata
= et_entityset
return     = lt_return.

* get the data provider facade
TRY.
lo_dp_facade
= /iwbep/if_mgw_conv_srv_runtime~get_dp_facade( ).
CATCH /iwbep/cx_mgw_tech_exception.
RAISE EXCEPTION TYPE /iwbep/cx_mgw_tech_exception.
ENDTRY.

* call the delta token functionality
TRY.
CALL METHOD /iwbep/cl_query_result_log=>create_update_log_entry_hash
EXPORTING
io_tech_request_context 
= io_tech_request_context
io_dp_facade            
= lo_dp_facade
ir_service_document_name
= mr_service_document_name
ir_service_version      
= mr_service_version
it_entityset            
= et_entityset
CHANGING
ev_delta_token          
= lv_delta_token.
CATCH /iwbep/cx_qrl_locked.
RAISE EXCEPTION TYPE /iwbep/cx_qrl_locked.
CATCH /iwbep/cx_qrl_delta_unavailabl.
RAISE EXCEPTION TYPE /iwbep/cx_qrl_delta_unavailabl.
ENDTRY.


* export the delta token
es_response_context
-deltatoken = lv_delta_token.






ZCL_GW_PRODUCT_DPC_EXT-> /IWBEP/IF_MGW_APPL_SRV_RUNTIME~GET_ENTITYSET_DELTA



The coding of the method:


DATA:
lt_return            
TYPE TABLE OF bapiret2,
lv_entity_set_name   
TYPE string,
lt_product           
TYPE zcl_gw_product_mpc=>tt_product,
ls_product           
TYPE zcl_gw_product_mpc=>ts_product,
lt_deleted_product   
TYPE zcl_gw_product_mpc=>tt_product,
ls_deleted_product   
TYPE zcl_gw_product_mpc=>ts_product,
lv_delta_token       
TYPE string,
lo_dp_facade         
TYPE REF TO /iwbep/if_mgw_dp_facade. 

FIELD-SYMBOLS:
<lt_deleted_product> 
TYPE zcl_gw_product_mpc=>tt_product,
<lt_product>         
TYPE zcl_gw_product_mpc=>tt_product.lv_entity_set_name

= io_tech_request_context->get_entity_set_name( ). 

IF lv_entity_set_name = 'Products_DQ'.



 
*   retrieve all relevant entries for this entityset
CALL FUNCTION 'BAPI_EPM_PRODUCT_GET_LIST'
TABLES
headerdata
= lt_product
return     = lt_return.

*   get the data provider facade
TRY.
lo_dp_facade
= /iwbep/if_mgw_conv_srv_runtime~get_dp_facade( ).
CATCH /iwbep/cx_mgw_tech_exception.
RAISE EXCEPTION TYPE /iwbep/cx_mgw_tech_exception.
ENDTRY.

*   call the delta token functionality
TRY.
CALL METHOD /iwbep/cl_query_result_log=>create_update_log_entry_hash
EXPORTING
io_tech_request_context 
= io_tech_request_context
io_dp_facade            
= lo_dp_facade
ir_service_document_name
= mr_service_document_name
ir_service_version      
= mr_service_version
it_entityset            
= lt_product
IMPORTING
et_deleted_entityset    
= lt_deleted_product
et_entityset            
= lt_product
CHANGING
ev_delta_token          
= lv_delta_token.
CATCH /iwbep/cx_qrl_locked.
RAISE EXCEPTION TYPE /iwbep/cx_qrl_locked.
CATCH /iwbep/cx_qrl_delta_unavailabl.
RAISE EXCEPTION TYPE /iwbep/cx_qrl_delta_unavailabl.
ENDTRY.

*   export the delta token
es_response_context
-deltatoken = lv_delta_token.

*   export the deleted entity set
copy_data_to_ref
( EXPORTING
is_data
= lt_deleted_product
CHANGING
cr_data
= er_deleted_entityset ).


*   export the changed entity set
copy_data_to_ref
( EXPORTING
is_data
= lt_product
CHANGING
cr_data
= er_entityset ).
 

ENDIF.





Test the service




  1. Start the Gateway Client by calling transaction /IWFND/GW_CLIENT

  2. Enter the following URI to test your implementation:   /sap/opu/odata/sap/ZGW_PRODUCT_SRV/Products_DQ

  3. You should get a response containing all products. Almost at the end of the XML
    document, you will get your delta token (of course in your response you will
    get a different ID)  <linkrel="delta" href="Products_DQ?!deltatoken= '005056A2025C1EE2BFE687AFDC2FAAF4_20130807073741'" />

  4. Now enter the URI from above again, but this time add your delta token (copy and paste it from your response). You URI should look something like this (but will have a different ID):/sap/opu/odata/sap/ZGW_PRODUCT_SRV/Products_DQ?!deltatoken='005056A2025C1EE2BFE687AFDC2FAAF4_20130807073741'

  5. After you click on execute, you will notice that the response does not contain any products at all. This is because the Delta Token functionality returns only changed records and no products were changed in the meantime.

  6. You could now change a product and execute exactly the same URI as in step (4). The response should contain exactly one product.




Technical Restrictions



Key Fields of Entity Type



There are strict restrictions regarding the key fields of Entity Types that are used with the basic delta query functionality:



  • The maximum legth (output length) of a key field must not exceed 200 characters

  • If several key fields were defined for one Entity Type then the combined legth (output length) of those key fields must not exceed 200 characters

  • The key field type must have a fixed legth; therefore types of variable legth (e.g. type String) are not permitted



JSON Format



Currently the JSON format is not supported in combination with the delta query functionality.



Expand System Query Option ($expand)



Currently the Expand System Query Option ($expand) is not supported in combination with the delta query functionality.



Paging




Both the server-driven paging and the client-driven paging are not supported in combination with the delta query functionality. As a result, the following query options are not supported in combination with the delta query functionality:




  • Skip Token System Query Option ($skiptoken)

  • Top System Query Option ($top)

  • Skip System Query Option ($skip)




Further Notes



Clean-Up Report



The Delta Token functionality based on Delta Request Log Component uses two database tables to store hash values: /IWBEP/D_QRL_HDR and /IWBEP/D_QRL_ITM. Once this offline functionality is used productively the size of these tables will increase with each request. We offer areport that can be run to clean up outdated entries: /WBEP/R_CLEAN_UP_QRL. We suggest that you schedule a job for this report so that it is executed periodically, for instance on a daily basis.

28 Comments