Enhanced OData Parsing

[Former Member]

I recently posted a blog regarding the ABAP implementation of various aspects of OData URI schemes. While preparing for part two of that series I came across some aspects of Gateway OData handling that seem to be lacking, so before I continue on to more in that series I thought it would be good to share how I’ve dealt with this (code junkies will need to page down a bit for that).

For the record, I’m discussing the state of play with Gateway 2.0, SP5.

Update June 2013: with the official release of Fiori recently, the SAPUI5 integration with Gateway and Eclipse are affected by some "unused" settings discused here.

What’s bugging me?

Currently, the service document and metadata publish the basis for a contract that the OData client should be agreeing to. These contain SAP annotations that further stipulate the access conditions. These annotations are in the SAP namespace. I also cannot find any public documentation of what the annotations mean – if it’s there it’s in some obscure location. It certainly isn’t available at http://www.sap.com/Protocols/SAPData at the time of writing. 

It would therefore seem that it is up to us (Gateway service implementers) to explain the rules that annotations are setting out.  I’m not going to get into all of them, but the ones of interest here are:

• Addressing conditions
• Filtering conditions

Addressability

There is an option in the Gateway model to restrict direct addressing of an entity set. This is a good idea if the entityset can be large and should really be accessed in the context of a preceding entity. For example, sales order items are not normally accessed without knowing the order header that they belong to. If we granted unlimited access to an entityset called ‘salesitems’ it would probably run for a week.

The ‘addressable’ annotation is a Boolean that decides the mode of access.

Here is where the annotation states for entity sets are set in Gateway Builder. Also note the ‘Req. filter’ annotation – more on that below.

If we turn on the addressability option, we can restrict the access – cool! Let’s make ‘foos’ addressable regenerate and have a look at the service document…

What the …?

Why was the addressable setting turned off for all my newly modeled entities? That should mean they can’t be reached in the first place, but I know from testing that they worked fine. Furthermore, when I made foos addressing active, its annotation disappeared from the service document.

Purely from an OData client perspective, this is interpreted as:

·      Entity sets, by default, are not addressable. This is obviously not right!

·      If there is no “addressable” annotation, assume that its value is true. This only makes sense if the above point operated correctly.

So the client has this fuzzy spec and may not take any notice of it.

‘bars’ is supposedly not addressable and should be reached by a navigation from a foo (foo is parent to bars). Guess what – ignoring the “contract”, I can get ‘bars’ and it dumps thousands of them back to me. There is no runtime check on this “rule”.

I’d prefer something that looks like a rule configuration option (some will argue that it’s not a rule) to actually have some enforcement. I suppose I could live with that except there’s another itch I need to scratch…

Update: you will find that some of the Eclipse tools will not expose entitysets as a service entry point if they are not addressable. If you cannot see an entityset that you know should be there, check the addressability settings.

Filter annotations

‘foos’ are my root in the model and I want to be able to run some sort of query on them – in fact, I’m going to tell the client that they can only access foos by filter conditions that my model dictates. This is to protect my server in terms of performance. Luckily, there are annotations that I can use to tell the client all about this.

As mentioned above, there is a box for this too.

I may also specify which foo entity properties can be filtered. Here I choose ‘Category’ and ‘Currency’ – so I can only have one or both of these properties in a ‘foos’ filter string.

Service document now states:

And metadata for the foo shows:

Did you spot it? Yes, while the service document is willing to tell us what is not filterable, it’s somewhat circumspect in telling us what IS filterable! At least there’s a pattern emerging, even if it’s not official:

If it’s not false it’s true.

Fair enough and not particularly original – except if all my properties were filterable the client wouldn’t even have a notion that there was any kind of distinction.

‘foos’ is supposedly filter-only and should be filtered by category and/or currency. Guess what – ignoring the “contract”, I can get ‘foos’ without a filter and it dumps thousands of them back to me. I can use a filter on any property and it dumps thousands of them back to me.

There is no runtime check on either of these “rules”.

Trusted client?

Picking holes in all of the above may be pedantic, after all we are developing the client and know about this stuff. Oh I forgot, the idea was to open up SAP data to non-SAP clients wasn’t it! We can expect a knock on the door from Bill’s Witnesses.

Given that the annotations spec seems non-existent, are we really going to trust clients to abide by the contract? What we need are some of those pesky messages from SAP telling them that they’ve been naughty – because they will be!

Let’s get coding!

What are we coding? I’m going to show you how I solved the absence of the rule validation for addressability and filtering.

The following enhancements are shown for guidance and are copied at your own risk; since they are within a core component you should satisfy yourself that the enhancements work with all viable URI formats.

Also note that it is only really viable to make this enhancement for GET and QUERY, it will block any change operations; do not use if the entity is going to be altered by the service.

The primary checks on entity set rules are actually quite easy to implement and both can be placed in one enhancement.

We can intercept the request at the point where the Gateway creates it as a URI object and validates the form of the URI. The class for the URI object is /IWCOR/CL_DS_URI.

Various generic URI checks are made in the constructor of this object so it’s surprising that it wasn’t extended to look at some of the more particular entity settings.

One of the methods called by the constructor is handle_entity_set. This validates the URI in terms of what entity set, if any is being addressed. The validation is on the OData protocol compliance, i.e. is the URI in the right format and does it contain an entity, key, system query options, etc. However, at this point the entity set name has been extracted so it is just a matter of reading the set definition and comparing it to the URI content. After a bit of investigation I decided that it could be enhanced so I went ahead.

I opened an implicit enhancement at the pre-method point in handle_entity_set.

* Check for the need to filter entitysets according to model definition.

data: zz_entity_set type ref to /iwcor/cl_ds_edm_named,
    zz_annotation type ref to /iwcor/if_ds_edm_annotations.

* when the key is blank, this is an access on an entityset, not a navigation through it.
  if iv_key_predicate is initial.
*   cast to annotation interface
    zz_entity_set ?= io_entity_set.
*   get the annotation object from the set
    zz_annotation ?= zz_entity_set->/iwcor/if_ds_edm_annotatable~get_annotations( ).
    if  zz_annotation is bound.

*     check the filter requirement. If the definition requires a filter, there should be a
*    ‘$filter’ key in the parameter list.
      if zz_annotation->get_annotation_attribute( iv_name      = 'requires-filter'
                   iv_namespace = /iwcor/if_ds_edm=>gc_namespace_sap ) = 'true'.
        read table mt_query_parameter
          with key name = '$filter' transporting no fields.
        if sy-subrc <> 0.
          raise exception type zcx_ds_uri_syntax_error
              exporting
                textid  = zcx_ds_uri_syntax_error=>entityset_requires_filter
                segment = zz_entity_set->/iwcor/if_ds_edm_named~get_name( ).
        endif.
      endif.

*     check addressability

*     assume that sets are addressable unless a 'false' is specifically declared.
      if zz_annotation->get_annotation_attribute( iv_name      = 'addressable'
                iv_namespace = /iwcor/if_ds_edm=>gc_namespace_sap ) = 'false'.
          raise exception type zcx_ds_uri_syntax_error
              exporting
                textid  = zcx_ds_uri_syntax_error=>entityset_not_addressable
                segment = zz_entity_set->/iwcor/if_ds_edm_named~get_name( ).
      endif.
    endif.
  endif.

N.B. If the URI contains a key predicate, e.g. “(key=‘somekey’)”, the left-side cardinality is ‘from 1’ so any filtering considerations would apply to the right hand side, which is not dealt with at this point in the URI handling. The right hand side is most likely a navigation and that opens up the question of navigation set filters which would require an enhancement in the appropriate spot.
This enhancement only deals with immediately addressed entity sets.

I have also subclassed the GW exception CX_DS_URI_SYNTAX_ERROR so that I could a) add some more meaningful messages, and b) have the Gateway core trap the exception as if it were standard.

The addressable requirement check needs nothing further to be added.

The filter requirement can still be checked in more detail. If a filter is marked as required, I can then check that the properties in the filter are ones nominated as filterable.

The code for this goes into the “<entity>_get_entityset” method of the data provider extension;  no enhancement is required.
The following check makes sure that anything declared in the filter is a “filterable” property.  A business rule exception is raised if the filter input is incorrect.

data: lo_model type ref to /iwbep/cl_mgw_odata_model,
        lo_metadata_provider type ref to /iwbep/if_mgw_med_provider,
        lv_internal_service_name  type /iwbep/med_grp_technical_name,
        lv_internal_service_version type /iwbep/med_grp_version,
        lv_ent_id type  /iwbep/if_mgw_med_odata_types=>ty_e_med_entity_id,
        ls_entitytype ref to /iwbep/if_mgw_med_odata_types=>ty_s_med_entity_type,
        lv_property_name type /iwbep/if_mgw_med_odata_types=>ty_e_med_entity_name,
        ls_property type ref to  /iwbep/if_mgw_med_odata_types=>ty_s_med_property.

  field-symbols: <filter> type /iwbep/s_mgw_select_option.

* If a filter is supplied, cross check the inbound properties in the filter against the model
* definition.

  if it_filter_select_options is not initial.
*---get service details from context
    mo_context->get_parameter(
      exporting
        iv_name  = /iwbep/if_mgw_context=>gc_param_isn
      importing
        ev_value = lv_internal_service_name ).

    mo_context->get_parameter(
      exporting
        iv_name  = /iwbep/if_mgw_context=>gc_param_isv
      importing
        ev_value = lv_internal_service_version ).

*   Get a model instance
    lo_metadata_provider = /iwbep/cl_mgw_med_provider=>get_med_provider( ).
    lo_model ?= lo_metadata_provider->get_service_metadata(
                   iv_internal_service_name    = lv_internal_service_name
                   iv_internal_service_version = lv_internal_service_version ).

*   Get the entity definition ID.
    ls_entity = lo_model->get_entity( iv_entity_name = 'foo'   ).

    loop at it_filter_select_options assigning <filter>.
      try.
          lv_property_name = <filter>-property.
          ls_property  = lo_model->get_entity_property(  iv_entity_id = ls_entity->entity_id
                                    iv_property_name = lv_property_name ).
        catch /iwbep/cx_mgw_med_exception .
          continue.
      endtry.
*     Exception if the filter switch is "off".   (Apparently ' ' is true in this context!)
      if ls_property->filterable = abap_undefined.
        raise exception type /iwbep/cx_mgw_busi_exception
          exporting
            textid       = /iwbep/cx_mgw_busi_exception=>filter_not_supported
            filter_param = <filter>-property.
      endif.
    endloop.

  endif.