- Introduction
- Context
- Case 1: No BTP pre-filtering or post-filtering
- Case 2: BTP post-filtering
- Case 3: BTP pre-filtering
- Case 4: Split BTP pre-filtering
- Conclusions
Introduction
In the late 19th century, the scientific community started to face unexpected issues, some of them quite serious, upon proven experiments for which no accepted formula was able to produce reasonable prediction or explanation. This originated the entire collapse of the current physics, now called ‘classic physics’, into a new frame ignited by the quantum mechanics. The game-changing problem was named ‘the ultraviolet catastrophe’.
Similarly to what happened back then –hopefully less ontologically problematic–, the SAP transition from classic to Fiori-oriented development brings about some anomalies not especially easy to overcome without a systemic revolution of the technology itself. One of the problems we have experienced regards the paging mechanism when rendering a Fiori list report via back-end OData service.
Context
Let’s assume the following scenario:
On-premise OData service.
BTP Fiori list report application consuming the back-end service via custom entity.
Respecting the Clean Core principles, the report combines information from the back-end –only read–, and local BTP tables. To ensure integrity, the key fields are the same on both sides.
Consequently, the selection filters can refer to both back-end and BTP fields.
The BTP application, fields and tables underneath dominate the entire execution engine. This is to say, BTP-located filtering, if any, must lead the back-end selection.
Case 1: No BTP pre-filtering or post-filtering
Scenario
No BTP-located filter fields are set, so there is no need to implement BTP filtering before or after the service call. In real executions, this is not very likely to happen, but anyway interesting to start exposing the problem.
Behaviour
All pages are correctly delivered through scrolling. Considering n items are hit in the back-end, when reaching the last page, all of them are rendered.
Case 2: BTP post-filtering
Scenario
Here, BTP-located filter fields are set. The idea is to retrieve matching information from the back-end and, next, filter by the BTP conditions.
Behaviour
If the page is not reduced during the post filtering, i.e., no line is impacted by the BTP filtering, then it is correctly shown and the scroll for the next one continues to be enabled.
Else, if the page size is reduced, the system considers it is the last so no more scrolling is enabled; even trying to scroll down no more service calls are run. The page cropping leads then to missing information.
Case 3: BTP pre-filtering
Scenario
If the post-filtering does not work, why not first filter in BTP and then use the key fields to retrieve from the back-end only exactly what is needed?
Behaviour
Here, x items are pre-filtered in BTP. If, additionally, back-end filtering is set, the final number of retrieved items, say n, can potentially be less than x.
Anyway, the strategy seems finely focused but the problem arises when too many items are pre-filtered so the URL to call the service is too long to be accepted and leads to an exception.
Case 4: Split BTP pre-filtering
Scenario
Considering some trust was put in the previous step, maybe we could split the filtering and manually call the service in small packages to avoid problems with the URL.
Behaviour
To avoid the cropped page problem, the main purpose is to always build complete pages to keep the execution alive; otherwise, it stops. The challenge is to seamlessly advance through packages as if no additional calls were executed.
In BTP, x items are filtered and supposed to be sent to the back-end. The URL length is first checked:
If it is short enough to be accepted, then no split is needed and the call can be executed as conventionally.
In other case, the logic will split the x items into small packages of s; the package size must be greater than the page size so as to avoid the scroll disabling.
For the first call, the s items hit n back-end items, so the report counter shows n as the total number of items, but this value is not correct, as there are more items waiting to be sent.
By scrolling down, the complete list of n items related to the first package is delivered correctly, so for now no need to process other packages.
If the current page size is less than the determined size, i.e., if the page is cropped, this means we have reached the end of the package, so the next one can be processed.
The next package is then processed making sure the skip is reset to retrieve all back-end hits from the beginning. The cropped page is then completed with p - q items from the new package.
For next page the skip is automatically set to the page size and because there is no option to keep track of the actual index we are in –stateless processing–, a gap of missing information can occur.
Conclusions
The development of BTP reports via custom entities where local filtering is supposed to restrain the information coming from the back-end introduces a series of problems not easy to overcome. Business may claim classic development where all the data and logic was set on-premise did not carry this type of problems.
As it happened to physics, the transition to modern SAP development presents by comparison some obstacles for which quick adaptation is needed. The answer to this problem, something like quantum mechanics in our recurrent reference, goes through using a more powerful technology; for the time being, external entities based on HANA virtual tables are the best alternative to avoid the paging catastrophe.
Bluesky
