When do we say that an application performance is good enough?
users will have no patience with an app that is not responsive but a performance problem in an ABAP report might only be revealed when it dumps with in production because of the sheer size of the data to be processed there. So how can we check if a process with an acceptable run-time in the development and quality systems will also have an
acceptable response time for larger input?
Let us first note there what
acceptable response time means. We might OK an one-time extraction process that runs for 10 hours in the background, or a report that takes a day to process a million entries in a data migration context, but veto an one-minute-wait for a single record access.
So the run-time alone is not a good performance indicator, we should focus on
order-of-growth comparisons instead. But the analysis of algorithms can be quite complex or have some prerequisites many are not comfortable with. However, there is an empirical estimation method that is as easy as writing an ABAP unit test.
Some Theory
Caveat: We cannot avoid some maths here, but in those times of Covid19, I hope you might have seen
animations explaining why
exponential growth can be bad.
Let us assume we have a process that takes more time with a larger input (e.g. higher number of
"rows" like in this
concrete example of an ABAP problem).
From our vantage point (software), the running time T is described as a function T(N) of an input size N (e.g. number of
"rows"). We model this dependency as power law
where c is a constant.
What is the effect of doubling the size of the input on the running time?
Since the formula is relatively simple, the
order-of-growth b can be easily estimated by comparing T(N) with T(2N) based on the
doubling hypothesis, b is equal to
- log( T(2N) / T(N) ) / log( 2 ) .
If the order of growth b is 2 (quadratic) or higher, you have a potential disaster at hand if the input size cannot be kept small.
Some Code
I implemented this logic as a
unit test. by using the fact that b is also equal to
METHOD profile.
CLEAR rs_time.
rs_time-size = iv_size.
DATA(lv_start) = mi_timer->get_runtime( ).
run_data( iv_size ). " <---- The Code under Test
rs_time-time = mi_timer->get_runtime( ) - lv_start.
IF iv_time GT 0.
rs_time-lg_ratio = log10( rs_time-time / iv_time ).
ENDIF.
ENDMETHOD. "profile
METHOD performance.
* empirical analysis
CONSTANTS c_precision TYPE f VALUE '1e-2'.
DATA lv_size TYPE syindex VALUE 1.
mi_timer = cl_abap_runtime=>create_hr_timer( ).
DATA(ls_time) = profile( iv_size = 0
iv_time = 0 ).
DO 4 TIMES.
ls_time = profile( iv_size = lv_size
iv_time = ls_time-time ).
lv_size = lv_size * 10.
ENDDO.
IF ( ls_time-lg_ratio - c_precision ) > 1 .
cl_abap_unit_assert=>fail( msg = | Performance: { ls_time-lg_ratio } | ).
ENDIF.
ENDMETHOD.
In this case, the input size are 1, 10, 100, 1000. The result is based on the comparison between the run-time for 100 and the run-time for 1000 "rows". So if you can measure your run-time as a function of the size of the input set, then you can already compute an order-of-growth estimation from 2 measurements.
Ref.:
Introduction to Programming in Java by R. Sedgewick, K. Wayne
First published as
https://www.informatikdv.de/a-cheap-performance-evaluation/