ABAP: Fast JSON Serialization

[TimoStark]

tldr; Install https://github.com/timostark/abap-json-serialization and enjoy the fastest possible JSON serialization. The result will be a 10x faster JSON serialization and deserialization compared to /UI2/CL_JSON at the same quality. Be warned though: Read the limitation section first. 

Oh no - another JSON Serialization Blog Post? Hey - At least no blog about excel exports 🙂

So why are we in need of a "new" way for JSON serialization? The reason is simple: Runtime! Especially when working with custom REST services with a big payload you will notice a lot of runtime getting lost in JSON serialization. Loosing 30% of your runtime in JSON serialization makes me very unhappy (when I just optimized my more difficult business class).

So, what are our goals:

1. Fast


2. Support Camel-Case


3. Support real booleans and numbers


4. Not need to be easy or generic (I will accept a bad life as a developer if it is fast and reliable).

There are already multiple solutions out there - just to mention the most important ones:

• /UI2/CL_JSON (plain ABAP)


• CALL TRANSFORMATION  (Kernel solution, using XSLT or Simple Transformation Mapping)


• CL_TREX_JSON_SERIALIZER (plain ABAP)


• XCO_CP_JSON  (S/4 Only, not stable, extremely slow and crashes - therefore let's keep that one out for the moment)

So how are they behaving from runtime perspective. Let's take a very simple example and serialize 5.000 lines of SFLIGHT lines and a very complex and deep structure:

So what does that tell us?

Not really surprisingly the only feasible solution on a ABAP stack is the usage of CALL TRANSFORMATION - as this is executed directly in the Kernel, thus not depending on slow ABAP String concat and/or field-symbol traversal.

It might be strange but always remember: Building up strings using concats and traversing over field-symbols inside a structure is very slow in ABAP compared to native languages --> Where possible Kernel Modules like Simple Transformations are preferable performance wise.

There are however quality problems when using CALL TRANSFORMATION ID:

1. No Camel-Case


2. No real "booleans" (instead 'X' is printed.. tell that somebody outside of the SAP world)


3. No real NUMC (instead leading 0s are printed)

There is one solution which was already mentioned in a blog post, using a custom ABAP transformation to at least support camel case. Unfortunately, that throws away the performance benefit as the fast kernel module has to go up to the ABAP stack for a simple "to-camel-case" transformation.

My suggested solution is, that we use CALL TRANSFORMATION for what it is actually thought: to transform data using ST transformations (Simple Transformation). Remark: CALL TRANSFORMATION can also be used for XSLT Transformation (which are much more powerful but also slower - see remark by @Sandra Rossi), but this is simply not required here. This means we are creating an own Simple transformation for the structure/table-type we want to serialize (nested structures are of course possible).

Let's see an example transformation for the table SFLIGHT (shortened):

Nobody wants to write that code (and for sure nobody with a right mind will want to keep that transformation up to date) - but let's first see the runtime impact.

==> The solution is around 10 times faster than /UI2/CL_JSON, while having the same quality as a result.

As already said of course nobody wants to write these ST mappings - especially for deeply nested structures this is horrible.

Therefore, I've published a small helper program ZJSON_TO_XSLT under MIT license to GitHub which allows you to directly create those transformation for any structure/table

 

Output (next to the generated transformation).

Execute the transformation using normal CALL TRANSFORMATION call:

DATA(lo_writer_json) = cl_sxml_string_writer=>create( type = if_sxml=>co_xt_json ).

CALL TRANSFORMATION ZSFLIGHT SOURCE root = lt_flights RESULT XML lo_writer_json.

DATA(lv_json) = cl_abap_codepage=>convert_from( lo_writer_json->get_output( ) ).

In my customer projects I am using the API called in the program in a regular job (including a mapping-table) which updates the transformations on the development system in a regular manner. If you want to spend a lot of time you could even create the transformations "live" as local objects on the first access. I personally do not like the approach of local development objects though.

==> Using JSON serialization with fixed transformations, you can get an extremely fast JSON serialization and deserialization while still having high quality.

Limitations:

• A big word of warning: The solution is thought for performance critical development. The solution comes with very high costs: you have to think of an additional development object (the transformation). Even if it updates automatically, it can get out-of-date, you can forget it or it can get corrupted.


• Summed up: If you do not have a problem (i.E. customer complains about slow type-ahead, where you need response times in ms) do not create additional problems using a more complex solution mentioned here.


• The solution works as long as you know the exported JSON types upfront ( i.E. have static data-types). For dynamic data-structures this solution will not work.