Application Development Discussions
Join the discussions or start your own on all things application development, including tools and APIs, programming models, and keeping your skills sharp.
cancel
Showing results for 
Search instead for 
Did you mean: 

Unable to decompress large data with CL_ABAP_UNGZIP_BINARY_STREAM

Former Member
0 Kudos
771

Hello all,

i would like to stream the huge amount of XML data to the application server compressed. It seems that the class pair CL_ABAP_GZIP_BINARY_STREAM / CL_ABAP_UNGZIP_BINARY_STREAM should do this job.

So far the compression works, because the compressed chunks are small enough. I considered the dependency between the buffer length and the overall amount of bytes to be compressed.

The decompression seems to produce some kind memory leak or at least fails for unknown reason. For relatively small compressed data amounts the CL_ABAP_UNGZIP_BINARY_STREAM works just well. Larger files, even if decompressed within a loop in small portions (max 4KB large chunks or smaller), can not be decompressed without to increase the buffer size. For very large files (decompressed in loop in a small chunks) the buffer can not be just large enough and the procedure ends up in an out of memory.

So either i use CL_ABAP_GZIP_BINARY_STREAM / CL_ABAP_UNGZIP_BINARY_STREAM pair wrong or there are some troubles with the memory management inside the C implementation of these classes.

If somebody knows about this problem or has got an idea on how to resolve it, any help would be very welcome!

Here the error message i get:

An exception occurred that is explained in detail below.
The exception, which is assigned to class 'CX_SY_COMPRESSION_ERROR'
---
The function IctDecompressStream returns the return code 30

Code that i use to decompress:


  
* buffer
data l_gzip_buff_decomp type xstring.
* buffer size
data l_gzip_buff_len_decomp type i value 16777216.

* empty hex
data c_empty_x type x.

 * chunk of compressed data to be decompressed at once
data   l_buff type x length 4096.
 data: uref       type ref to user_outbuf.
 data: csref       type ref to cl_abap_ungzip_binary_stream.

      create object uref.

      create object csref
        exporting
          output_handler = uref.


      call method csref->set_out_buf( 
                                 importing out_buf = l_gzip_buff_decomp
                                                 out_buf_len = l_gzip_buff_len_decomp ).

      l_file = 'very-large_file.xml.gz'.
      open dataset l_file in binary mode for input.

      do.
        read dataset l_file into l_buff length l_len.

        if l_len > 0.
          call method csref->decompress_binary_stream
            exporting
              gzip_in     = l_buff
              gzip_in_len = l_len.
        else.
          close dataset l_file.
          exit.
        endif.
      enddo.
* close the stream and flush the unzipped buffer
 call method csref->decompress_binary_stream_end
      exporting
        gzip_in     = c_empty_x
        gzip_in_len = 0.

2 REPLIES 2

former_member643919
Participant
0 Kudos
100

Hi Gena,

I'm facing exactly the same problem as you...

Since this post is an old one, I imagine that you may not remember, but I have to try...

Have you solved it? If yes, could you please tell me how?

I've tried to use CL_ABAP_GZIP and CL_ABAP_UNGZIP_BINARY_STREAM and I'm getting the same error 30 at the  IctDecompressStream function.

Tks in advance,

Flavio.

0 Kudos
100

Maybe you can create a new post to ask the other 1 million people? You'll probably be more lucky