Technology Blogs by SAP
Learn how to extend and personalize SAP applications. Follow the SAP technology blog for insights into SAP BTP, ABAP, SAP Analytics Cloud, SAP HANA, and more.
cancel
Showing results for 
Search instead for 
Did you mean: 
mandy_krimmel
Advisor
Advisor
9,501
This blog describes the options for message zipping and compressing of messages in Cloud Integration. It explains the different flow steps and the respective configuration options.

ZIP and Compress Capabilities in Cloud Integration


In some integration scenarios messages have to be exchanged in a zipped and/or compressed format. Cloud Integration offers different flow steps for different use cases.

ZIP Encoder


The ZIP Encoder can be used to zip a single message during integration flow processing.

The ZIP Encoder can be found in the palette under Message Transformers -> Encoders -> ZIP Compression. This flow step has no further configuration parameters, it simply zips a message into a zip archive.

GZIP Encoder


The GZIP Encoder can be used to compress a single message with the gzip algorithm during integration flow processing.

The GZIP Encoder can be found in the palette under Message Transformers -> Encoders -> GZIP Compression. This flow step has no further configuration parameters, it simply compresses a message.

ZIP Decoder


The ZIP Decoder can be used to un-zip a single message from a zip archive during integration flow processing. Only one message may be contained in the archive, or else an error will be raised.

The ZIP Decoder can be found in the palette under Message Transformers -> Decoders -> ZIP Decompression. This flow step has no further configuration parameters, it simply un-zips a message from a zip message archive.

GZIP Decoder


The GZIP Decoder can be used to un-compress a single gzip-compressed message during integration flow processing.

The GZIP Decoder can be found in the palette under Message Transformers -> Decoders -> GZIP Decompression. This flow step has no further configuration parameters, it simply un-compresses a message.

ZIP Splitter (available with the update in May 2020)


The ZIP Splitter can be used to un-zip a zip archive containing multiple files. It splits them into single messages and processes them one after the other.

The ZIP Splitter can be found in the palette under Routing -> Splitter -> Zip Splitter. This flow step has two configuration parameters:


Using the File Name Pattern field, you can define a pattern for the files from the zip archive, which have to be processed. If you, for example, want to process only xml files from the archive and ignore the others, you can define *.xml. If no information is given in this field all files from the archive are processed.

Expressions such as ab*, a.*, *a*, ?b, and so on, are supported. Note that the expression * replaces no character or an arbitrary number of characters. The expression ? replaces exactly one arbitrary character.

Examples:

  • If you specify file*.txt as the File Name Pattern, the following files are processed: file1.txt, file2.txt, as well as file.txt and file1234.txt, and so on.

  • If you specify file?.txt as the File Name Pattern, the following files are processed: file1.txt, file2.txt, and so on, but not the files file.txt or file1234.txt.


By using the Stop on Exception field, you can define if the split process shall stop if there has been an error during the processing of one of the split messages or if the split process shall continue to process the next file from the archive.

ZIP Aggregate (available with the update in May 2020)


The ZIP Aggregate or ZIP Gather can be used to zip several messages coming from different branches or splits into a zip archive.

The ZIP Gather can be found in the palette under Routing -> Gather. In the Gather step you can select Zip as Aggregation Algorithm. This Aggregation Algorithm can be used for all Incoming Formats. The flow step has one configuration parameter:


In the File Name field you can enter a simple expression to create the file name for the files in the archive. For example, with ${header.id}.xml you can create the files with different names based on the header id. The header has to be set before the gather step. You need to make sure that the file name is unique within the archive, or else there will be an error during processing because multiple files cannot have the same name within a zip archive. If nothing is defined in the field, the header CamelFileName is evaluated. In case neither an expression nor the CamelFileName header is specified during runtime a unique file name is generated.

TAR Splitter (will be available with the update in June 2020)


The TAR Splitter can be used to un-tar a tar archive containing multiple files. It splits them into single messages and processes them one after the other.

The TAR Splitter can be found in the palette under Routing -> Splitter -> Tar Splitter. This flow step has two configuration parameters:


Using the File Name Pattern field, you can define a pattern for the files from the tar archive, which have to be processed. If you, for example, want to process only xml files from the archive and ignore the others, you can define *.xml. If no information is given in this field all files from the archive are processed.

Expressions such as ab*, a.*, *a*, ?b, and so on, are supported. Note that the expression * replaces no character or an arbitrary number of characters. The expression ? replaces exactly one arbitrary character.

Examples:

  • If you specify file*.txt as the File Name Pattern, the following files are processed: file1.txt, file2.txt, as well as file.txt and file1234.txt, and so on.

  • If you specify file?.txt as the File Name Pattern, the following files are processed: file1.txt, file2.txt, and so on, but not the files file.txt or file1234.txt.


By using the Stop on Exception field, you can define if the split process shall stop if there has been an error during the processing of one of the split messages or if the split process shall continue to process the next file from the archive.

TAR Aggregate (will be available with the update in June 2020)


The TAR Aggregate or TAR Gather can be used to combine several messages coming from different branches or splits into a tar archive.

The TAR Gather can be found in the palette under Routing -> Gather. In the Gather step you can select Tar as Aggregation Algorithm. This Aggregation Algorithm can be used for all Incoming Formats. The flow step has one configuration parameter:


In the File Name field you can enter a simple expression to create the file name for the files in the archive. For example, with ${header.id}.xml you can create the files with different names based on the header id. The header has to be set before the gather step. You need to make sure that the file name is unique within the archive, or else there will be an error during processing because multiple files cannot have the same name within a zip archive. If nothing is defined in the field, the header CamelFileName is evaluated. In case neither an expression nor the CamelFileName header is specified during runtime a unique file name is generated.

 
24 Comments
maik_bosch
Contributor
0 Kudos
HI Mandy,

good to see progress there 🙂

Thanks for the update and regards

Maik
malog
Explorer
0 Kudos
Hi Mandy,

 

Does this ZIP splitter works with .7z files?

 

Thanks for the update..keep safe..

 

Matt
mandy_krimmel
Advisor
Advisor
Hello Matt,

no .7z files are not supported, only zip.

Regards

Mandy

 
barttruijens
Discoverer
0 Kudos
Hello Mandy,

Does the ZIP Decoder handle zip files which are secured with a password ?

Regards

Bart
mandy_krimmel
Advisor
Advisor
0 Kudos
No, this is not supported.

Best regards

Mandy
arunmaarirajha
Explorer
0 Kudos
Hi Mandy,

Nice blog explaining the zip/unzip functionality.

Please let me know if gunzip (.gz) can be extracted using GZIP Decompress?

Thanks,

Arun Maari Rajha

 
mandy_krimmel
Advisor
Advisor
Yes, the GZIP decoder can extract .gz files.

Best regards

Mandy
arunmaarirajha
Explorer
0 Kudos
thank you for confirming!
freek-creetion
Discoverer
0 Kudos

Hello Mandy,

 

Thanks for the nice blog.

 

I have a use case where I need to send a GZIP containing multiple files via AS4. AS4 standard only supports GZIP. If I read correctly the ZIP Aggregate or ZIP Gather will not help me since they are not supporting GZIP, correct? Is there another way in CPI to bundle multiple files in a GZIP?

 

Kind Regards,

 

Freek

mandy_krimmel
Advisor
Advisor
0 Kudos
No, there unfortunately there is no such configuration option available, you would have to use script to do this specific use case.

Best regards

Mandy
freek-creetion
Discoverer
0 Kudos
Thank you for the quick reply. I'll look into using a script.

Kind Regards,

Freek
former_member777112
Discoverer
0 Kudos

HI Mandy, how are y doing?

Congratulations for the blog and if you can help, i will be grateful.

I have a very simple flow in SAP CPI, I use a "content modifier", add "Authorization" and "Content-Type", but it doesn't work. The zip file is not sent, it even appears for download in the trace, but it does not reach the server.

 

Integration Flow -> Runtime configuration

 

HTTPS -> Connection

 

Content Modifier -> Message Header

 

HTTP -> Connection

 

The zipped file must go in the body of the submission, we can see it in the payload. Although, when it arrives at the endpoint we forwarded, the file does not exist in the body.
mandy_krimmel
Advisor
Advisor
0 Kudos
Do I get it right, that the flow not uses the zip encoder at all? I would suggest to open a ticket with the question. This is not easily answered without all the details and a sample flow.

Best regards

Mandy
former_member777112
Discoverer
0 Kudos








The flow starts receiving a zip in the body of the file, inside SCPI we handle the authentication part and forward the same zip in the body to the destination. Although, as I said, the destination acknowledges not receiving this file.








Ícone "Verificada pela comunidade"

















mandy_krimmel
Advisor
Advisor
0 Kudos
The processing needs to be traced and checked in detail. Please open a ticket and attach the integration flow and the trace details.

BR

Mandy
govind1122
Discoverer
0 Kudos
Hi Mandy,

I have a scenario in SAP CPI like Sender is SFTP which contains multiple files so i have to pick all those files and sends to target in a single .zip file, So if we unzip the file we have to get all the sender files.

Kindly please help me on this, As i tried multiple ways.

Thanks

DG
mandy_krimmel
Advisor
Advisor
0 Kudos
Hi,

I would envision that a flow with a poll-enrich (Define Poll Enrich | SAP Help Portal) step with sftp adapter in combination with a looping process call (Define Looping Process Call | SAP Help Portal) should do the trick to collect all the files.

Either store them in a data store and afterwards use the data store select to get all the files and process them as one message in a zip encoder. Or use the Aggregator step to do this. But this would only work for XML messages witch would then be combined in a big XML.

You could also store the files as exchange property and use a script to create a zip file containing all the files.

At the moment the zip encoder only encodes one file, not many. We are discussing about an extension in the aggregator step to create zip archives for multiple files, but this is nothing planned short-term.

Best regards

Mandy
allysajohn
Explorer
0 Kudos
Hi Mandy!

Thank you for the blog! It is easy to understand.

May I ask if the FileNamePattern only support one extension at one time? For example, I want to unzip only with extension .xml and .txt. How do I specify these two extensions in the FileNamePattern? I have used *.xml,*.txt but it seems does not work. Current workaround is I use two Zip Splitter which I specify only one extension at one FileNamePattern.

Looking forward for your reply.
mandy_krimmel
Advisor
Advisor
0 Kudos
Hello,

this is not possible, you would have to go the way you already do, via two zip splitters, each for one pattern.

Best regards

Mandy
tapas_cg
Explorer
0 Kudos

Hi @mandy_krimmel ,

Is there a possibility to create multiple file folders within CPI containing multiple files and Compressing and zipping this file folder.

Example, all the files to be stored in "ABC" folder. "DEF" folder containing this "ABC" folder and the final Parent Folder "DEF" is only compressed and zipped.   

DEF.zip >> ABC (file folder unzipped) >> {contains multiple files}

Thanks

Tarun

 

mandy_krimmel
Advisor
Advisor
0 Kudos

Hi Tarun,

please try to put the complete path into the filename field (e.g. /DEF/ABC/file<x>.xml) in the ZIP Aggregator configuration, then the whole directory will be zipped. In the receiver adapter you then only specify the root folder where you store the DEF.zip

Please try if this works.

Best regards

Mandy

tapas_cg
Explorer
0 Kudos

Hi Mandy, 

Thankyou very much for your suggestion. I tried the same, however it does not seems to be working.

The output file "DEF.zip" does gets generated, but it neither have any subfolders or files inside it. Below are the ZIP Aggregator and Receiver SFTP Adapter details for your reference. Kindly suggest for any more alternate ways if possible via CPI.

tapas_cg_0-1716871749172.png

tapas_cg_1-1716871980314.png
Thanks

Tarun

 

 

mandy_krimmel
Advisor
Advisor
0 Kudos

Hello Tarun

I would suggest to use tracing to understand where the file/payload gets lost. Is it in the gather step? Maybe you use a dedicated file name and not *.xml for testing? 

If you do not find the problem, I would suggest you open a ticket on LOD-HCI-PI-CON-SOAP so that the experts could have a look whats going wrong. Please share the solution afterwards.

Thank you

Mandy

tapas_cg
Explorer
0 Kudos

Hi Mandy, 

We had raised the Support Ticket with SAP and they have responded back with a solution and it seems to have worked. In Gather, we provided the complete folder structure along with the filename of files to be gathered and zipped together as below :   

tapas_cg_2-1718335770389.png

 

On the Receiver SFTP we provided the folder name of the Parent Folder only which was required to be zipped and placed at the target SFTP folder. 

tapas_cg_1-1718335679030.png

 

Thanks For your feedback and support in this issue resolution,

Regards,

Tarun