How to archive files in SAP CPI FTP/SFTP Sender

[stephen_xue]

Scope

SAP CPI provides FTP/SFTP sender adapter used to pickup files from the source file server. There are quite a few good blog posts on

• how to configure a FTP/SFTP sender adapter from scratch


• how to pick up file from an on-premise server

This blog post will not provide overview configure steps for a FTP/SFTP sender adapter. For generic and comprehensive steps, please refer to these ones:

https://blogs.sap.com/2018/11/16/cloud-integration-how-to-connect-to-an-on-premise-sftp-server-via-c...

https://blogs.sap.com/2020/07/08/cloud-integration-connecting-to-ftps-servers-using-the-ftp-adapter/

Instead, the focus will only be on  how to archive files by using the adapter configuration.

In Scope

• Archive files


• Change filename


• Reletive/absolute path


• Java simple expression


• Timezone ID

Out Scope

• BTP trial setup


• CPI on boarding


• Cloud Connector


• Generic FTP/SFTP adapter configuration

Background

Under the tab 'Processing', there are four options of the field 'Post-Processing'. These are:

• Delete file


• Keep file and mark as processed


• Keep file and process again


• Move file

The option 'Move File' is to archive file and this is the option that I want to analyse and introduce in the below text. I found that although there are many blog posts on how to configure the adapter, whereas not too many on how to configure archive files. But nevertheless, I assume this option should be the most widely used. Here are the scenarios for your reference and all of them are tested working. Hopefully you can find it useful.

Initial Status

This is my FTP folder structure and all of the below scenarios will be based on this situation.

/  (root folder)

 |

 |------test  (folder)

 |          |

 |          |------ Source.txt  (file)

 |          |

 |          |______ archive (folder)

 |     

 |______archive (folder)

Scenarios

1. Absolute Path

1.1 Move File into sibling folder

The source file /test/Source.txt will be picked by the adapter and moved into /archive by using absolute path. Filename will not be changed. /test and /archive are sibling folders.

A absolute path starts with a '/'. Since the path type is absolute, there is no big difference between sibling folder, or parent folder, or child folder actually.

Specify the path without providing filename. The source file will be moved to the /archive folder with the same file name.

1.2 Move file into child/parent folder

As explained, the absolute path doesn't tell much difference among these different situations. Just provide the path without filename and the file will be moved.

for chid folder, if the file Source.txt is to be moved into /test/archive, just fill the path to the above field.

for parent folder, if the file Source.txt is to be moved into /, the same, just fill it to the above field.

2. Relative Path

Relative path doesn't start with a '/'.Its context is always starting from the current folder.

2.1 Move file to Child Folder

The format is <child folder name>

For example, if we plan to move /test/Source.txt into /test/archive, just fill archvie in the Archive Directory field.

or ./archive will still work.

2.2 Move file to Parent Folder or Sibling Folder

Parent Folder:

To move into one layer up, use ../

To move into two layers up, use ../../

For example to move /test/Source.txt to / , use ..

Sibling Folder

to move /test/Source.txt to /archive, use ../archive

3. Static File Name

I found there is no way to specify a static file name in the Archive Directory field. As in the above scenarios, if the filename is not provided, the target file name will be exactly the same as the source one. This is the only way to "set a static file name". Otherwise, no matter what static file name has been provided, the adapter will create a new folder by using the filename provided and move the source file into that new folder.

For example, if we plan to move /test/Source.txt into /archive folder and change its name into myTest.txt

We configure it by using reletive path as below

However the reality is:

The adapter will create a new folder named myTest.txt under the folder ../archive and place the source file Source.txt to that new folder with no name change.

find the bellowing newly generated highlighted line starting with letter 'd', which stards for directory

 

4. Dynamic File Name

Variables are in form of Simple Expression. It can be a time, a file name part or a property etc. Please check ref2 for detail information.

The only way to change source file name is either via partially dynamic filename or fully dynamic filename.

For example:

we want to move the file from /test/Source.txt into /test/archive and change the name into Source.bak

The high lighted part is the name variable. For detail information, please check File Expression Language in ref3.

 

5. Dynamic Path

In SAP note 2638196(ref1), it provides an example for dynamic folder. I will not create my example for it. Check it here:

<Archive_Path>/${date:now:yyyyMMdd}/${file:name.noext}.bak
result: /20180430/CPI_Test.bak

6. Timestamp

use the date simple expression to generate a timestamp, for example ${date:now:yyyyMMdd}

Please fine the expession in ref5 for more detail of the datetime format.

For example,

we want to move the file from /test/Source.txt into /test/archive and change the name into Source_<timestamp>.txt

One of the possible configuration can be:

Three variables have been used. The middleware one generates a timestamp

This is a target filename

The time stamp is in UTC timezone.

7. Time Zone

Sometimes we might need to generate timestamp in timezones other than UTC.

Its format is:

${date-with-timezone:now:<Timezone ID>:<Date time in Simple DateTime Format>}

For the list of timezone id, please check ref4

the <Timezone ID> can be: NZ for New Zealand, Australia/Perth for Perth, CET for central Europe etc.

For Date Time in Simple DateTime Format, please be aware that this is part of filename. It should follow the path/filename naming rule as well. for example character ':' cannot be used here.

here are some variables,

${date-with-timezone:now:NZ:yyyyMMddHHmmss}

Add timestamp in New Zealand timezone

${date-with-timezone:now:Brazil/East:yyyyMMddHHmmss}

Add timestamp in Brazil East timezone

${date-with-timezone:now:ROC:yyyyMMdd'T'HHmmss}

Add timestamp in Taiwan timezone, use a upper 'T' to separate date and time

 

Conclusion

No static filename can be provided. Only dynamic filename will work.

By using the simple expression, the timestamp can be added to the archived filename.

Timezone ID can be used to get timestamp of any timezones.

 

Reference

1. How to archive files in CPI SFTP Adapter


2. Get to know Camel’s Simple expression language in SAP Cloud Integration


3. File Expression Language


4. List of tz database time zones


5. Java SimpleDateFormat