Secure File Exchange Using SAP NetWeaver PI - the Big Picture

[wolfgang_hegmann]

Background

Just recently I was asked how to realize an automated file transfer between an on-premise system (e.g. SAP ERP) and an external business partner with an increased level of security including:

1. Sending files over SFTP (FTP over SSH) to a FTP server of a business partner - so "security in transit"
2. Authentication on the FTP server using certificate (instead of user/password).
3. PGP file encryption - so "security at rest". File signing was not required, but is supported as well

Since these security measures are all standard, there are numerous ways to realize this (scripts, MFT toolsets, etc.). Since we have NetWeaver PI already in place for 100+ integration scenarios, I did setup a such a case to verify if this scenario is sufficiently supported using the PI SFTP adapter and PGP module released in 2012.

I found plenty of blogs on SCN covering each individual aspect which helped me a lot to built up this scenario in a very short time. This blog shall combine the hard facts on the PI configuration required to achive such a secure file transfer setup and should be understandable by non-PI experts too.

Solution

To illustrate how PI fits in here, I used two UML/TAM diagrams.

Outbound - we are sending data in files to the business partner.

ERP triggers the file transfer to the FTP server. Depending on the inbound consumption schedule business partners's side the target application data store is updated with a short time delay.

Inbound - we are receiving data in files from our business partner.

Business partner's application either push out new data in files right after the transaction is done or periodically (e.g. once a day). PI polls for new files periodically. If a new file is present, file is read and the content handed over to ERP.

The asterisks indicate the required keys (either SSH or PGP) both parties need to have in order to authenticate against the FTP server and encrypt/decrypt files.

Side remark: File signing is also supported by the PGP module, but was not used in this particular scenario. In case of signing, also the other PGP key would be necessary, e.g. for sending files the own private key would be used for signing on top of the partners public key required for encryption.

Technical Details

Important aspects for implementation in PI

To realize such a scenario with PI, a few important aspects need to be considered here (If you are non-PI person and you are still reading this blog - RESPECT! - Now lets come to the beef😞

1. FTP logon using public key authentication: This is more secure than basic authentication using userid and password. To do such a thing, the business partner hosting the FTP service needs a SSH public key from you which the business partner will place inside the FTP account folder structure. This key has to match the private key the PI server later uses to logon. To get such a thing created, you need to use the SAP NW AS Java Key Store to create a key file. Since this cannot be used by a ssh server right away, you have to convert it using several open source tools. I found two options to do this: Option 1 and Option 2. I was successful with Option 2. The logon credentials in the PI communication channel then need to refer to the entry in the AS Java Key Store.
2. PGP file cryptography: the PI PGP module can not only encrypt and decrypt files but also sign them. This is nicely explained in two blogs - one for encryption and one for decryption. Also see SAP Help Portal for more info.
3. File naming: Since meaningful file names (after encryption) have to be like employees20130628.csv.gpg and the PI feature to place timestamps is only able to put it next to the last file extension and is probably to detailed (would look like employees.csv20130628-164411-969.gpg) you have to use Dynamic Configuration (DC) combined with variable substition. There is a great blog describing how you freely define parts of the file name, like in my case to set a suitable timestamp at exactly the right location in the file name.
4. File Content Conversion (FCC): FCC as own settings tab is not available in SFTP adapter and therefore has to be done with using MessageTransformBean. Simple conversions using MessageTransformBean are nicely explained in the SAP Help Portal. On top of this, the execution of MessageTransformBean has to be embedded between Dynamic Configuration and Variable substitution.

Adapter Module Sequence

So the sequence to place files on a FTP server with SSH and PGP encryption using a PI communication channel is like this:
1) DynamicConfigurationBean: Read the timestamp from XI header and place it in a variable. Has to be done before step 2 otherwise the XI header is no longer accessible.
2) MessageTransformBean: Convert XML into a flat file.
3) PGPEncryption: The file stream is encrypted according the the PGP config (e.g. ASCII armored).
4) CallSAPAdapter (ModuleProcessorExitBean for SFTP): Finally the file name is put together (using variable substitution) with a meaningful timestamp and two file extensions like .csv.gpg added at the end and the encrypted file is transferred via SSH.

For the other direction it is exactly the other way around - except that Dynamic Configuration might not be required - depends on the case.I am considering to capture the filename of the files picked up and write it to the application log on in the target ERP system.

Benefit for IT and LOBs

By looking at all of the specifics above what you need to consider while realizing this with PI, you might be thinking "what an effort to do such a simple thing". Compared to hard-coded scripts on top of PGP and SFTP - which can be triggered from ABAP - this looks very complicated. It is for the first time, as with many things you do for the first time. So what is the benefit?

On a technical level, there is never any file written on a (potentially less secure file share) neither in unprotected or protected format. Therefore such a file cannot be read, altered or used as a basis for an additional file placement. The sending application triggers PI to create and place a file on the business partner's side in an automated way. This approach leaves little options to read or forge the data.

This means in case of very sensitive and critical data (e.g. payroll) it is imperative to reduce the risk of data manipulation. Even if manipulation still would happen, it better have to be re-traceble when and who made the change. PI supports both of these requirements.

Next Steps

So whats next? I am thinking of documenting this step by step with screenshots. Please comment if you like to see that on SCN.