- SAP Managed Tags:
This blog post describes how to configure the connection and processing settings of the FTP sender and receiver adapter. This adapter will be available for SAP Cloud Integration customers with the 04-July-2020 release. This blog post describes the basic concepts as well as the configuration options.
Introduction to the FTP Adapter
The File Transfer Protocol was initially specified in 1971 and although encryption capabilities have been added to the specification in the meantime, the FTP/FTPS protocol is rated less secure than the SFTP protocol. Therefore, we recommend you to select the SFTP protocol as your first choice when setting up a new scenario which SAP Process Integration has been supporting with a dedicated SFTP Adapter for years. The new FTP Adapter should be used only for those scenarios in which you already have an existing FTP Server which you cannot connect to with SFTP.
In contrast to other protocols FTP uses two connections on two different ports between the client and the server, the control connection and the data connection. The connection mode specifies how the data connection is established. There are two different modes: active mode and passive mode. SAP Cloud Platform's FTP adapter only supports passive mode.
FTP uses a control connection and a data connection
In the passive mode, the FTP adapter (which is always the client) sends a EPSV/PASV command to the server in order to receive the server's data connection port. This port is within a port range specified on the server. The FTP server administrator should be able to lookup this range which will be required once you want to configure SAP Cloud Connector. For details regarding the Cloud Connector configuration see section "Cloud Connector Configuration" below.
The data transferred over the network can be of different types. The FTP adapter supports Binary and by default the FTP Adapter uses streaming.
The authentication is done either by username/password or by anonymous login. For both options the Integration Developer maintains a credential name in the adapter. The credential itself has to be created in the Security Material monitor as a User Credential artifact. This User Credential artifact contains the username and the password. For anonymous login use username = "anonymous" and an empty password. But this might deviate based on the requirements of your FTP server.
Transport Security can be achieved by setting an encryption parameter. If you choose "Plain FTP - no encryption" all transmission will be done in clear text. The default port for the control connection is 21. Two options are available to encrypt the communication:
- Explicit FTPS: After the initial connection the client sends a “AUTH TLS” command to the server which initiates a TLS handshake (the FTP server must support TLS). The subsequent communication is encrypted. The default port for the control connection is 21.
- Implicit FTPS: The connection to the FTP server is established as a TLS connection. In this case, usually a different port for the control connection is used, default 990.
Once you have selected an FTPS encryption, the server certificate needs to be trusted by SAP Cloud Integration. Ensure that you have uploaded the intermediate and root certificate to the SAP Cloud Integration keystore. Please check Manage Keystore Entries for how to upload certificates to the keystore. One option to get the server certificate and its chain is to execute the FTP Connectivity Test and download the files from the test result.
The FTP adapter performs a host verification which ensures that the control and data connection share the same host. You cannot have two different hosts.
The FTP adapter does not support TLS session re-use.
FTP Sender Adapter: Setup & Examples
The FTP sender adapter can be used to read files from an FTP server. It is a polling adapter which means that it is scheduled to look for files once or periodically, according to a predefined pattern.
Find detailed information about all possible parameters in the Configure the FTP Sender Adapter documentation.
The following screenshots demonstrate a sample Integration Flow that reads all .txt-files from a /sender directory using explicit encryption and connecting via SAP Cloud Connector.
The Source tab contains general connection and file access parameters.
FTP Sender - Source Tab
The Processing tab contains more advanced processing parameters. For example, the Read Lock Strategy ensures that the FTP adapter only polls the files if it has an exclusive read-lock on the file. Or the Post-Processing option which defines what should happen to the file after it has been processed successfully.
FTP Sender - Processing Tab
The Schedule tab allows you to specify when and how often each of your runtime nodes should perform a poll attempt.
FTP Sender - Schedule Tab
FTP Receiver Adapter: Setup & Examples
The FTP receiver adapter can be used to write files to an FTP server. Either you use the FTP Receiver adapter together with an End Message step or a Send step.
Find detailed information about all possible parameters in the Configure the FTP Receiver Adapter documentation.
The following screenshots demonstrate a sample Integration Flow that writes a file.txt into directory /sendstep using explicit encryption and connecting via SAP Cloud Connector.
The Target tab contains general connection parameters as well as the directory and file name to be written.
FTP Receiver - Target Tab
The Processing tab contains more advanced processing parameters. For example, the Handling for Existing Files is set to Append in the sample which will add the Integration Flow's payload to the end of an existing file.
FTP Receiver - Processing Tab
Dynamic Setting of FTP Receiver Adapter Parameters
The FTP receiver adapter supports a wide range of dynamic configuration possibilities. The dynamic configuration can be used, for example, once you require to connect to different FTP servers from the same Integration Flow.
Text-based fields can contain dynamic input. You can configure the entry fields Directory, File Name, Address, Location ID and Credential Name dynamically by using a header (${header.abc}) or a property (${property.abc}) as shown below.
FTP Receiver - Dynamic Text Fields
As shown in the screenshot above, the drop-down field Proxy Type has the option Dynamic.
If you choose Dynamic, you have to define the actual value in the respective SAP property.
Attribute |SAP property |Type |Values ----------------------------|-----------------------|-------------------|--------------------------------------- Proxy Type |SAP_FtpProxyType |String |‘internet’ and ‘onPremise’
If the property is not defined during runtime, an error is thrown.
Checkboxes, additional drop-down lists and integer fields are configurable dynamically if you define the values in predefined SAP properties. The following table shows the names of the properties for the different configuration options:
Attribute |SAP property |Type |Values ----------------------------|-----------------------|-------------------|--------------------------------------- Timeout |SAP_FtpTimeout |int |Values of type integer Max. Reconnect Attempts |SAP_FtpMaxReconnect |int |Values of type integer Reconnect Delay |SAP_FtpMaxReconDelay |int |Values of type integer Automatically Disconnect |SAP_FtpDisconnect |boolean, string |‘true’, ‘false’ Change Directories Stepwise |SAP_FtpStepwise |boolean, string |‘true’, ‘false’ Create Directories |SAP_FtpCreateDir |boolean, string |‘true’, ‘false’ Handling for Existing Files |SAP_FtpAfterProc |String |‘Overwrite’, ‘Append’, ‘Fail’, ‘Ignore’ Flatten Filenames |SAP_FtpFlattenFileName |boolean, string |‘true’, ‘false’
If a property is not set, the runtime uses the value defined in the channel.
FTP and SAP Cloud Connector Configuration
Most scenarios using the FTP adapter use the SAP Cloud Connector to establish the connection to corporate FTP servers. This allows you to connect via TCP to your on-premise FTP server, so no incoming ports have to be opened in your firewall which is often not allowed by your security policy. Refer to the Cloud Connector documentation for general setup details. The FTP adapter's on-premise connectivity utilizes the SOCKS5 proxy supported in SAP Cloud Connector version 2.10 and higher.
Note: If you have not already installed SAP Cloud Connector and connected it to your SAP Cloud platform account, with which you subscribed to SAP Cloud Integration, download SAP Cloud Connector from SAP Development Tools page and follow its installation documentation.
The following information covers FTP adapter specifics only. If you are using encryption (FTPS), implicit or explicit, you must configure the Virtual Host equal to the Internal Host. This is required for a hostname verification to be successful which ensures that the server certificate is issued for the hostname it connects to. Otherwise the Integration Flow fails during runtime.
A sample configuration is shown in the following screenshot.
Usually, the Cloud Connector setup for a single server requires just a single entry in the "Cloud To On-Premise -> ACCESS CONTROL" screen. For the FTP connectivity, one entry for the control connection and one entry per port within the data connection port range is required. As mentioned above the FTP server administrator should provide this port range. As of now, the Cloud Connector does not support maintenance of ranges. Therefore, we require one entry per port which might require a lot of maintenance effort.
To mitigate this effort, SAP Cloud Connector offers two ways which are more convenient for mass creation of Access Control entries than creating them manually on the Administration page. The two options are:
- SAP Cloud Connector API to create the Access Control entries
- Import Feature for Access Control entries
The SAP Cloud Connector API to create the Access Control entries allows you to create a single entry with a HTTP POST request. So you would require some kind of automation for sending those POST request for a large number of ports. A single sample request looks like this, be aware to replace all masked values ("< >"). Refer to the API documentation for details.
curl --request POST \
--url https://<SCC_host_and_port>/api/v1/configuration/subaccounts/<region_url>/<subaccount>/systemMappings \
--header 'authorization: Basic <base64_username_password>' \
--header 'content-type: application/json' \
--header 'x-csrf-token: <cvrf_token>' \
--data '{
"virtualHost": "ftp-server.sample",
"virtualPort": "55001",
"localHost": "ftp-server.sample",
"localPort": "55001",
"protocol": "TCP",
"backendType": "nonSAPsys"
}'
The other option is the import functionality on the administration screen for Access Control which accelerates the creation of a lot of entries. For this option, you would have to prepare one json file containing a backend entity per port. If you download one of your entries created manually, you get a template to fill out. A sample for possible content is shown below:
{
"backends": [{
"cloudHost": "ftp-server.sample",
"cloudPort": "55001",
"localHost": "ftp-server.sample",
"localPort": "55001",
"protocol": "TCP",
"backendType": "nonSAPsys",
"hostInHeader": "virtual",
"authMode": "NONE",
"resources": []
}, {
"cloudHost": "ftp-server.sample",
"cloudPort": "55000",
"localHost": "ftp-server.sample",
"localPort": "55000",
"protocol": "TCP",
"backendType": "nonSAPsys",
"hostInHeader": "virtual",
"authMode": "NONE",
"resources": []
}, ...
]
}
Once you established the connection successfully, be aware that the SAP Cloud Connector interrupts the connection after a timeout. We recommend to set the Automatically Disconnect option inside the FTP Adapter to true for connections using Proxy Type On-Premise.
Troubleshooting
For any connectivity issue I recommend checking the FTP Connectivity test first. The connectivity test will show you meaningful error messages. In addition it offers you the possibility to download the server certificate or to check your user permissions to the FTP server's folders. Please check out the documentation for FTP Connectivity Tests.
FTP Connectivity Test - Error Message
The above screenshot is an example for an error message. This one is indicating that there is a Access Control entry missing for ftp-server.sample:21. The SAP Cloud Connector offers comprehensive logging as well, and this message is visible as well in the Audits screen with an entry: "Denying access to system ftp-server.sample:21". In this case adding the required access control entry would fix the issue.
Conclusion
I hope this blog post helps you to understand the basic concepts of FTP/FTPS and enables you to smoothly configure the SAP Cloud Integration FTP Adapter. Besides certain limitations mentioned in the text, examples are shown for sender and receiver, I also described the dynamic configuration features as well as the SAP Cloud Connector configuration.
12 Comments
