SFTP duplicate file checking

[#1]

Dear Experts,

         I have done the development  SFTP->PI->File(FTP) and its working fine, But I don't want to process the duplicate file.

Issue in Detail:

   Every day our client will placing the file in SFTP, one file per day and no need to process the duplicate file. So that iam checking the duplicate file in channel configuration. So everyday we need to pick the latest file from SFTP using PI.

  PI is picking the latest file and updating in FTP and after 2hrs PI is again picking some of the random duplicate file from SFTP folder and updating in FTP and I increased the modification check in SFTP channel configuration, but the problem is still persists.

Kindly give some solution to resolve this issue if any body face this.

Best Regards,

Monikandan.

[Former Member]

I used this feature and it definetly works.

As far as I know, it remembers every for each file the size, last modified timestamp and filename.

This values are saved for 2 weeks in standard, which can be modified by setting the advanced parameter duplicateCheckPersist (in minutes)

After this time, it will read the old files again!

Of course, if someone touches a file and changes the changedate, it will be read again.

And you should change your processing sequence to Date.

By the way, the "modification check" will not change the duplicate behaviour. It's used to prevent reading unfinished files.

It just reads a file once, waits your given seconds, and then check again if the file has been changed. If you have many files in a directory, a high value will cause a lot of waiting time...

Also check SFTP-Adapter FAQ OSS-Note 1692819

[monikandan_p]

Thanks for your reply Heiko,

I already check with date and time stamp but again it picking the duplicate file and client side they are not modifying any files from SFTP folder.

Kindly guide to resolve.

Best Regards,

Monikandan.

[Former Member]

Actually, it doesn't matter if you set those advanced parameters. That's just for logging in your case.

But have you tried to set duplicateCheckPersist to a larger number, like 525600 (which should be one year)?

Of course, it will some files again in the first step - because it forgot that they've been read - but after that it shouldn't happen for one year.