If duplicate rows are coming from source, then you can use Select distinct option in Query transform and if duplicates are already there in target. You can delete them in target db first and use table comparison (row by row) and put target table to compare with source table.
If a table has a unique id, than place that in a compare column and P-Keys in PK column in the transform editor, if no unique id place all the columns in compare columns and p-keys in pk column. Ensure you are using Row by Row method.
Now when the source is flat file, then you can do select distinct and order by Primary keys in Query transform and attach it to the final target table and ensure you are making it a permanent table and Auto correct load option is on and Merge and Upsert is on as well in the target table.
Further to this, Feel free to ask any clarifications that you may have.
Regards,
Vibhor
You must be a registered user to add a comment. If you've already registered, sign in. Otherwise, register and sign in.