SAP CDC Dataflow - Lookup Component Handling Multiple Source Fields & Data Type Mismatch

[Rvanjari]

Hi All , 

I am working with Dataflow to import accounts from CDC - SFTP to Gigya. Before importing, we perform a lookup to check if an account with the given email and a custom data field (stored as a string in CDC) already exists. However, when we receive the SFTP file, this data field contains a numeric value, causing the lookup to fail as it does not find a matching account. To address this, I first convert the numeric value to a string before performing the check. However, this approach does not always work—sometimes, even when the field type appears as a string, it is stored as a numeric value in CDC. How can I handle this scenario effectively?

[Oleh_Ilchyshyn1]

Hi @Rvanjari,

Quick clarification before to answer:
1) Are you reading .csv or .json file?

[Rvanjari]

Hi , Its .csv file

[Oleh_Ilchyshyn1 · Accepted solution]

Hi @Rvanjari,
Based on your comment below, the solution is the following:
In component file.parse.dsv, you need to uncheck the inferTypes parameter (set to false).

Because while it is set to true, the CDC will attempt to parse string values into Boolean, any Number type, and so on. If you don't need it, please uncheck this.
You can find the description of that parameter here:
https://help.sap.com/docs/SAP_CUSTOMER_DATA_CLOUD/8b8d6fffe113457094a17701f63e3d6a/414ce1d070b21014b...

Hope it will resolve your lookup issue.
If not, you can provide an example of your .csv file and the custom field you keep in CDC for lookup, and I will provide another solution.

Regards,
Oleh.

[Rvanjari · Accepted solution]

Thanks , i think this should work , thanks for the solution