Load multiple csv files using sap data intelligence

[#1]

Hi Team

we are trying to load mulitple csv files with same format present in azure container to snowflake using sap DI.

Structure file consumer operator doesn't allow wild card character *.csv

Please suggest the workflow and operators with which we can achieve this requirement.

Thanks,

Shubha

[andrea_emtemark]

Hi,

Similar topic, I'm trying to get several CSV files into one table (and save in a HANA DB) but they should use the same Column header/name. the data transformer is stopping me since its duplicates of the title/column name.

Also I added a column called "source", and I want add "source 1" and "source 2" to every row to help identify which source its come from.

This is how I built the pipe, it worked before to write several CSV files into one table on HANA.

This is Gen1, but i also tired Gen2...

[former_member609638]

hi ankit

Thanks for the inputs.

Yes, tried using list and read file operators we are able to read the files but unable to load the file to target HANA table.

Could you please share your complete graph snippet ?

[ankitsharma_71]

Hi Shubha,

Below is a graph snippet to load data into the HANA table:

You have to mention the table name and columns in the Hana client operator.

Also this sample graph is available under scenario templates in Modeler.

Thanks,

Ankit

[ankitsharma_71]

Hi Shubha,

Below is one of the method that I have used to read multiple csv files with same format:

1. Use List File operator in the Graph, mention connection Id and Path of the file (Here don't mention the filename). There is one "Filter" Option in operator, in that select "With Regular expression" and enter filename like this - "Filecsv_".

2. Connect the output of ListFile with Input of Readfile operator and select Read(From Input) .

3. Execute the graph, and it will read all the files starting with name (Filecsv_) one by one.

Please let me know if you are able to find any other approach.

Thanks,

Ankit