cancel
Showing results for 
Search instead for 
Did you mean: 

Strange processing behavior while uploading CSV file using Hot Folder import

former_member611299
Participant
0 Kudos

I followed the instruction from the below wiki post

https://wiki.hybris.com/display/forum/hot+folder+error+file

and extended AbstractImpexRunnerTask and override the processFile method which sends an email on error during CSV processing. I am trying to upload a Category CSV file with an error. Below CSV has additional ';' in the first line.

 bs-lorealbrandcasting123;L'Oreal Casting Creme;;;L'Oreal Casting Creme;;
 bs-lorealbrandabsolue;L'Oreal Mousse Absolue;L'Oreal Mousse Absolue;;

Impex Header

 <bean id="batchMicroSiteCategoryConverter" class="de.hybris.platform.acceleratorservices.dataimport.batch.converter.impl.DefaultImpexConverter">
         <property name="header">
             <value>#{defaultImpexCategoryHeader}
                 #Insert Categories
                 INSERT_UPDATE MicrositeCategory;code[unique=true];name[lang=$lang];description[lang=$lang];allowedPrincipals(uid)[default='customergroup'];$catalogVersion                  
             </value> 
         </property>
         <property name="impexRow">
             <value>;{+0};{1};{2};;;</value>  
         </property>
     </bean>


Q.1 - Uploading this file generates an error on line 1 and inserts the second category but it does not call my class which extends AbstractImpexRunnerTask.

 <int:service-activator input-channel="batchFilesTranHeader" output-channel="batchFilesImp" ref="batchHeaderRunnerTask"
         method="execute" />
     <bean id="batchHeaderRunnerTask" class="com.superdrug.core.dataimport.batch.task.AbstractCSVUploadImpexRunnerTask">
         <lookup-method name="getImportService" bean="importService" />  
         <lookup-method name="getImportConfig" bean="importConfig" />
         <property name="hostUrl" value="#{hybrisProperties['website.hybris.http']}" /> 
     </bean>    


Q2 - If I reverse the two lines in CSV file as below there are no errors generated and it does call my class which extends AbstractImpexRunnerTask and inserts both the categories.

 bs-lorealbrandabsolue;L'Oreal Mousse Absolue;L'Oreal Mousse Absolue;;
 bs-lorealbrandcasting123;L'Oreal Casting Creme;;;L'Oreal Casting Creme;;


This is very strange behavior and I am not able to understand what is causing this.. If anyone can throw some light on this it will be really helpful.

Accepted Solutions (1)

Accepted Solutions (1)

amitpanwar
Advisor
Advisor
0 Kudos

put the values of your CSV in double quotes. It should work fine then.

former_member611299
Participant
0 Kudos

Thanks Amit,

I tried uploading the both the files with the double quoted and its working fine. Its calling the extended processFile method from AbstractCSVUploadImpexRunnerTask as well.

 **Sample file** 

 "bs-lorealbrandcasting123";"L'Oreal Casting Creme";;;"L'Oreal Casting Creme";;
 "bs-lorealbrandabsolue";"L'Oreal Mousse Absolue";"L'Oreal Mousse Absolue";;

But the problem is that these files will be send by third party vendor in the same format without double quotes. Is there a way to convert the CSV files in double quotes in Hybris before processing the import ?

Answers (6)

Answers (6)

raghavendra_desu
Employee
Employee
0 Kudos

I am not sure, weather this is the right way or not, but its my suggestion. Why can't we write our own impex converter and map it through the spring.xml, and in our custom impex converter we can add double quotes before and after of a column value which will resolves this problem.

can you please try it once.

former_member611299
Participant
0 Kudos
 #Impex for Case 2
 INSERT_UPDATE MicrositeCategory;code[unique=true];name[lang=$lang];description[lang=$lang];allowedPrincipals(uid)[default='customergroup'];$catalogVersion
                             
 ;bs-lorealbrandabsolue;L'Oreal Mousse Absolue;L'Oreal Mousse Absolue;;;;;;;
 ;bs-lorealbrandcasting123;L'Oreal Casting Creme;;;L'Oreal Casting Creme;;;;;;;

Both the lines are processed and inserted successfully

 #Impex for Case 1
 ;bs-lorealbrandcasting123;L'Oreal Casting Creme;;;L'Oreal Casting Creme;;;;;;;
 ;bs-lorealbrandabsolue;L'Oreal Mousse Absolue;L'Oreal Mousse Absolue;;;;;;;

line 1 throws the error

Former Member
0 Kudos

Check the impex file generated in processing folder in second case. Does it contain two lines? As per logs only one value line got run [ processed: 1, no lines dumped ]. If the impex contains single value line, the problem can be in converter.

Former Member
0 Kudos

Maybe no CR after last line? Try to add an empty line ";;;;;;" at the end.

former_member611299
Participant
0 Kudos

I am not able to reply to your message with the below text. So I am adding a new msg..

This is the error message I am getting for the first scenario:

ERROR [task-scheduler-6] (00000CCI) [CronJobErrorHandler] de.hybris.platform.impex.jalo.ImpExException: Can not resolve any more lines ... Aborting further passes (at pass 2). Finally could not import 1 lines![HY-123]
ERROR [task-scheduler-6] [DefaultImportService] Import has caused an error, see logs of cronjob with code=00000CCI for further details

Below is the message in unresolved_lines.csv for the first case

INSERT_UPDATE MicrositeCategory;code[unique=true];name[lang=en];description[lang=en];allowedPrincipals(uid)[default='customergroup'];catalogversion(catalog(id[default=superdrugProductCatalog]),version[default='Staged'])[unique=true,default=superdrugProductCatalog:Staged]
    ,,error finding existing item : column='catalogversion' value='L'Oreal Casting Creme', , column 5: cannot resolve value 'L'Oreal Casting Creme' for attribute 'catalogversion';bs-lorealbrandcasting123;L'Oreal Casting Creme;;;L'Oreal Casting Creme;;;;;;;

For the second case both lines are processed successfully with the below log message

INFO  [task-scheduler-5] (00000CCJ) [ImpExImportJob] Starting ImpEx cronjob "ImpEx-Import"
INFO  [task-scheduler-5] (00000CCJ) [Importer] Finished 1 pass in 0d 00h:00m:00s:010ms - processed: 1, no lines dumped (last pass 0)
INFO  [task-scheduler-5] [DefaultImportService] Import was successful (using cronjob with code=00000CCJ)
amitpanwar
Advisor
Advisor
0 Kudos

Could you post the error logs?

former_member611299
Participant
0 Kudos

Thanks Amit, Please check the below message for the error details. I am not able to submit it here as there is character limit.

Former Member
0 Kudos

Can you post the error message you get?

former_member611299
Participant
0 Kudos

Thanks Konrad, Please check the below message for the error details. I am not able to submit it here as there is character limit.