Solr exact match search issue with material codes contain special characters

[former_member606401]

Hi Experts,

We have a requirement, If the User search with exact material number then User will be redirected to PDP page of that material.

Description: Hybris will check for exact match with the following filters :- -> Ignore Case (like upper case or lower case of the alphabets of the searching material number) -> Remove any special character, like hyphen ‘-‘, Underscore ‘_’, Pound ‘#’, Parentheses ‘(‘ and ‘)’, Forward slash ‘/’, Period ‘.’

Use cases : 1. Material codes can be either numbers or alphanumeric with or without special character. (e.g. FW35X600XL02, FW35-7002E3L65, ZF-HFW35C812XL80) 2. A Material code can be present as sub-string in another Material code. (e.g. FW35-8011, FW35-8011T1) 3. A Material code can be present as a Sub-String in Material Names.

Could you please provide suggestions on how to achieve this.

Thanks, Murali

[vinay_malempati]

Hi ,

If you want to do only with solr then you can try using adding three new fields in your solr properties one each for the attributes you want exact match and give field type as "sortabletext(KeywordTokenizer does no actual tokenizing, so the entire input string is preserved as a single token)" and give these fields the highest boost compared to the rest, and once you have search results your life simple just compare the sanitized input text with the required fields and if match it is Diwali :)

[radek_michalczyk]

Hi,

yes, an approach that you described regarding DB solution is what I meant. Regarding JOINs performance - this should not be a huge issue, this depends on data volume but at the end it is quite a standard operation for DB.

You may consider the creation of DB indexes for fields that you are using in your search.

Regarding solution in Solr, in the comment above described high level how field should be set in Solr - that would be special field used only for code, customerMaterialCode and customerSerialCode.

Then solution would be similar to one with DB but instead of DB you search in Solr in this field alone, when you find the answer you redirect to product detail page if not you perform standard Solr query.

To differentiate between query for exact code search and second query you could potentially used concept described in here: https://help.sap.com/viewer/9d346683b0084da2938be8a285c0c27a/1808/en-US/cac34797e1d0460c9966adbbb3be...

depend on context you may define search in different way.

However, there is one more point based on your answer. Am I getting this right that customerMaterialCode and customerSerialCode are different per each customer (potentially)? That may end up with a lot of data.... With this exact match does customer search for product code and: customerMaterialCode customerSerialCode that belongs to him?

example:

product A:

code AA

customerMaterialCode AA-1 for Customer 1 customerMaterialCode AA-2 for Customer 2

customerSerialCode AA-1 for Customer 1 customerSerialCode AA-2 for Customer 2

If I am Customer 2 I search for AA-1 do I find product above or not because values AA-1 are defined for Customer 1 only?

[former_member606401]

Hi , The complete scenario that we have is, we allow users to search globally on the basis of four fields : Product Code, CustomerMaterialNumber, CustomerSerailNumber, Product Name.

If the User input string matches exactly with any of the the three (Product Code, CustomerMaterialNumber, CustomerSerailNumber ), then we redirect user to PDP else have to show all search results on PLP.

Also, there is no customer specific codes restriction, customer can search for any product code, customerMaterialCode customerSerialCode, i.e. If I am Customer 2 and I search for AA-1 then I can find the product above.

Regarding solution in Solr, in the comment above described, in that case also we have to make query to solr two times, is it ? If so, then how would i tell solr first to make search query on that basis of fields defined as type of custom field type and if no results then make standard solr search query ? Can you please provide more details about this.

Also, in your very first comment, you have mentioned about a solr approach in which we require to do quite a lot of customization, can you please provide more details on that as well.

[former_member693079]

Hi , This may solved by adding custom field type at SOLR schema. Create custom field type in schema as below: 1. For case sensitive --> use LowercaseFilter 2. For removing special characters use WordDelimiterFilter with only concateAll option enabled. 3. Assign that field type to material codes.

Sample field type:

 </fieldType>

This will generated tokens by removing special characters & lowercased at index time & query time, to get it matched.

Hope this will help you.

Thanks,

[radek_michalczyk]

Hi,

interesting question but I can share few ideas but not sure which one would be the best. 1) It is a quite common request and in past projects we actually plugin our logic in Search Controller. Once user provides term we actually searched in DB not in Solr for this exact match. Only when there was no exact mach we did standard search in Solr.

Reasons for this approach: it is easy and fast to implement + usually perform well In your case the difference is that lower case and special character handling. For this one you could have interceptor created - everytime you save a product in DB you strip special characters and save this simplified version in additional field. This will slow down product creation and modification but will result in quite fast query to DB.

If you want to go with Solr solution I would consider two solutions:

2) make this field a facet - when indexing and when passing a facet - then you will have extact search

3) use text search functionality but over one special field (which means some customization, to perform differently search for exact match and for standard search if there is no exact match). This field in Solr would require different - with Tokenizer that will create just one token (for exact match) and some filters to remove special characters and do the lower case part.

That is high level answer if you will go with DB search (that would be my suggestion) then it should be sufficient. If you will decide with Solr approach I can provide you more details but you will need to do quite a lot of coding and research by yourself as this is more complex.

[former_member606401]

Hi , Thanks for your suggestion.

Actually apart from Product-Code, we have another two fields (customer specific material numbers and serial numbers, maintained as relation with Product ) similar to Product-Code. So as per the first approach, the DB one , this is what i'm going to do in order to achieve this :

1. Going to create one custom flexible search query , which will search for a exact match after comparing user input with the three fields(code, customerMaterialCode, customerSerialCode) by performing JOIN on corresponding schema for the same.

2. If search query return any result , then we'll redirect user to PDP, else let the Solr to perform Standard search.

Now my question is, will it have any performance effect as i'm going to create a flexible search query which will search for result by performing JOIN on three schema ?

Also, Can you please provide more details on the third approach.

Thanks, Murali