Underlying Algorithm for Scoring in Fuzzy Search

[deodutt_dwivedi]

Hi Gurus,

It would be of great help if someone can tell me the underlying algorithm being used for Fuzzy Search Scoring in HANA. I need to understand this in order to do dry runs on test data. I tried using TF/IDF but was unable to infer and calculate the scoring results. Additionally i went through the HANA Search developer guide and other documents as well but unfortunately could not find anything.

Regards,

Deo

[deodutt_dwivedi]

Any pointer on different similarCalculationModes and their algorithmic difference is welcomed.

Regards,

Deo

[Piet]

Hi Deodutt,

I have the same question, were you able to find the answer?
I also would like to understand how exactly the score is calculated in order to make an educated decision.

Thanks,

Piet

[deodutt_dwivedi]

Hi Piet,

No, i did not receive any answer on the same. I compared the results in HANA with implementing algorithms present in python-levinstein library but there is no exact match. So probably, HANA is using some propriety algorithm for Fuzzy Search. Outside HANA, Jaro or Jaro-Winkler can be used to achieve similar results with some tweaks but definitely not the exact same result set.

Meanwhile, you can check the weights, similarCalculationModes and other features in HANA Search Developer Guide which can impact the scoring based on your data set.

Regards,

Deo

[Piet]

Hi Deodutt, thanks for your answer. In the end I performed some black box testing using different cases and different values voor the similaritycalculationmode. Not ideal, but I guess we have no other choice, since the algorithms in deed seem to be proprietary. Regards, Piet