Handling of Cyrillic values

[flukic]

Hi all,

we will soon start working with the Russian CDC instance (already have EU instance in place and Marketing Cloud).

Considering we would have integration with Marketing Cloud (our EU CDC instance is already integrated) we would need to somehow handle both latin and cyrillic values for some attributes.
Some possible workaround for this would be to create custom fields that should contain e.g. cyrillic value. For example have profile.firstName (default) contain latin value and data.cyrillic.firstName contain cyrillic value.

However, the biggest issue here is how to transliterate (cyrillic -> latin or vice versa). Especially on the fields where a user can input something on the form freely (not limited by e.g. dropdown).

Did anyone have a similar case and how did you approach this? Any advice would be helpful. Thanks!

[OlegBash]

What is the exact question? ? 🙂 how to deal with characters or how to pass via services?

1) to transliterate - it is just characters replacement. If you need a table of mapping - I will provide 🙂

2) to pass Cyrillic values via odata services - you should use js-function: encodeURI

values4odata = encodeURI(CyrillicValue.toString())

[flukic]

Hi Oleg,

I would say it is both really. The main question is how to transliterate and store both Cyrillic and Latin values based on the input form which is in Cyrillic in this case? Then later, when sending the data to Marketing Cloud - is some additional logic needed to send the Cyrillic values?

So I guess your answer covers both questions it seems.

So basically there would need to be some value handling in the IdentitySync dataflow to Marketing Cloud (your odata example)?

And I would be interested in some tips or table for the mapping scenario. Thank you very much!

[OlegBash]

Unfortunately, I do not know details about Marketing Cloud. And how to deal with the issue there.

As for general questions:

1) to pass the cyrillic symbol in correct way - you need to encode them.

In JavaScript it is done with function encodeURI( ).

in ABAP there are:
class CL_HTTP_UTILITY method IF_HTTP_UTILITY~DECODE_X_BASE64
class CL_ABAP_CONV_OUT_CE

I guess the same function there is almost in any framework.

2) one of the sample of using regex for transliteration is here

https://gist.github.com/diolavr/d2d50686cb5a472f5696

Mapping table for transliteration

You can save text as html and open it in e.g. GoogleChrome (i do not know how to paste readable table except picture and code)

<table >
            <thead style="text-align:center">
                <tr>
                    <th>Russian</th> <th>Latin</th>
                </tr>
            </thead>
            <tbody>
                <tr>
                    <td>A</td> <td>A</td>
                </tr>
                <tr>
                    <td>Б</td> <td>B</td>
                </tr>
                <tr>
                    <td>В</td> <td>V</td>
                </tr>
                <tr>
                    <td>Г</td> <td>G</td>
                </tr>
                <tr>
                    <td>Д</td> <td>D</td>
                </tr>
                <tr>
                    <td>Е</td> <td>Е</td>
                </tr>
                <tr>
                    <td>Ё</td> <td>YO</td>
                </tr>
                <tr>
                    <td>Ж</td> <td>ZH</td>
                </tr>
                <tr>
                    <td>З</td> <td>Z</td>
                </tr>
                <tr>
                    <td>И</td> <td>I</td>
                </tr>
                <tr>
                    <td>Й</td> <td>Y</td>
                </tr>
                <tr>
                    <td>К</td> <td>K</td>
                </tr>
                <tr>
                    <td>Л</td> <td>L</td>
                </tr>
                <tr>
                    <td>М</td> <td>M</td>
                </tr>
                <tr>
                    <td>Н</td> <td>N</td>
                </tr>
                <tr>
                    <td>О</td> <td>O</td>
                </tr>
                <tr>
                    <td>П</td> <td>P</td>
                </tr>
                    <tr>
                    <td>Р</td> <td>R</td>
                </tr>
                <tr>
                    <td>C</td> <td>S</td>
                </tr>
                <tr>
                    <td>T</td> <td>T</td>
                </tr>
                <tr>
                    <td>У</td> <td>U</td>
                </tr>
                <tr>
                    <td>Ф</td> <td>F</td>
                </tr>
                <tr>
                    <td>X</td> <td>KH</td>
                </tr>
                <tr>
                    <td>Ц</td> <td>TS</td>
                </tr>
                
                <tr>
                    <td>Ч</td> <td>CH</td>
                </tr>
                <tr>
                    <td>Ш</td> <td>SH</td>
                </tr>
                <tr>
                    <td>Щ</td> <td>SCH</td>
                </tr>
                <tr>
                    <td>Ъ</td> <td>-</td>
                </tr>
                <tr>
                    <td>Ы</td> <td>Y</td>
                </tr>
                <tr>
                    <td>Ь</td> <td>-</td>
                </tr>
                <tr>
                    <td>Э</td> <td>E</td>
                </tr>
                <tr>
                    <td>Ю</td> <td>YU</td>
                </tr>
		<tr>
                    <td>Я</td> <td>YA</td>
                </tr>
            </tbody>
        </table>
<br>

[former_member709813]

Hi Filip,

Typically, you will have to use the JS events in the screensets (ie: onFieldChange, onBeforeSubmit ) to validate the data in the inputs. To ensure that the inputs only accept the expected value.

You can use JS+Regex to ensure that data.firstNameCyrillic contains ONLY Cyrillic characters, If there's a different value, then show an error attached to the field showing a message like: "Latin characters are not supported in this field".

If you need a more secure validation for the fields, then you can use Extensions - but this will require you to host a server-side endpoint to verify the data.

Hope this helps.

[flukic]

Hi Victor,

thanks on the validation advice - this is definitely something that is needed.

My question was more in the direction of how to handle the situation where user can enter data in the form in Cyrillic and it is needed to store both Cyrillic and Latin value in CDC.

[former_member709813]

Hi flukic,
IF the consumer type in the input ONLY in Cyrillic and you want to store the input + the translation in CDC, then what you can do, is to use the onBeforeSubmit event of the screen-sets to translate it and add it to the formModel, so both - Cyrillic and Latin - will be stored. You will need to use a third-party service to make the translation as CDC does not have that capability.

Notice that, you could use any other screen-set event, and then use gigya.accounts.setAccountInfo to store the translation in the callback of the "translator-service" AJAX call.
Hope it helps.