codepage on sap application server and database NLS parameters

[Former Member]

Hi all ,

I have done unicode conversion for one of my customer . As per SAP ,it uses UTF-16 as the internal encoding and for database it uses UTF-8 or UTF -16

In my system it has UTF-16 for SAp system and UTF-8 for oracle 10G database

My doubts are

1. why UTF-16 and UTF-8 for same sap system

2. whether this encoding is used only for RAM or memory storage or it is for

file system storage as well

3. whay all NLS parameters like NLS_LANG,NLS_CHARACTER is UTF8

If anybody has any link for this or any document please let me know

Thanks ,

Ganesh

[Former Member]

Hi all ,

I have done unicode conversion for one of my customer . As per SAP ,it uses UTF-16 as the internal encoding and for database it uses UTF-8 or UTF -16

In my system it has UTF-16 for SAp system and UTF-8 for oracle 10G database

My doubts are

1. why UTF-16 and UTF-8 for same sap system

2. whether this encoding is used only for RAM or memory storage or it is for

file system storage as well

3. whay all NLS parameters like NLS_LANG,NLS_CHARACTER is UTF8

If anybody has any link for this or any document please let me know

Thanks ,

Ganesh

[Former Member]

Hi!

You may have a look at following link:

https://www.sdn.sap.com/irj/sdn/go/portal/prtroot/docs/library/uuid/a0af361f-3b22-2a10-04aa-965d1e8f...

/cheers

[Former Member]

Hi Ganesh,

1.) The answer is disk space. If you compare the different databases, they need different amount of disk space for the same data. If you have a standard system with only the initial languages english and german, then a system, stored to a UTF-8 database, does not grow very much due to Unicode conversion. If you have lots of languages in the system or you run a asian language, then your system could grow quite a bit. If you choose UTF-16 (this means you run MS SQL Server or MaxDB), than all languages grow by a comparable rate but much more than a UTF-8 with initial language support. The exact numbers are very data dependent, but 30 percent could be easily achieved. Oracle is (afaik) the only database able to support both encoding. All others supported by SAP only support one of the encoding. Therefore SAP has to support both UTF-8 and UTF-16.

2.) In memory SAP decided (probably out of performance reasons) to use UCS-2 internally because this uses fixed charachter length. To me this seems to perform much better than any variable character width string handling. The conversion is done on the fly, probably by the database client, but I'm not sure.

3.) On Oracle you have to use for NLS_LANG in environment and you should choose the correct CHARACTER SET and NATIONAL CHARACTER SET in CREATE DATABASE statement. This should be normally executed by SAPinst which will set the correct values.

Kind regards

Ralph

[Former Member]

Hey Ralph ,

Thanks a lot for the answer however do you want to say now that

1. For whatever data is put into SAP buffers from database will use UTF-16 representation and whatever data on the disk ie sapdata folders will be stored as per UTF-8

2. Since oracle suppots both UTF-8 and UTF-16 what if I change the NLS parameters from UTF-8 to UTF-16

Is that possible

3. Also my same memory which has it's part for database as well as WAS of SAP will have two different character representation

Thanks in advance

-Ganesh

[Former Member]

Hi Ganesh,

to 1.) Yes, in memory a fixed length character representation is used and on disk a variable length. Therefore a conversion is done reading and writing character data from and to disk.

to 2.) No, SAP supports only UTF-8 on Oracle (afaik). Believe me, you don't want UTF-8 either. Your database with UTF-16 would be ~30% bigger than with UTF-8 and most UTF-16 characters would begin with a Nil byte.

to 3.) No, both use only one encoding. The database uses UTF-8 for both encoding (CHARACTER SET and NATIONAL CHARACTER SET) and SAP WAS uses UCS-2. All by SAP supported databases nowadays support databases with ASCII and Unicode character data in the same database even in a single table. In Oracle CHARACTER SET sets the encoding for data type CHAR and VARCHAR2 and NATIONAL CHARACTER SET set the encoding for NCHAR and NVARCHAR2 data.SAP used (and may be still use) NVARCHAR2 data type in Java schema and VARCHAR2 in ABAP schema. Because Java schema is always in Unicode, SAP configures an ASCII database to use WE8DEC in CHARACTER SET and UTF8 in NATIONAL CHARACTER SET. If you install a Unicode database both parameters are set to UTF8.

Regards

Ralph