topic Re: Unicode conversion vs. UTF8 in Application Development and Automation Discussions

Unicode conversion vs. UTF8

Former Member — Thu, 18 Feb 2010 14:28:08 GMT

I currently run a dual stack ECC system with no languages installed (other than default EN/DE).

SQL> select value from v$nls_parameters

2 where parameter = 'NLS_CHARACTERSET';

VALUE

WE8DEC

Is there any reason I need to change the characterset to UTF8?

Can I change to UTF8 without doing a full blown SAP unicode conversion? and Why - if ever - would I have to perform a unicode conversion?

Re: Unicode conversion vs. UTF8

Former Member — Thu, 18 Feb 2010 14:38:06 GMT

I currently run a dual stack ECC system with no languages installed (other than default EN/DE).

SQL> select value from v$nls_parameters

2 where parameter = 'NLS_CHARACTERSET';

VALUE

WE8DEC

Is there any reason I need to change the characterset to UTF8?

Can I change to UTF8 without doing a full blown SAP unicode conversion? and Why - if ever - would I have to perform a unicode conversion?

Further info:

SQL> select property_value from database_properties

2 where upper(property_name) = 'NLS_NCHAR_CHARACTERSET';

PROPERTY_VALUE

UTF8

SQL> select value from v$nls_parameters

2 where upper(parameter) = 'NLS_NCHAR_CHARACTERSET';

VALUE

UTF8

SQL> select value from v$nls_parameters

2 where parameter = 'NLS_CHARACTERSET';

VALUE

WE8DEC

Does the above mean I never have to do a unicode conversion? Under which sceanrio would I have to?

Re: Unicode conversion vs. UTF8

nils_buerckel — Thu, 18 Feb 2010 16:00:03 GMT

Hi,

I would not recommend to simply change the database encoding to Unicode - this does not work.

You will need to have an SAP Unicode Kernel as well.

In order to have a proper Unicode system, you will have to convert all char like data from Non-Unicode to

Unicode. This can be done with Import / Export procedere.

Please have a look at SAP notes 1322715 (FAQs) and 1051576 (docu, I assume you have a single code page system).

Best regards,

Nils Buerckel

SAP AG

Re: Unicode conversion vs. UTF8

Former Member — Thu, 18 Feb 2010 16:05:09 GMT

Hi Nils,

Thanks for responding. I am still unsure why I have to convert (export) to Unicode. Is SAP changing the ABAP dictionary data types during this process? Is that the reason?

The data of the SAP J2EE application and the data of the ABAP SAP application are separated, not only by a separate database schema, but also by the use of different data types. So it is possible to operate a Unicode-based SAP J2EE application (with a UTF8 character set) and a non-Unicode-based SAP application (with a WE8DEC character set) on the same database at the same time, without having to install a Unicode-only database.

So I am not sure what a Unicode only database will get me...

Re: Unicode conversion vs. UTF8

Former Member — Thu, 18 Feb 2010 19:57:11 GMT

To convert a non-Unicode system to Unicode, all character data in the non-Unicode database must be converted to Unicode. The default conversion method is to export the entire database using SAPinst, create a new Unicode database (system copy), and then import the database using SAPinst again. The actual data conversion to Unicode is done during the export.

Normal character set is defined by the parameter NLS_CHARACTERSET

u2022 NLS_CHARACTERSET value contains the database character set name which determines the set of characters that can be stored in the database.

u2022 Normal character set is used for data stored in CHAR, VARCHAR2, CLOB columns.

National character set is defined by the parameter NLS_NCHAR_CHARACTERSET

u2022 National character set is used for data stored in NCHAR, NVARCHAR2 and NCLOB columns

Note: The national character set is the character set which is defined in oracle database in addition to normal character set

So by ALTER DATABASE NATIONAL CHARACTER SET UTF8 but not doing a unicode conversion I am unsure what that is getting me and still do not understand why I have to do a full conversion.

Re: Unicode conversion vs. UTF8

nils_buerckel — Fri, 19 Feb 2010 10:29:04 GMT

Hi Sandeep,

two different questions here:

1) Is it necessary to use an ABAP Unicode system in case of a dual stack system ?

SAP notes 73606 and 975768 should answer this one.

Excerpts:

3. Unicode is the mandatory system type:

b) for SAP systems which deploy Java applications (for example J2EE

applications, WebDynpro applications)

c) for SAP ABAP systems which communicate with Java components (for

example via the SAP Java Connector)

"As reliability of connection between Java and non-Unicode backend cannot

be ensured, usage of Java features with non-Unicode backend has been

deprecated."

Therefore technically it may still work, but SAP support is not guaranteed.

2) Why Unicode conversion ?

The Non-Unicode code page used (probably 1100 in your case) is different from utf-8 (or more precisely cesu-8) used by Oracle databases in case of SAP systems. Therefore the Non-Unicode data needs to be converted to Unicode (via Export / Import).

Best regards,

Nils Buerckel

SAP AG

Edited by: Nils Buerckel on Feb 19, 2010 11:30 AM

Re: Unicode conversion vs. UTF8

Former Member — Fri, 19 Feb 2010 13:23:02 GMT

Thanks for your input Nils.

While we do have webdypro development and a dual stack I believe the java part is covered by the below:

NLS_NCHAR_CHARACTERSET which is UTF8.

As for the ABAP it is still WE8DEC but since we are english language only - so any data entered via java/jco etc will be english - we cant justify converting to unicode.

...again unless i am missing something...

Re: Unicode conversion vs. UTF8

nils_buerckel — Fri, 19 Feb 2010 15:49:22 GMT

Hi Sandeep,

I can just say that SAP strongly recommends to make use of Unicode in your case.

Otherwise I see the risk that SAP support does not help in case of I18N problems.

By the way - the Euro or trademark symbol is only supported in Unicode - not sure whether you include these characters in English-only ...

Best regards,

Nils Buerckel

SAP AG