topic Re: XML encoding UTF-16 issue in Application Development and Automation Discussions

XML encoding UTF-16 issue

Former Member — Fri, 28 Feb 2014 13:37:46 GMT

Hello

I currently need to produce an XML file for a customer and one of the requirements is to have utf-16 encoding.

Using the XSLT_TOOL transaction, I have already set-up everything in my XML tree and file is correctly produced except for the encoding.

If I use XSTRING for the xml output, I get UTF-8.

If I use STRING for the XML output, I get ISO-8859-1 as encoding instead of UTF-16.

I am not able to get the UTF-16 encoding in my document.

Is there any configuration somewhere in SAP that must be done to enable this UTF-16 functionality.

My SAP system is on ECC6 EhP level 4.

All posts I have been checking are mentioning the reverse issue.

Thanks in advance for all your precious help.

Regards,

Parvez

Re: XML encoding UTF-16 issue

Former Member — Fri, 28 Feb 2014 14:44:20 GMT

Class CL_ABAP_CONV_X2X_CE can be used to change encoding of xstring. The input and output encodings are passed to create static method which then returns an object reference.

Re: XML encoding UTF-16 issue

Former Member — Fri, 28 Feb 2014 14:53:28 GMT

Hello Manish,

thanks for your help. Would you have some example to show me? I am having those methods

Re: XML encoding UTF-16 issue

Former Member — Fri, 28 Feb 2014 15:03:37 GMT

Hi,

I had the same problem and solved this way:

TYPE-POOLS: abap.

DATA: data_tab TYPE soli_tab WITH HEADER LINE.

CALL FUNCTION 'GUI_UPLOAD'

EXPORTING

filename = 'c:\Temp\id.xml'

filetype = 'ASC'

TABLES

data_tab = data_tab

EXCEPTIONS

file_open_error = 1

file_read_error = 2

no_batch = 3

gui_refuse_filetransfer = 4

invalid_type = 5

no_authority = 6

unknown_error = 7

bad_data_format = 8

header_not_allowed = 9

separator_not_allowed = 10

header_too_long = 11

unknown_dp_error = 12

access_denied = 13

dp_out_of_memory = 14

disk_full = 15

dp_timeout = 16

OTHERS = 17.

IF sy-subrc <> 0.

* MESSAGE ID sy-msgid TYPE sy-msgty NUMBER sy-msgno

* WITH sy-msgv1 sy-msgv2 sy-msgv3 sy-msgv4.

ENDIF.

LOOP AT data_tab.

REPLACE FIRST OCCURRENCE OF SUBSTRING 'utf-16' IN data_tab

WITH 'utf-8' IGNORING CASE.

IF sy-subrc = 0.

MODIFY data_tab.

EXIT.

ENDIF.

ENDLOOP.

DATA: sap_codepage TYPE cpcodepage,

codepage TYPE abap_encoding.

CALL FUNCTION 'SCP_CODEPAGE_BY_EXTERNAL_NAME'

EXPORTING

external_name = 'UTF-8'

IMPORTING

sap_codepage = sap_codepage

EXCEPTIONS

not_found = 1

OTHERS = 2.

IF sy-subrc <> 0.

MESSAGE ID sy-msgid TYPE sy-msgty NUMBER sy-msgno

WITH sy-msgv1 sy-msgv2 sy-msgv3 sy-msgv4.

ENDIF.

codepage = sap_codepage.

CALL FUNCTION 'GUI_DOWNLOAD'

EXPORTING

filename = 'c:\Temp\id_utf8.xml'

filetype = 'ASC'

codepage = codepage

write_bom = 'X'

TABLES

data_tab = data_tab

EXCEPTIONS

file_write_error = 1

no_batch = 2

gui_refuse_filetransfer = 3

invalid_type = 4

no_authority = 5

unknown_error = 6

header_not_allowed = 7

separator_not_allowed = 8

filesize_not_allowed = 9

header_too_long = 10

dp_error_create = 11

dp_error_send = 12

dp_error_write = 13

unknown_dp_error = 14

access_denied = 15

dp_out_of_memory = 16

disk_full = 17

dp_timeout = 18

file_not_found = 19

dataprovider_exception = 20

control_flush_error = 21

OTHERS = 22.

IF sy-subrc <> 0.

* MESSAGE ID SY-MSGID TYPE SY-MSGTY NUMBER SY-MSGNO

* WITH SY-MSGV1 SY-MSGV2 SY-MSGV3 SY-MSGV4.

ENDIF.

Marco

Re: XML encoding UTF-16 issue

Former Member — Fri, 28 Feb 2014 15:20:07 GMT

The example is present in class documentation.

EDIT: See this snippet.

DATA utf8 TYPE xstring. 
DATA utf16 TYPE xstring. 
utf16 = '4D006100720063006F00'. "Marco 
DATA lr_conv TYPE REF TO cl_abap_conv_x2x_ce. 
TRY . 
    CALL METHOD cl_abap_conv_x2x_ce=>create 
      EXPORTING 
        in_encoding  = '4103' 
        in_endian    = 'L' 
        out_encoding = 'UTF-8' 
        out_endian   = 'L' 
        input        = utf16 
      RECEIVING 
        conv         = lr_conv. 
    CALL METHOD lr_conv->convert_c( n = 5 ). 
    CALL METHOD lr_conv->get_out_buffer 
      RECEIVING 
        buffer = utf8. 
  CATCH cx_parameter_invalid_type . 
  CATCH cx_parameter_invalid_range . 
  CATCH cx_sy_codepage_converter_init . 
ENDTRY. 
 

Re: XML encoding UTF-16 issue

Former Member — Fri, 28 Feb 2014 16:24:58 GMT

I forgot: my pgm was made for converting UTF-16 -> UTF-8, to make a UTF-8 -> UTF16 conversion you have to call FM SCP_CODEPAGE_BY_EXTERNAL_NAME this way:

CALL FUNCTION 'SCP_CODEPAGE_BY_EXTERNAL_NAME'

EXPORTING

external_name = 'UTF-16LE' "little endian

IMPORTING

sap_codepage = sap_codepage

EXCEPTIONS

not_found = 1

OTHERS = 2.

IF sy-subrc <> 0.

MESSAGE ID sy-msgid TYPE sy-msgty NUMBER sy-msgno

WITH sy-msgv1 sy-msgv2 sy-msgv3 sy-msgv4.

ENDIF.

and, of course, make

REPLACE FIRST OCCURRENCE OF SUBSTRING 'utf-8' IN data_tab

WITH 'utf-16' IGNORING CASE.

Marco

Re: XML encoding UTF-16 issue

Former Member — Fri, 28 Feb 2014 17:53:35 GMT

Hi Parvez,

below is the implementation tested and approved.

DATA: converter TYPE REF TO cl_abap_conv_obj,

prc_bom_string TYPE xstring,

lv_str TYPE string,

lv_xstr TYPE xstring.

* String receiving XML in UTF-8

lv_str = 'XML here'.

* Instantiate class conversion passing the code Litle Endian (4103)

CREATE OBJECT converter

EXPORTING

outcode = '4103'

miss = 'S'

broken = 'M'

use_f1 = 'X'

EXCEPTIONS

invalid_codepage = 1

internal_error = 2.

CALL METHOD converter->convert

EXPORTING

inbuff = lv_str

inbufflg = 0

outbufflg = 0

IMPORTING

outbuff = lv_xstr

EXCEPTIONS

internal_error = 1

OTHERS = 2.

* Identification Litle Endian (UTF-16)

prc_bom_string = cl_abap_char_utilities=>byte_order_mark_little.

* Adding character to lv_xstr

CONCATENATE prc_bom_string p_xstring

INTO p_xstring IN BYTE MODE.

Best Regards.

Re: XML encoding UTF-16 issue

Former Member — Mon, 03 Mar 2014 07:07:27 GMT

Hello Marco,

thanks for your reply.

After following your instructions, I have been able to replace the encoding in the file.

In Notepad, the file opens correctly, but when I try to open it in IE I get a blank page. In another browser, I see my data, but without the XML tags.

Is this normal??

I have tried my original codes in another environment, which is more recent and overthere, my file is correctly created in UTF-16 and opens correctly in IE and without XML tag in other browsers.

I think that there must be some configuration that add this codepage/utf-16 encoding into my sap system.

Thanks

Parvez

Re: XML encoding UTF-16 issue

Former Member — Mon, 03 Mar 2014 07:08:42 GMT

Hello,

thanks for your reply.

Your example, take UTF-16 and converts to UTF-8. I need the reverse.

I tried playing woth you codes, but the file is not properly generated.

Thanks.

Parvez

Re: XML encoding UTF-16 issue

Former Member — Mon, 03 Mar 2014 07:19:36 GMT

Your xml may be starting with:

<?xml version="1.0" encoding="UTF-8">

When you change the text encoding to UTF-16, you also need to change the encoding in starting line to UTF-16, something like this using search/replace.

<?xml version="1.0" encoding="UTF-16">

Re: XML encoding UTF-16 issue

Former Member — Mon, 03 Mar 2014 07:52:57 GMT

Hello Manish,

replacing the whole line does not bring any success .

Thanks for your reply.

I am checking with the admin team for a comparison of the 2 environments to determine why there is this strange behaviour.

Regards,

Parvez

Re: XML encoding UTF-16 issue

Former Member — Mon, 03 Mar 2014 08:16:00 GMT

I have been able to find out that the codepage of the 2 systems are different. (2nd column below)

Does anyone know how to change the code page?

Re: XML encoding UTF-16 issue

Former Member — Mon, 03 Mar 2014 08:43:37 GMT

I am pretty sure it isn't necessary. Can you upload the xml that is showing as blank in web browser?

Re: XML encoding UTF-16 issue

Former Member — Mon, 03 Mar 2014 10:39:20 GMT

<?xml version="1.0" encoding="UTF-16"?>

Re: XML encoding UTF-16 issue

Former Member — Mon, 03 Mar 2014 10:45:43 GMT

If the content is in an xml file whose text is encoded in UTF-16, it would open properly in browser.

It means your file has an encoding other than UTF-16.

Check this attachment which was created using your text and UTF-16 encoding.

Re: XML encoding UTF-16 issue

Former Member — Mon, 03 Mar 2014 10:59:16 GMT

That's correct.

My file was not created using UTF-16, but ISO-8859-1 encoding. Replacing the encoding line is not enough to make the file valid.

As indicated above, my sap system does not have the appropriate code page as being default. That's why I am getting this situation.

I am currently waiting on the admin team's reply regarding this configuration and the client's reply to know if their system will accept utf-8 or iso-8859-1 files.

Thanks to eveyone for their help.

Parvez