XML encoding UTF-16 issue

[Former Member]

Hello

I currently need to produce an XML file for a customer and one of the requirements is to have utf-16 encoding.

Using the XSLT_TOOL transaction, I have already set-up everything in my XML tree and file is correctly produced except for the encoding.

If I use XSTRING for the xml output, I get UTF-8.

If I use STRING for the XML output, I get ISO-8859-1 as encoding instead of UTF-16.

I am not able to get the UTF-16 encoding in my document.

Is there any configuration somewhere in SAP that must be done to enable this UTF-16 functionality.

My SAP system is on ECC6 EhP level 4.

All posts I have been checking are mentioning the reverse issue.

Thanks in advance for all your precious help.

Regards,

Parvez

[Former Member]

Class CL_ABAP_CONV_X2X_CE can be used to change encoding of xstring. The input and output encodings are passed to create static method which then returns an object reference.

[Former Member]

Hello Manish,

thanks for your help. Would you have some example to show me? I am having those methods

[Former Member]

The example is present in class documentation.

EDIT: See this snippet.

1. DATA utf8 TYPE xstring.
2. DATA utf16 TYPE xstring.
3. utf16 = '4D006100720063006F00'. "Marco
4. DATA lr_conv TYPE REF TO cl_abap_conv_x2x_ce.
5. TRY .
6.     CALL METHOD cl_abap_conv_x2x_ce=>create
7.       EXPORTING
8.         in_encoding  = '4103'
9.         in_endian    = 'L'
10.         out_encoding = 'UTF-8'
11.         out_endian   = 'L'
12.         input        = utf16
13.       RECEIVING
14.         conv         = lr_conv.
15.     CALL METHOD lr_conv->convert_c( n = 5 ).
16.     CALL METHOD lr_conv->get_out_buffer
17.       RECEIVING
18.         buffer = utf8.
19.   CATCH cx_parameter_invalid_type .
20.   CATCH cx_parameter_invalid_range .
21.   CATCH cx_sy_codepage_converter_init .
22. ENDTRY.

/.

[Former Member]

Hi,

I had the same problem and solved this way:

TYPE-POOLS: abap.

DATA: data_tab TYPE soli_tab WITH HEADER LINE.

CALL FUNCTION 'GUI_UPLOAD'

   EXPORTING

     filename                = 'c:\Temp\id.xml'

     filetype                = 'ASC'

   TABLES

     data_tab                = data_tab

   EXCEPTIONS

     file_open_error         = 1

     file_read_error         = 2

     no_batch                = 3

     gui_refuse_filetransfer = 4

     invalid_type            = 5

     no_authority            = 6

     unknown_error           = 7

     bad_data_format         = 8

     header_not_allowed      = 9

     separator_not_allowed   = 10

     header_too_long         = 11

     unknown_dp_error        = 12

     access_denied           = 13

     dp_out_of_memory        = 14

     disk_full               = 15

     dp_timeout              = 16

     OTHERS                  = 17.

IF sy-subrc <> 0.

*   MESSAGE ID sy-msgid TYPE sy-msgty NUMBER sy-msgno

*           WITH sy-msgv1 sy-msgv2 sy-msgv3 sy-msgv4.

ENDIF.

LOOP AT data_tab.

   REPLACE FIRST OCCURRENCE OF SUBSTRING 'utf-16' IN data_tab

           WITH 'utf-8' IGNORING CASE.

   IF sy-subrc = 0.

     MODIFY data_tab.

     EXIT.

   ENDIF.

ENDLOOP.

DATA: sap_codepage TYPE cpcodepage,

       codepage TYPE  abap_encoding.

CALL FUNCTION 'SCP_CODEPAGE_BY_EXTERNAL_NAME'

   EXPORTING

     external_name = 'UTF-8'

   IMPORTING

     sap_codepage  = sap_codepage

   EXCEPTIONS

     not_found     = 1

     OTHERS        = 2.

IF sy-subrc <> 0.

   MESSAGE ID sy-msgid TYPE sy-msgty NUMBER sy-msgno

           WITH sy-msgv1 sy-msgv2 sy-msgv3 sy-msgv4.

ENDIF.

codepage = sap_codepage.

CALL FUNCTION 'GUI_DOWNLOAD'

   EXPORTING

     filename                = 'c:\Temp\id_utf8.xml'

     filetype                = 'ASC'

     codepage                = codepage

     write_bom               = 'X'

   TABLES

     data_tab                = data_tab

   EXCEPTIONS

     file_write_error        = 1

     no_batch                = 2

     gui_refuse_filetransfer = 3

     invalid_type            = 4

     no_authority            = 5

     unknown_error           = 6

     header_not_allowed      = 7

     separator_not_allowed   = 8

     filesize_not_allowed    = 9

     header_too_long         = 10

     dp_error_create         = 11

     dp_error_send           = 12

     dp_error_write          = 13

     unknown_dp_error        = 14

     access_denied           = 15

     dp_out_of_memory        = 16

     disk_full               = 17

     dp_timeout              = 18

     file_not_found          = 19

     dataprovider_exception  = 20

     control_flush_error     = 21

     OTHERS                  = 22.

IF sy-subrc <> 0.

* MESSAGE ID SY-MSGID TYPE SY-MSGTY NUMBER SY-MSGNO

*         WITH SY-MSGV1 SY-MSGV2 SY-MSGV3 SY-MSGV4.

ENDIF.

Marco

[Former Member]

I forgot: my pgm was made for converting UTF-16 -> UTF-8, to make a UTF-8 -> UTF16 conversion you have to call FM SCP_CODEPAGE_BY_EXTERNAL_NAME this way:

CALL FUNCTION 'SCP_CODEPAGE_BY_EXTERNAL_NAME'

   EXPORTING

     external_name = 'UTF-16LE' "little endian

   IMPORTING

     sap_codepage  = sap_codepage

   EXCEPTIONS

     not_found     = 1

     OTHERS        = 2.

IF sy-subrc <> 0.

   MESSAGE ID sy-msgid TYPE sy-msgty NUMBER sy-msgno

           WITH sy-msgv1 sy-msgv2 sy-msgv3 sy-msgv4.

ENDIF.

and, of course, make

   REPLACE FIRST OCCURRENCE OF SUBSTRING 'utf-8' IN data_tab

           WITH 'utf-16' IGNORING CASE.

Marco

[Former Member]

Hello Marco,

thanks for your reply.

After following your instructions, I have been able to replace the  encoding in the file.

In Notepad, the file opens correctly, but when I try to open it in IE I get a blank page. In another browser, I see my data, but without the XML tags.

Is this normal??

I have tried my original codes in another environment, which is more recent and overthere, my file is correctly created in UTF-16 and opens correctly in IE and without XML tag in other browsers.

I think that there must be some configuration that add this codepage/utf-16 encoding into my sap system.

Thanks

Parvez

[Former Member]

Your xml may be starting with:

<?xml version="1.0" encoding="UTF-8">

When you change the text encoding to UTF-16, you also need to change the encoding in starting line to UTF-16, something like this using search/replace.

<?xml version="1.0" encoding="UTF-16">

[Former Member]

Hello Manish,

replacing the whole line does not bring any success .

Thanks for your reply.

I am checking with the admin team for a comparison of the 2 environments to determine why there is this strange behaviour.

Regards,

Parvez

[Former Member]

Hi Parvez,

below is the implementation tested and approved.

DATA: converter      TYPE REF TO cl_abap_conv_obj,

         prc_bom_string TYPE xstring,

         lv_str TYPE string,

         lv_xstr TYPE xstring.

* String receiving XML in UTF-8

  lv_str = 'XML here'.

* Instantiate class conversion passing the code Litle Endian (4103)

   CREATE OBJECT converter

     EXPORTING

       outcode          = '4103'

       miss             = 'S'

       broken           = 'M'

       use_f1           = 'X'

     EXCEPTIONS

       invalid_codepage = 1

       internal_error   = 2.

   CALL METHOD converter->convert

     EXPORTING

       inbuff         = lv_str

       inbufflg       = 0

       outbufflg      = 0

     IMPORTING

       outbuff        = lv_xstr

     EXCEPTIONS

       internal_error = 1

       OTHERS         = 2.

* Identification Litle Endian (UTF-16)

   prc_bom_string = cl_abap_char_utilities=>byte_order_mark_little.

* Adding character to lv_xstr

   CONCATENATE prc_bom_string p_xstring

         INTO p_xstring IN BYTE MODE.

Best Regards.

[Former Member]

Hello,

thanks for your reply.

Your example, take UTF-16 and converts to UTF-8. I need the reverse.

I tried playing woth you codes, but the file is not properly generated.

Thanks.

Parvez

[Former Member]

I have been able to find out that the codepage of the 2 systems are different. (2nd column below)

Does anyone know how to change the code page?

[Former Member]

I am pretty sure it isn't necessary. Can you upload the xml that is showing as blank in web browser?

[Former Member]

<?xml version="1.0" encoding="UTF-16"?>

<TEST><MANDT/><VBELN>0001</VBELN><ERDAT>0000-00-00</ERDAT><ERZET>00:00:00</ERZET><ERNAM/><ANGDT>0000-00-00</ANGDT><BNDDT>0000-00-00</BNDDT><AUDAT>0000-00-00</AUDAT><VBTYP/><TRVOG/><AUART/><AUGRU/><GWLDT>0000-00-00</GWLDT><SUBMI/><LIFSK/><FAKSK/><NETWR>0.0</NETWR><WAERK/><VKORG/><VTWEG/><SPART/><VKGRP/><VKBUR/><GSBER/><GSKST/><GUEBG>0000-00-00</GUEBG><GUEEN>0000-00-00</GUEEN><KNUMV/><VDATU>0000-00-00</VDATU><VPRGR/><AUTLF/><VBKLA/><VBKLT/><KALSM/><VSBED/><FKARA/><AWAHR>000</AWAHR><KTEXT/><BSTNK/><BSARK/><BSTDK>0000-00-00</BSTDK><BSTZD/><IHREZ/><BNAME/><TELF1/><MAHZA>0</MAHZA><MAHDT>0000-00-00</MAHDT><KUNNR/><KOSTL/><STAFO/><STWAE/><AEDAT>0000-00-00</AEDAT><KVGR1/><KVGR2/><KVGR3/><KVGR4/><KVGR5/><KNUMA/><KOKRS/><PS_PSP_PNR>00000000</PS_PSP_PNR><KURST/><KKBER/><KNKLI/><GRUPP/><SBGRP/><CTLPC/><CMWAE/><CMFRE>0000-00-00</CMFRE><CMNUP>0000-00-00</CMNUP><CMNGV>0000-00-00</CMNGV><AMTBL>0.0</AMTBL><HITYP_PR/><ABRVW/><ABDIS/><VGBEL/><OBJNR/><BUKRS_VF/><TAXK1/><TAXK2/><TAXK3/><TAXK4/><TAXK5/><TAXK6/><TAXK7/><TAXK8/><TAXK9/><XBLNR/><ZUONR/><VGTYP/><KALSM_CH/><AGRZR>00</AGRZR><AUFNR/><QMNUM/><VBELN_GRP/><SCHEME_GRP/><ABRUF_PART/><ABHOD>0000-00-00</ABHOD><ABHOV>00:00:00</ABHOV><ABHOB>00:00:00</ABHOB><RPLNR/><VZEIT>00:00:00</VZEIT><STCEG_L/><LANDTX/><XEGDR/><ENQUEUE_GRP/><DAT_FZAU>0000-00-00</DAT_FZAU><FMBDAT>0000-00-00</FMBDAT><VSNMR_V/><HANDLE/><PROLI/><CONT_DG/><CRM_GUID/><UPD_TMSTMP>0.0</UPD_TMSTMP><MSR_ID/><TM_CTRL_KEY/><SWENR/><SMENR/><PHASE/><MTLAUR/><STAGE>0</STAGE><HB_CONT_REASON/><HB_EXPDATE>0000-00-00</HB_EXPDATE><HB_RESDATE>0000-00-00</HB_RESDATE><LOGSYSB/><KALCD/><MULTI/></TEST>                                                                                                                                                         

[Former Member]

If the content is in an xml file whose text is encoded in UTF-16, it would open properly in browser.

It means your file has an encoding other than UTF-16.

Check this attachment which was created using your text and UTF-16 encoding.

[Former Member]

That's correct.

My file was not created using UTF-16, but ISO-8859-1 encoding. Replacing the encoding line is not enough to make the file valid.

As indicated above, my sap system does not have the appropriate code page as being default. That's why I am getting this situation.

I am currently waiting on the admin team's reply regarding this configuration and the client's reply to know if their system will accept utf-8 or iso-8859-1 files.

Thanks to eveyone for their help.

Parvez