topic Re: ABAP HTTP body Chinese characters garbled code in Application Development and Automation Discussions

ABAP HTTP body Chinese characters garbled code

former_member526351 — Fri, 20 Oct 2017 07:52:46 GMT

Hi all experts,

I need to upload a file by HTTP Post with content type multipart/form-data. All is well, but if the file name is chinese, it will become garbled code.

The screenshot below is the main code how to set the file name

I used Tool Wireshark to catch the HTTP stream, in the Wireshark you can find the file name become garbled code ‘###’

The follow the code

FORM frm_upload_file  USING pv_token TYPE string

                             pv_path TYPE string

                             pv_fname TYPE string

                   CHANGING  cv_flg TYPE c

                             cv_msg TYPE string.

  DATA: lv_error_msg TYPE string,

        lv_url  TYPE string,

        lv_body TYPE string,

        lv_asjson TYPE xstring,

        lv_str TYPE string.



  DATA: lo_http_client TYPE REF TO cl_http_client,

        lo_http_request TYPE REF TO cl_http_request,

        li_http_client  TYPE REF TO if_http_client.



  DATA: lv_filename     TYPE string,

        lv_filelength   TYPE i,        "上次文件长度

        lv_header       TYPE xstring,

        lv_length       TYPE i.        "转成xstring后的长度



  TYPES: BEGIN OF lty_xml,

          c(400) TYPE x,

         END OF lty_xml.



  DATA: lt_xml TYPE TABLE OF lty_xml,

        lv_xml_target TYPE xstring.



  DATA: lv_code TYPE string,

        lv_response_msg TYPE string.



  cl_gui_frontend_services=>gui_upload(

    EXPORTING

      filename                = pv_fname

      filetype                = 'BIN'

  IMPORTING

      filelength              = lv_filelength

      header                  = lv_header

    CHANGING

      data_tab                = lt_xml

   EXCEPTIONS

      file_open_error         = 1

      file_read_error         = 2

      no_batch                = 3

      gui_refuse_filetransfer = 4

      invalid_type            = 5

      no_authority            = 6

      unknown_error           = 7

      bad_data_format         = 8

      header_not_allowed      = 9

      separator_not_allowed   = 10

      header_too_long         = 11

      unknown_dp_error        = 12

      access_denied           = 13

      dp_out_of_memory        = 14

      disk_full               = 15

      dp_timeout              = 16

      not_supported_by_gui    = 17

      error_no_gui            = 18

         ).

  IF sy-subrc <> 0.

    cv_flg = 'E'.

    cv_msg = '上传文件失败'.

    RETURN.

  ENDIF.



  CALL FUNCTION 'SCMS_BINARY_TO_XSTRING'

    EXPORTING

      input_length = lv_filelength

    IMPORTING

      buffer       = lv_xml_target

    TABLES

      binary_tab   = lt_xml.





  lv_length = xstrlen( lv_xml_target ).







  CONCATENATE 'http://' gv_ip '/fs/v1/upload/' INTO lv_url.

  CONCATENATE lv_url '?access_token=' pv_token  INTO lv_url.





  CALL METHOD cl_http_client=>create_by_url

    EXPORTING

      url                = lv_url

    IMPORTING

      client             = li_http_client

    EXCEPTIONS

      argument_not_found = 1

      plugin_not_active  = 2

      internal_error     = 3

      OTHERS             = 4.

  IF sy-subrc <> 0.

    cv_flg = 'E'.

    cv_msg = '创建url失败'.

    RETURN.

  ENDIF.



*set http method POST

  CALL METHOD li_http_client->request->set_method(

    if_http_request=>co_request_method_post ).





*set protocol version

  li_http_client->request->set_version(

   if_http_request=>co_protocol_version_1_1 ).



*content type

  CALL METHOD li_http_client->request->set_content_type

    EXPORTING

      content_type = 'multipart/form-data'.



  CLEAR lv_str.



  CONCATENATE 'access_token=' pv_token INTO lv_str.

  CALL METHOD li_http_client->request->if_http_entity~set_header_field

    EXPORTING

      name  = 'cookie'

      value = lv_str.



  CALL METHOD li_http_client->request->if_http_entity~set_header_field

    EXPORTING

      name  = 'Accept'

      value = 'text/html, image/gif, image/jpeg, *; q=.2, */*; q=.2'.



  CALL METHOD li_http_client->request->if_http_entity~set_header_field

    EXPORTING

      name  = 'Content-Type'

      value = 'multipart/form-data;charset=utf-8'.



**form  body

  DATA: it_formulario TYPE tihttpnvp,

        wa_formulario LIKE LINE OF it_formulario,

        part TYPE REF TO if_http_entity.



  "path

  part = li_http_client->request->if_http_entity~add_multipart( ).

  CALL METHOD part->set_header_field

    EXPORTING

      name  = 'content-disposition'

      value = 'form-data;name="path"'.



  CALL METHOD part->append_cdata

    EXPORTING

      data = pv_path.



  "filelen

  part = li_http_client->request->if_http_entity~add_multipart( ).

  CALL METHOD part->set_header_field

    EXPORTING

      name  = 'content-disposition'

      value = 'form-data;name="filelen"'.



  lv_str = lv_length.

  CONDENSE lv_str NO-GAPS.

  CALL METHOD part->append_cdata

    EXPORTING

      data = lv_str.

  .



  "filename

  CALL FUNCTION 'SO_SPLIT_FILE_AND_PATH'

    EXPORTING

      full_name     = pv_fname

    IMPORTING

      stripped_name = lv_filename

*     FILE_PATH     =

    EXCEPTIONS

      x_error       = 1

      OTHERS        = 2.

  IF sy-subrc <> 0.

    RETURN.

* Implement suitable error handling here

  ENDIF.



  CLEAR lv_str.

  CONCATENATE 'form-data;Content-Type:application/octet-stream;charset=utf-8;name="uploadfile"; filename="' lv_filename '"' INTO lv_str.



  part = li_http_client->request->if_http_entity~add_multipart( ).

  CALL METHOD part->set_header_field

    EXPORTING

      name  = 'content-disposition'

      value = lv_str.

*      value = lv_xstr.



  CALL METHOD part->append_data

    EXPORTING

      data = lv_xml_target.



  CALL METHOD part->set_content_type

    EXPORTING

      content_type = 'text/html; charset=utf-8'.



  CALL METHOD li_http_client->send

    EXPORTING

      timeout                    = 200

    EXCEPTIONS

      http_communication_failure = 1

      http_invalid_state         = 2

      http_processing_failed     = 3

      OTHERS                     = 4.

  IF sy-subrc NE 0.

    li_http_client->get_last_error( IMPORTING message = lv_error_msg ).

    cv_flg = 'E'.

    cv_msg = lv_error_msg.

    RETURN.

  ENDIF.



* Receive the Response Object

  li_http_client->receive( EXCEPTIONS http_communication_failure = 1

                                   http_invalid_state         = 2

                                   http_processing_failed     = 3 ).

  IF sy-subrc <> 0 .

    li_http_client->get_last_error( IMPORTING message = lv_error_msg ).

    cv_flg = 'E'.

    cv_msg = lv_error_msg.

    RETURN.

  ENDIF.



  lv_body =  li_http_client->response->get_cdata( ).

  li_http_client->close( ).



  CLEAR lv_asjson.

  PERFORM frm_json_names_to_upper USING lv_body lv_asjson.



  CALL TRANSFORMATION id SOURCE XML lv_asjson RESULT code = lv_code message = lv_response_msg.



  IF lv_code = '0'.

    cv_flg = 'S'.

    cv_msg = '上传成功'.

  ELSE.

    cv_flg = 'E'.

    cv_msg = lv_code && ':' && lv_response_msg.

  ENDIF.

Re: ABAP HTTP body Chinese characters garbled code

iftah_peretz — Sat, 13 Jan 2018 13:32:57 GMT

Hi Guozheng,

As it's a big hassle to build a mockup of your problem (even impossible as I don't know the whole process and business need) all I can offer are 3 suggestions:

Try to use some other type of encoding for the code page (e.g. utf-16).
Try changing the Hex representation of the file name (the problem could be when uploading the file in the gui_upload).
A workaround - for example, map your file name into some unique numerator. The mapping shold be accessible on the receiving end (like a common DB table, XML attribute you can successfully transmit with the file etc.) and pass that numerator in the HTTP request and decode it back to its name on the receiving end.

Re: ABAP HTTP body Chinese characters garbled code

Former Member — Sat, 13 Jan 2018 17:00:27 GMT

I think, method set_header_field is not smart enough to convert your 3? character chinese filename into UTF-8.

part = li_http_client->request->if_http_entity~add_multipart( ).
  CALL METHOD part->set_header_field
    EXPORTING
      name  = 'content-disposition'
      value = lv_str.
*      value = lv_xstr

Therefore it is shown as ### in Whireshark (looks like a UTF-16 into plain ASCII conversion).

Try to convert lv_filename into UTF-8, before you add it as header field.

CONCATENATE 'form-data;Content-Type:application/octet-stream;charset=utf-8;name="uploadfile"; filename="' lv_filename '"' INTO lv_str.

Re: ABAP HTTP body Chinese characters garbled code

Sandra_Rossi — Sat, 13 Jan 2018 20:18:53 GMT

Difficult to find the solution, but this answer can be used as a starting point: https://stackoverflow.com/a/35573100/9150270

HTTP encoding is explained in RFC 5987 (https://tools.ietf.org/html/rfc5987) : header fields may contain a value expressed in UTF-8 with URL escaping. Thus, 你好 (hello), which corresponds in UTF-8 to 6 bytes E4 BD A0 E5 A5 BD, which correspond to URL escaping %E4%BD%A0%E5%A5%BD, should be expressed:

parmname*=UTF-8''%E4%BD%A0%E5%A5%BD

Content-Disposition is explained in RFC 6266 (https://tools.ietf.org/html/rfc6266) : it says you may need to mention both filename and filename* if the receiver doesn't understand filename* (as it may sound impossible to translate a Chinese name into US-ASCII characters, choose any generic name).

Thus you should have (for hello):

Content-Disposition: form-data;Content-Type:application/octet-stream;charset=utf-8;name="uploadfile";filename*=UTF-8''%E4%BD%A0%E5%A5%BD;filename="file"

Re: ABAP HTTP body Chinese characters garbled code

Former Member — Sat, 13 Jan 2018 22:27:19 GMT

With this solution from Sandra, ";charset=utf-8" becomes useless, because it has no influence on the parameter encoding.

Content-Disposition:form-data;Content-Type:application/octet-stream;name="uploadfile";filename*=UTF-8''%E4%BD%A0%E5%A5%BD;filename="file"

Re: ABAP HTTP body Chinese characters garbled code

Sandra_Rossi — Sun, 14 Jan 2018 09:29:48 GMT

In fact, char=utf-8 applies to the "part" body (section of the multipart HTTP), so it should not be confused with the encoding of the file name.

The code of the OP sends an XSTRING variable into the "part" body, which seems to be an XML, but impossible to know the character encoding as it's read from a file. So, the charset is either unknown or UTF-8, who knows.

By the way, the content-type could also be text/xml...

Re: ABAP HTTP body Chinese characters garbled code

former_member526351 — Thu, 29 Mar 2018 02:44:09 GMT

Thank you for your answer.

I send a message to SAP Help Center, they said it was the problem of the third party vendor which can not process the Chinese characters. But I don't know why it will be OK if I use the Java code which was provided by the vendor.

Actually at last I contact our vendor who provided this HTTP server， they told me that I can set the filename in the head of the HTTP request also, then the problem was solved.

The following is the abap code

  "filename
  CALL FUNCTION 'SO_SPLIT_FILE_AND_PATH'
    EXPORTING
      full_name     = pv_fname
    IMPORTING
      stripped_name = lv_filename
*     FILE_PATH     =
    EXCEPTIONS
      x_error       = 1
      OTHERS        = 2.
  IF sy-subrc <> 0.
    RETURN.
* Implement suitable error handling here
  ENDIF.

  CALL METHOD cl_http_utility=>if_http_utility~escape_url
    EXPORTING
      unescaped = lv_filename
*     options   =
    RECEIVING
      escaped   = lv_filename.


  CALL METHOD li_http_client->request->if_http_entity~set_header_field
    EXPORTING
      name  = 'filename'
      value = lv_filename.

Re: ABAP HTTP body Chinese characters garbled code

Sandra_Rossi — Thu, 29 Mar 2018 06:23:03 GMT

Thanks. "filename" header field doesn't sound standard at all, but if the provider understands it, then it's fine.