Question Re: File encoding in File Content Conversion in Technology Q&A

File encoding in File Content Conversion

Former Member — Fri, 13 Mar 2015 22:57:13 GMT

Hi Guys,

Can somebody please help me.

Basically, this is the first time I am working with this single and double byte Japanese chars and not sure how to produce the required output.

I have an IDOC --> PI ---> FCC scenario and have to produce the output flat file in Shift-JIS encoding.

I have tried the encoding Shift-JIS in Receiver Channel but when I open the output file in Notepad++ or Notepad it is showing me encoding ANSI UTF-8 and not Shift-JIS.

Can somebody please suggest how to achieve this and also let me know how to handle single and double byte Japanese chars and do I need to define the field type as byte in Data Type and in FCC?

Single byte : ウ

Double byte : 市

Can I use MessageTransform and TextCodepageConversion Bean in File Adapter to achieve this?

Please suggest.

Thanks,

Asif

Re: File encoding in File Content Conversion

RaghuVamseedhar — Sat, 14 Mar 2015 02:12:03 GMT

Asif,

Japanese characters can be encoded using UTF-8 (takes 3 bytes) or UTF-16 (takes 2 bytes). UTF-8

Encoding can be changed using below options.

Option 1: - You can use modules to change encoding.

Work with character encoding in SAP PI. Section 4, page 5.

Option 2: - XSLT mapping.

How to create XSLT Mapping in SAP PI / PO

Option 3: - Java mapping.

Handling Code page, Character encoding in SAP PI / PO

How to create Java Mapping in SAP PI / PO

Re: File encoding in File Content Conversion

Former Member — Sat, 14 Mar 2015 03:42:30 GMT

Hi Raghu,

Thanks for your reply.

I have checked the sap encoding document but I think I cannot use that if I am using File Content Conversion until I use the MessageTranformBean and provide all the content conversion detail over there.

Could you please tell me difference between File Encoding and Char encoding because in the Receiver File Adapter we only have file encoding parameter and if I used that and provide Shift-JIS then the output is not showing correct chars but PI log is showing that it is converted into Shift-JIS.

Receiver Adapter log.

Output

I have not checked but if I use the XSLT mapping after the message mapping then in File encoding I have to provide the Binary.

Have you worked on fieldFixedLengthType : byte? When do we use that option?

Regards,

Asif

Re: File encoding in File Content Conversion

RaghuVamseedhar — Sun, 15 Mar 2015 01:22:38 GMT

Asif,

All unicode characters (including Japanese) can be encoded using UTF-8 (or UTF-16). Please use UTF-8 encoding, unless target system insists on using some other encoding.

If target system insists, they want file content to be encoded in Shift-JIS (please try Cp943, as it is supper set. Link).

Please use one of the 3 modules mentioned in Work with character encoding in SAP PI.

Re: File encoding in File Content Conversion

Former Member — Sun, 15 Mar 2015 05:54:22 GMT

Hi Raghu,

Thanks for your reply.

Unfortunately, target system only reads the Shift-JIS file.

My only confusion is that do I need to use File encoding or char encoding because as per my understanding File encoding only works for XML case and not for Flat files so I have to use MessageTransformBean and then define all the FCC parameters.

Can you please confirm if the difference I have highlight above is correct or is there any other difference?

Hope to hear from you soon.

Regards,

Asif

Re: File encoding in File Content Conversion

Former Member — Sun, 15 Mar 2015 10:48:33 GMT

Hi Asif,

You can use File Encoding parameter for text files as mentioned in the below document

Work with character encoding in SAP PI


In the receiver file adapter you can apply the codepage to which the documents are
converted. This is only useful if you convert the content to a text or CSV file. If you want
to change the codepage for XML you have to be aware that the codepage in the file
receiver channel setting does not influence the XML declaration. Use the adapter module
XmlAnonymizerBean instead (refer to chapter 4.3) or use your own XSLT or Java
mapping program.

As per below sap note encoding for Shift-JIS you can mention as MS932.

1883836 - Japanese character becomes garbled when converting from/to Shift-JIS(SJIS) encoding

Or you can use TextCodepageConversionBean module.

Regards,

Praveen.

Re: File encoding in File Content Conversion

Former Member — Sun, 15 Mar 2015 11:55:58 GMT

Many Thanks Praveen.

I have tried as you have mentioned but no difference in the output and actually the CodepageConversion Bean is not doing anything because the document is not plain.

Raghu @ I have tried Cp943 but still the same output as above.

Any other suggestion guys.

Re: File encoding in File Content Conversion

RaghuVamseedhar — Sun, 15 Mar 2015 14:40:34 GMT

Asif,

Please try

Option 1:- Remove FCC in receiver channel. Use MessageTransformBean with content conversion. Use encoding MS932 or Shift-JIS in MessageTranformBean.

Option 2:- Please add XSLT mapping with MS932 or Shift-JIS after message mapping. This option worked for me in NWDS.


<?xml version="1.0" encoding="UTF-8"?>
<xsl:stylesheet version="1.0" xmlns:xsl="http://www.w3.org/1999/XSL/Transform">
  <xsl:output encoding="MS932" />
  <xsl:template match="/">
  <xsl:copy-of select="." />
  </xsl:template>
</xsl:stylesheet>

Re: File encoding in File Content Conversion

Former Member — Mon, 16 Mar 2015 09:12:29 GMT

Hi Asif,

May be your payload content type is application/xml TextCodepageConversionBean only works for content type text/plain or text/html.

Note that TextCodepageConversionBean module only converts the payload of the text MIME types such as (text/html, text/plain, etc) that does not carry the encoding information in its content. In other words, this module takes the text content as a sequence of characters in some encoding and converts it into an equivalent sequence of characters in the specified encoding.

please refer below note.

960663 - XI 3.0 Adapter Framework Text Codepage Conversion Module

Only File Encoding option works for me without the TextCodepageConversionBean module.

Audit log:

Receiver File:

Regards,

Praveen.

Re: File encoding in File Content Conversion

Former Member — Mon, 16 Mar 2015 10:32:36 GMT

Hi Praveen/Raghu,

I have solved the issue by using MessageTransformBean and use charset in Content Type and it works for me and I am getting the expected output.

Thanks for all your help guys. I really appreciate.

Re: File encoding in File Content Conversion

Former Member — Mon, 16 Mar 2015 10:38:18 GMT

Hi Asif,

I am glad to hear that issue is resolved, please close the thread as mentioned below blog.

Regards,

Praveen.