Question Re: Character Converting Problem at File Adapter in Technology Q&A

Character Converting Problem at File Adapter

Former Member — Wed, 24 Feb 2010 20:17:46 GMT

I am having a character conversion problem bringing in a XML file into XI.

It is converting the data wrong.

I have a file adapter picking up Audit.xml encoding is utf-8, use binary mode to pick it up.

XI takes the file and converts it into a text file.

I compared the hex of Audit.xml source file and the Audit.txt file and the characters are different for a specific few.

See attached image for details

Please let me know if any additional information will help

http://i9.photobucket.com/albums/a68/tkc204/SAP%20XI/Audit.png

Re: Character Converting Problem at File Adapter

sunilchandra007 — Wed, 24 Feb 2010 20:33:57 GMT

Hi Chirag,

Try with File Type as Text at both sender and receiver file adapter.

Regards,

Sunil Chandra

Re: Character Converting Problem at File Adapter

Former Member — Wed, 24 Feb 2010 20:48:29 GMT

Hi Sunil,

Thanks for the response.I have tried that. I still have the same problem I have tried the following

Text - UTF-8 to Text UTF-8

Binary to Text UTF-8

UTF-8 to Binary

Re: Character Converting Problem at File Adapter

sunilchandra007 — Wed, 24 Feb 2010 20:58:59 GMT

Might be the specific character is not available in UTF-8. Try File Encoding with ISO-8859-1 or UTF-16 .

Regards,

Sunil Chandra

Re: Character Converting Problem at File Adapter

Former Member — Wed, 24 Feb 2010 21:19:47 GMT

I will try that, but if the XML header has

<?xml version="1.0" encoding="UTF-8" ?>

doesnt it override your xi setting?

Re: Character Converting Problem at File Adapter

sunilchandra007 — Wed, 24 Feb 2010 21:38:18 GMT

Check section 16 for file encoding https://service.sap.com/sap/support/notes/821267 . Also have a look at 3.1 of http://www.sdn.sap.com/irj/scn/go/portal/prtroot/docs/library/uuid/502991a2-45d9-2910-d99f-8aba5d79fb42?quicklink=index&overridelayout=true .

Regards,

Sunil Chandra

Re: Character Converting Problem at File Adapter

Former Member — Wed, 24 Feb 2010 23:00:23 GMT

Hey Thanks, Ive actually have reviewed those before. The file the source system is sending to xi is also not opening up in Internet Explorer.

The source system is 1252 and outputs xml to utf-8

I think there is an issue between those two.

But since the hex is changing in XI, the source system won't look at the data.

Re: Character Converting Problem at File Adapter

Former Member — Thu, 25 Feb 2010 04:28:00 GMT

HI Chirag,

Please post the Audit.txt and Audit.xml data where you feel it's getting changed.So,that we can have a look and try to solve it.

Regards,

Nutan

Re: Character Converting Problem at File Adapter

Former Member — Thu, 25 Feb 2010 06:09:42 GMT

Hi Chirag,

Try like this,

Keep the sender mode as binary

and receiver channel as TEXT with Windows-1252 encoding

Regards

Suraj

Re: Character Converting Problem at File Adapter

Former Member — Thu, 25 Feb 2010 07:06:23 GMT

Appears that did the trick.

The data is appearing the same as the source. I need to do additional testting to verify that the issues is resolved.

Thanks

Re: Character Converting Problem at File Adapter

stefan_grube — Thu, 25 Feb 2010 07:59:21 GMT

> and receiver channel as TEXT with Windows-1252 encoding

First ask which codepage is expected by the receiver.

You cannot just put any codepage you like

Re: Character Converting Problem at File Adapter

Former Member — Thu, 25 Feb 2010 10:51:36 GMT

Then there is something wrong with the input XML document. It often happens that external applications send with declared XML header encoding UTF-8, but in reality it is ISO-8859-1. The ugly thing is, most of the time the problem will not be recognized, because most of the standard characters are encoded the same in UTF-8 and ISO-8859-1. But have a look at the ascii chars > 127, there you will see the problem.

You will note that problem if you just open the input file e.g. in Internet Explorer: if it displays some garbage for special characters or if it cannot open the file at all, then the sender uses the wrong encoding declaration.

So instead of tuning the XI parameters, better request the sender to send a correct XML

CSY

Re: Character Converting Problem at File Adapter

stefan_grube — Thu, 25 Feb 2010 12:06:32 GMT

> Then there is something wrong with the input XML document. It often happens that external applications send with declared XML header encoding UTF-8, but in reality it is ISO-8859-1.

You are right, that happens often. But here the input file is correct..

As you see in hex code, the characters are valid UTF-8 characters.

S.R.Suraj has already provided a solution for the issue, so I think this is solved.

Re: Character Converting Problem at File Adapter

Former Member — Thu, 25 Feb 2010 13:12:31 GMT

The Source team claimed it was UTF-8. But after rigorous investigation it wasn't.

Re: Character Converting Problem at File Adapter

Former Member — Thu, 25 Feb 2010 13:22:36 GMT

Actually for this case Christian is right the source xml that I am receiving wasn't valid UTF-8. The program generating the xml was just putting the encoding - utf-8 tag on top because that's what the code told it to do.

The actual data is Windows-1252 in xml. So when XI reads it, it uses the utf-8 tag on top and is expecting UTF-8, but it was getting something else causing the unsupported characters to get converted.

The HEX was different on the Source XML comparing it to what XI was generating after File Adapter.

After making the output change to Windows 1252. It changes back to the expect HEX.

Source - is at Windows 1252

XI - it will use UTF-8 (no way to change it)

Target - forced to create in Windows 1252

Other helpful tools I used: W3C Validator - upload the file and found that I wasn't receiving UTF-8

I hope this is clear and helps someone else in the future.

Thanks For everyone's input.

Chirag