Read UTF-16 (BOM=FFFE) into table without #

[Former Member]

Hello Forum,

the following issue is consuming up my time, perhaps one of you guys can give me a hint.

I have a text file, encoded in UTF-16 with a BOM of FFFE. I tried allready different solutions like posted . These solutions seem to work until a certain extent - open dataset (in text mode) displays the files after conversion with cl_abap_conv_in_ce but still the output looks like this:

#J#o#b# #'#s#t#a#t#i#s#t#i#c

The problem in my opinion is that the conversion does not correctly convert:

4A 00 to the capital letter "J" but to "J#" which means he does not know what to do with the '00'.

Has anybody a hint for me? I have searched the OSS quite a while, as well as the forum and tried out a lot by mself now, but no solution...

Thanks,

Dennis Hoofe

[Former Member]

Hi,

Is your file separated by tab or space ? Because when you upload a file separated by tab or space, you will get # in between two fields. Here, # represents horizontal tab. Now, in your cse, 4A is getting converted properly in J and next # represents the empty space in between 4A and 6F.

Reward points if the answer is helpful.

Regards,

Mukul

[Former Member]

Hi Dennis,

Try using Encoding DEFAULT in Open Dataset statement.

Ashvender

[Former Member]

Hi Ashvender,

I am opening the file at the moment the following way (after the previous mentioned solution):

  
OPEN DATASET pa_file IN TEXT MODE ENCODING default FOR INPUT.
  DO.
    READ DATASET PA_FILE INTO text.
    IF SY-SUBRC <> 0.
      EXIT.                                   "End of file
    ENDIF.
 write: / text.

Output:

#J#o#b# #

Thanks for your answer,

Dennis

[Former Member]

Hi Dennis,

Can you paste sample data of two line.

Ashvender

[Former Member]

Hi Ashvender,

this is how the file looks in notepad (before any conversion takes place):

Job 'statistics' : Step 1, 'output' : Began Executing 2007-02-02 11:45:47

20072 2 norm 843 [SQLSTATE 01000]

20072 2 book 447 [SQLSTATE 01000]

And this is how it looks in hexadezimal (just the first few lines):

FF FE 4A 00 6F 00 62 00 20 00 27 00 73 00 74 00

This above mentioned hex-line stands for: "Job 'st" and first 2 bytes FFFE are the BOM.

SAP would do now something like this out of the HEX line:

J#o#b# and so on. When opening the text file with

open dataset in text mode encoding default

the system dumps. When I open it with

open dataset in text mode encoding default at position 3

it's alright, besides it's showing these '#' for every '00'.

Thanks,

Dennis

[Former Member]

Hi Dennis,

You have mentioned that you are getting data like this:

#J#o#b# #'#s#t#a#t#i#s#t#i#c

There is one solution to this: Do the following.

Upload the data in internal table using Open dataset encoding Utf-16. This will fill the internal table with data like this:

#J#o#b# #'#s#t#a#t#i#s#t#i#c

After close dataset write the following statement:

REPLACE ALL OCCURRENCES OF '#' IN TABLE it_vdata WITH c_space IN CHARACTER MODE.

NOTE: DECLARE C_SPACE like this:

CONSTANTS:c_space TYPE c value ''.

This will solve your problem & data will become like this in internal table:

Job 'statistics'

Ashvender

[Former Member]

Hi Dennis,

Has the solution sovled your problem or not? If yes close the thread & reward points.

Ashvender

[Former Member]

Hi,

Is your file separated by tab or space ? Because when you upload a file separated by tab or space, you will get # in between two fields. Here, # represents horizontal tab. Now, in your cse, 4A is getting converted properly in J and next # represents the empty space in between 4A and 6F.

Reward points if the answer is helpful.

Regards,

Mukul

[athavanraja]

check this code sample from Thomas Jung. This may be of interest to you

https://www.sdn.sap.com/irj/sdn/go/portal/prtroot/docs/library/uuid/7a13f367-0401-0010-47ba-eab0b15c...

Regards

Raja

[Former Member]

Hello everybody,

thank you very much for your ideas and hints. I will try these things and will reward points later this or next day.

EDIT: Since it works now with a workaround outside SAP I will not invest any more time into a SAP based solution. The conversion runs just fine, when uploading the text file via GUI Upload, which makes me suspicious that the way of uploading the file via ftp might be one problem - or the underlying file system - I do not know. Thanks everybody for the assistance, I will reward the points and close the toppic. Maybe I find a solution later, when the pressure is off.

Thanks so far,

Dennis

P.s.: Ashvender, I cannot see a possibility to open a file with open dataset with UTF-16 since 6.20's open dataset does not seem to support UTF-16 (according to the documentation).

P.s.: With binary mode I loose my CR/LF...

Message was edited by:

Dennis Hoofe