Application Development and Automation Discussions
Join the discussions or start your own on all things application development, including tools and APIs, programming models, and keeping your skills sharp.
cancel
Showing results for 
Search instead for 
Did you mean: 
Read only

[Bug] Class CL_RSDA_CSV_CONVERTER does not process line end characters correctly in source data

dr_ajt
Participant
2,845

I have used the class CL_RSDA_CSV_CONVERTER and similar versions of the class several times to convert ABAP structures into CSV data and CSV data into ABAP structures. I've used it on several systems and with several clients.

This week I noticed a very glaring issued with the SRUCTURE_TO_CSV method, which appears to be an issued with every version of the class I have access to.

If the data contains embedded newlines within an single ABAP field (which is permitted and even documented in the class) then the new line is not properly escaped and the resultant CSV is malformed and unusable. To quote from the class' documentation:

* A field that contains embedded line-breaks must be surounded by double-quotes
*
*     So:
*
*       Field 1: Conference room 1
*       Field 2:
*         John,
*         Please bring the M. Mathers file for review
*         -J.L.
*       Field 3: 10/18/2002
*       ...
*     would convert to:
*
*       Conference room 1, "John,
*       Please bring the M. Mathers file for review
*       -J.L.
*       ",10/18/2002,...
*
*     Note that this is a single CSV record, even though it takes up more than
*     one line in the CSV file. This works because the line breaks are embedded
*     inside the double quotes of the field.

I generated some data this week that contained CRLF and to my horror the class did not do what it says it should do, because the line endings are not part of the class's special_chars list, so they are just ignored, and you get the wrong result.

I suspect that this has gone undetected for years because data rarely has imbedded line ends within it, I was outputting data with text fields and many of them had line ends within them, but no quotes or commas, so I spotted it at once.

Note that the example in the documentation works because the class detects the comma contained within it, which is on the special characters list, without it, the example would fail.

The solution is to extend the length of the special characters, and include the permitted line end characters (Unix and Windows).

I have used the class CL_RSDA_CSV_CONVERTER and similar versions of the class several times to convert ABAP structures into CSV data and CSV data into ABAP structures. I've used it on several systems and with several clients.

This week I noticed a very glaring issued with the SRUCTURE_TO_CSV method, which appears to be an issued with every version of the class I have access to.

If the data contains embedded newlines within an single ABAP field (which is permitted and even documented in the class) then the new line is not properly escaped and the resultant CSV is malformed and unusable. To quote from the class' documentation:

* A field that contains embedded line-breaks must be surounded by double-quotes
*
*     So:
*
*       Field 1: Conference room 1
*       Field 2:
*         John,
*         Please bring the M. Mathers file for review
*         -J.L.
*       Field 3: 10/18/2002
*       ...
*     would convert to:
*
*       Conference room 1, "John,
*       Please bring the M. Mathers file for review
*       -J.L.
*       ",10/18/2002,...
*
*     Note that this is a single CSV record, even though it takes up more than
*     one line in the CSV file. This works because the line breaks are embedded
*     inside the double quotes of the field.

I generated some data this week that contained CRLF and to my horror the class did not do what it says it should do, because the line endings are not part of the class's special_chars list, so they are just ignored, and you get the wrong result.

I suspect that this has gone undetected for years because data rarely has imbedded line ends within it, I was outputting data with text fields and many of them had line ends within them, but no quotes or commas, so I spotted it at once.

Note that the example in the documentation works because the class detects the comma contained within it, which is on the special characters list, without it, the example would fail.

The solution is to extend the length of the special characters, and include the permitted line end characters (Unix and Windows).

0 REPLIES 0