Question Re: Sort records before processing in Technology Q&A

Sort records before processing

Former Member — Wed, 04 Jul 2007 05:10:15 GMT

Hi SDNers,

My requirement is File - XI - R/3 (IDOC). Before processing the file, it should be sorted out based on a particular field.

Eg. The input file may have this pattern. I need to sort the file based on second field.

D,5601,Jai,Caritor

D,5000,Raj,Caritor

D,7000,Albert,Caritor

Records after sorting

D,5000,Raj,Caritor

D,5601,Jai,Caritor

D,7000,Albert,Caritor

I tried the following way. Read the whole record as a node, so there are 3 nodes for the above example. With in adapter module, retrieve all records and based on the second field sort and then construct the XML with each record containing all 4 fields. I am using DOM parser for this.

Issue: My actual input file has 27 fields and the size of the file is expected around 2 to 20 MB. It is really diffucult to construct the output XML. Is there any other alternative or a better way to handle such scenario?

Share your ideas.

Thank You.

Regards,

Jai Shankar

Re: Sort records before processing

bhavesh_kantilal — Wed, 04 Jul 2007 05:19:46 GMT

Jai,

Why not use simple content conversion and then use the Sort option in message mapping?

Regards

Bhavesh

Re: Sort records before processing

Former Member — Wed, 04 Jul 2007 05:23:25 GMT

one issue i definately see here is you are using DOM..cant imagine a 20 MB xml being loaded into DOM.

I would suggest you a change in your approach using SAX...load the contents of your file into a dynamically growing datastructure like vectors...and also put the key field into another datastructure which then you could sort easily...rearrange / copy from the original data strcture to a new datastructure with your own sorting logic which may be looking up for an object with perticular field value(key value)..

Re: Sort records before processing

Former Member — Wed, 04 Jul 2007 05:30:13 GMT

Bhavesh,

I cant use the sort function with in Mapping. Reason is it will sort the particular field, but I need the whole record to be sorted before the mapping can occur.

@Amol,

Will look into the SAX option and try to use the data structures like Vectors.

Thanks a lot for the ideas. If there is any thing else that I can look into kindly let me know.

Regards,

Jai Shankar

Re: Sort records before processing

bhavesh_kantilal — Wed, 04 Jul 2007 05:32:37 GMT

Wouldnt actually happen with Sort as well?

Map the Target Node with the Corresponding Sorted Field and the right context, it will work exactly as you want.

Regards

Bhavesh

Re: Sort records before processing

Former Member — Wed, 04 Jul 2007 06:02:37 GMT

Bhavesh,

I would expect the target XML for the above example as

<Record>
<Id>5000</Id>
<Name>Raj</Name>
<COmpany>Caritor</Company>
</Record>
<Record>
<Id>5601</Id>
<Name>Jai</Name>
<COmpany>Caritor</Company>
</Record>
<Record>
<Id>7000</Id>
<Name>Albert</Name>
<COmpany>Caritor</Company>
</Record>

Not just sorted values for <Id> fields. I tried with sort for Id field. It did not work out.

Am I missing some thing here?

Regards,

Jai Shankar

Re: Sort records before processing

bhavesh_kantilal — Wed, 04 Jul 2007 06:43:19 GMT

Jai,

This is what I did,

Target Record Mapping :

Id --> Context of Root Node --> Sort --> Record

ID Mapping

Id ---

SORTBYKEY ---SPLITBYVALU --> ID

Id -- --

Name Mapping

SORTBYKEY ---SPLITBYVALU --> Name

Name

Likewise for Company.Tried it and this works

Regards

Bhavesh

Re: Sort records before processing

Former Member — Wed, 04 Jul 2007 07:26:03 GMT

Many thanks Bhavesh. This will solve my issue in most cases:-) (except a few where I actually need to group the records based on some conditions rather than a sort for which I will take Amol's advice).

Regards,

Jai Shankar