Group, Sort and handle Duplicate XML records – using UDF

[RaghuVamseedhar]

Below User Define Functions (UDF) logic can be used to group, sort and handle duplicate XML records.

Note: -

1. Number of queue entries in each input field, to UDF should be equal (except key fields). Use ‘mapWithDefault’ for all optional input fields.
2. Input fields context should be removed. Use ‘removeContexts’ OR right click on field and then change ‘Context’.
3. Create UDF with Execution Type as ‘All Values of Queue’. Input all related fields to one 'multiple input and multiple output UDF'. When match is found, write all related fields to output.
4. Please tryout below examples in sandbox. And then solve your actual requirement.

Grouping XML records

Here is input XML and expected output XML: -

Mapping: -

UDF: -

UDF will search ‘Name_People’ in ‘Name_Record’, if it finds a match, corresponding ‘Where’ and ‘Phone’ will be written to target fields with Context Change. Context Change will be added to output ‘Record_out’ for each ‘Name_People’.

Sorting XML records

Here is input XML and expected output XML: -

Mapping: -

Mapping is same as above, add standard function 'sort' to key field. UDF logic is same as above. Output XML will be sorted with Name_People.

Handling Duplicate XML records

Here is input XML and expected output XML: - Duplicate records are ignored.

Mapping: -

UDF: -

HashSet will return true, if unique value is added. It will return false, if duplicate is added.

UDF code for reference

// Grouping XML records.

public void udf_getCorrespondingRecord(String[] Name_People, String[] Name_Record, String[] Where, String[] Phone, ResultList Record_out, ResultList Where_out, ResultList Phone_out, Container container) throws StreamTransformationException{

for(int i = 0; i < Name_People.length; i ++) {

     for(int j = 0; j < Name_Record.length; j ++) {

         if(Name_People[ i ].equals(Name_Record[ j ])) {

             Record_out.addValue(" ");

             Where_out.addValue(Where[ j ]);             Phone_out.addValue(Phone[ j ]);

             Where_out.addValue(ResultList.CC);    Phone_out.addValue(ResultList.CC);

         }

     }

     Record_out.addValue(ResultList.CC);

}

}

//If match has to be found using a combination of two fields. Please use logic something like this in above UDF:

//if((Name_People[ i ] + Where[ i ]).equals(Name_People[ j ] + Where[ j ]))

//------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

//------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

// Handle duplicate XML records.

public void udf_getUnique(String[] Id, String[] Name, String[] field1, ResultList Record_out, ResultList Id_out, ResultList Name_out, ResultList field1_out, Container container) throws StreamTransformationException{

Set<String> unique = new HashSet<String>();

for(int i = 0; i < Id.length; i ++) {

    if(unique.add( Id[ i ] )) { // True if unique.

      Record_out.addValue(" ");

      Id_out.addValue(Id[ i ]);                    Name_out.addValue(Name[ i ]);             field1_out.addValue(field1[ i ]);

      Id_out.addValue(ResultList.CC);   Name_out.addValue(ResultList.CC);   field1_out.addValue(ResultList.CC);

    }

}

}