Merging data in BOBJ Data Services

[Former Member]

Hi all,

May i know how are you guys merging data after matching using BOBJ Data Services in customer projects?

Other than using the automatic merging in Data Services, how is it done manually?

[Former Member · Accepted solution]

Merge: Combines incoming data sets, producing a single output data set with the

same schema as the input data sets.

Data Inputs

A data set from two or more sources with rows flagged as any operation code.

All sources must have the same schema, including:

u2022 The same number of columns

u2022 The same column names

u2022 The same data types of columns

If the input data set contains hierarchical data, the names and data types

must match at every level of the hierarchy.

Data Outputs

A data set consisting of rows from all sources, with any operation codes. The

output data has the same schema as the source data, including nested

schemas.

The output data set contains a row for every row in the source data sets. The

transform does not strip out duplicate rows. If columns in the input set contain

nested schemas, the nested data is passed through without change.

If the data types of columns in the sources do not match the target, add a

query in the data flow before the Merge transform. In the query, apply a data

type conversion to the columns with data types that do not match the target

column data types.

You must apply other operations such as DISTINCT in a query following the

Merge transform.

All the Best,

Madhu...

[Former Member · Accepted solution]

what i meant is,

when you are at a customer place, a customer has 1million records. Data Services can do the matching, but the customer wants to merge data manually.

How is it often done in the real world???? Do you just provide an excel spreadsheet to the customer???

[Former Member · Accepted solution]

Depending on the merge business rules it may be possible to implement many of the merge rules in the Data Services Best Record processing which is a post match process, at the dupe group level, and allows some simple rules to be implemented very easily and more complex rules to be implemented in custom Python.

[Former Member · Accepted solution]

I agree that you should try to incorporate the merge in Data Services to save your customers a lot of time and effort, but assuming DS is not an option, then yes, Excel is usually the next best solution mainly because the customers tend to be very comfortable with, and often pretty good at, using Excel to manipulate data.