Transformation Error - Duplicate Data Record Detected

[Former Member]

Dear BI experts, I'm hoping someone can help me with this particular BI7 problem, with regard to a transformation expert routine on a DSO.

We have created a transformation, which uses an expert routine. It takes source data drom an existing DSO, checks each SOURCE_PACKAGE record and then generates additional records (based upon look-ups to other DSOs) appending the records to RESULTS_PACKAGE. This process works fine in many of our other scenarios, exept for this one case:

In this specific instance we're getting an error message of "Duplicate data record detected (DS....., data package: 000001 , data record: 149)", with a message class of RSODSO_UPDATE, and a message no. of 19.

We have debugged the routine carefully, and checked the entries in RESULTS_PACKAGE, and there are, to the best of our knowledge, no duplicate records conatined in the table.

Once the routine has failed, we can then manually activate the data request in the DSO, and it activates OK, missing out the records that apear to have caused the error. We have examined the error stack carefully, and as far as we can see these records do not have duplicates, so we're a little bit confused as to where the problem is.

As this is a transactional full load, I don't appear to have the option on the DTP to ignore duplicates.

When I run the DTP in debug mode, the job runs through the simulation to completion without any error messages.

We're on patch level 21 within BI 7.0. Any help or advice on this would be greatly appreciated.

All the best,

Jeff.

[Former Member]

hello jeff, i too am having a similar problem. have you managed to fix it yet. if so, how ?

thanks

[Former Member]

Hi Glen, unfortunately we never resolved this. We were attempting to produce an InfoProvider for a fairly complex reporting requiement. As a way around the problem we decided to produce two separate InfoProviders and subsequently two separate reports. Not ideal, but we ran out of time and just couldn't find a way around that particular problem.

Please do let me know if you manage to resolve it.

All the best,

Jeff.

[Former Member]

hi jeff, i managed to fix my issue to some extent.

in my transformation, i was also copying over the 3 SAP technical fields that made up part of the key

SID, DATAPAKID, & RECORD

i realize that they were technical and of no use to my functional business logic, so i removed them to see what effect it would have. to my surprize the records ran in successfully.

now i am on to another weird problem.

let me know how you go, if you decide to re-visit your issue

cheers

glen

[Former Member]

Thanks for the info Glen, much appreciated. I'll give that a try and let you know how I get on.

All the best,

Jeff.

[Former Member]

hi jeff, by not copying the sap technical fields over, it did fix my issue. the other problem is something else.

good luck

[Former Member]

Hi Jeff,

Any luck with this? I am running into a similar error message. I am generating multiple records out of one record in an end routine, but I have ensured that there are no duplicates using code, so I am at a bit of a loss as of what to do next.

Thanks,

Abid

[former_member186445]

you need to fill the record number yourself in the routine otherwise the technical key of all the records you generate from the same source record will have the same key and therefor your load will dump on not uniqueness of key

add a counter and fill the field RECORD:

v_count = v_count + 1.

RESULT_FIELDS-record = v_count.

[Former Member]

Hi Abid, unfortunately we didn't find a solution to this problem.

From a previous reply to this thread you'll see that Glen managed to find a solution. However, this didn't work for our situation when we tried it.

I also filled in the Record No., as suggested above, but this didn't resolve the problem either.

Since my last posts, my colleague has now worked around the problem by creating an Infoset on the DSO's in question, using one as the primary and two other DSO's linked using outer joins, avoiding the need to use the transformation expert routine in this particular instance.

All the best,

Jeff.

[Former Member]

thanks, that was the solution. the RECORD field needs to be unique in this scenario.

[former_member186445]

you're welcome...

by the way, assigning points is the way of saynig thank you on sdn.

M.

[Former Member]

how do I assign here? I didn't ask the question, so I don't think I have the option.

[Former Member]

Other way of achieving it would be to define the semantic keys for the DSO in the DTP. That will solve the duplicacy thing.

[thomas_kpp2]

It sounds as if I have exactly the same problem as Jeff in the end routine (Duplicate data record, RSODSO_UPDATE, 019).

Some input files did work others not, although the content was neary identical.

> semantic keys for the DSO in the DTP ...

I UNchecked all keys of the semanatic group for the error stack. Know it works.

Oddly enough I did check these to avoid the error in the first place.

I removed a lot of the code and changes I made to resolve the issue.

Now it still seems to work fine, although I have the impression, that

nothing is different from the first version when I got the error.

> not copying the sap technical fields [SID, DATAPAKID, & RECORD] over, it did fix my issue

> fill the record number yourself ... add a counter and fill the field RECORD:

Sound logical, but I have never seen a need for this before. I have not tried this, because I solved it as described above.

[Former Member]

Jeff,

Are there any internal tables defined with unique key?

Does any of the select statements in your look up return mulitiple records for the key combination?

The above check points might give some direction.

[Former Member]

Hi Sharat, thanks for taking the time to reply.

All internal tables have been defined with non-unique keys, and I've also ensured that the "Unique Data Records" checkbox setting on the DSO is not ticked. The select statements don't return multiple records for the same key.

All the best,

Jeff.