topic Re: ABAP code to delete duplicate records in Application Development and Automation Discussions

ABAP code to delete duplicate records

a_ahmad — Wed, 17 Jul 2013 15:16:21 GMT

Hi,

I need help to write complete ABAP code for below scenario.

Request 101 loads data from PSA to WODSO and I have records like below in DSO active table /BIC/AFIGL00

request no. DocNo Line item Ccode Amount

101 9001 1 A100 100

101 9002 1 A100 100

(among lots of other fields. Just to clarify this data is based on 0fi_gl_4 datasource)

These records are loaded futher into SDSO

with delta request 102, I get records like

request no. docNo Line item Ccode Amount

102 9001 1 A100 100

102 9002 1 A100 100

If I load these futher into SDSO then I have wrong figures. These duplicate records under different request nos. should be deleted/ignored while load from WODSO to SDSO. Therefore, I think i need to have code as Start Routine

In some old posts I have got syntax

Sort <internal table>.

delete adjacent duplicates from <internaltable> comparing all fields.

but I need complete code syntax. Does <internal table> means DSO active table.

Thanks

Re: ABAP code to delete duplicate records

NagaPrakashT — Thu, 18 Jul 2013 07:32:04 GMT

Hi Ahmad,

Internal table is the SOURCE_PACKAGE[] in the start routine of the DSO.

Thanks,

Naga

Re: ABAP code to delete duplicate records

RaymondGiuseppi — Thu, 18 Jul 2013 07:51:07 GMT

Don't do that

Those records must be processed as some data of the accounting document was changed triggering the delta extraction (e.g; clearing date), just insure that amount/quantity key-figures are correctly copied to DSO:

In transformation, check via rule detail,

must be "MOV Overwrite" and not "SUM Summation".

If you use old 3.x flow, please specify.

Regards,

Raymond

PS: Also don't forget the keys of the DSO must always contain 0COMP_CODE, 0FISCPER (+0FISCVARNT) and 0AC_DOC_NO, 0ITEM_NUM to match BSEG primary keys.

Message was edited by: Raymond Giuseppi

Re: ABAP code to delete duplicate records

a_ahmad — Thu, 18 Jul 2013 09:50:12 GMT

thanks for replying

we have 7.3x system

WDSO - keys: fiscper, fiscvarnt, fiscper3, fiscyear, Ac_doc_no, Pstng_date, comp_code, chrt_accts, gl_account, item_num

from Datasource to WDSO : aggregation is 'overwrite'

from WDSO to SDSO: agg. is Summation

Reason: in WDSO we have line-item data but then further on we don't need doc_no level data. we need data at aggregated level only.

After first full load, we reconciled figures with ECC and this dataflow worked fine.

Problem rose after first delta load. From PSA to WDSO, delta failed because of duplicate records. I changed WDSO settings to allow duplicate records and it worked (you can say problem avoided . But after load from WDSO to SDSO, we reconciled figures again and found errors. We narrowed down: records in IC, SDSO, WDSO and found exact same records but with under different requests in WDSO. These records came through the delta. So when they are aggregated from WDSO to SDSO, they are creating wrong figures.

so, we are alright keeping these records in WDSO but then on these should be deleted/ignored.

I need help in writing ABAP code for this issue.

For data in each delta load request, data should be checked against all data of WDSO active table and duplicate records coming in delta request should be ignored for futher aggregation into SDSO.

e.g. Req 101 lying in WDSO and is aggregated into SDSO, now delta Req102 in WDSO, before load into SDSO this should be checked against all data of WDSO active table and if duplicate records are found, should not be aggregated in SDSO.

hope this clarifies

Regards

Ahmad

Re: ABAP code to delete duplicate records

RaymondGiuseppi — Thu, 18 Jul 2013 11:25:28 GMT

(I compare with a 731 SP 06 SAP_BW system)

IMHO when you ignored duplicate records, you caused the problem, some data changed in ECC was no longer pushed into BW DSO. First DSO data must be an exact replication of ECC, you should also have kept the keys of the standard DSO, to prevent such problem, did you change other attributes comparing DSO 0FIGL_O02 ?

Then from this correct DSO, after activation, you can send delta requests to following targets, and yes there use SUM summation on key-figures.

Can you post your data flow ?

Regards

Raymond

Re: ABAP code to delete duplicate records

a_ahmad — Thu, 18 Jul 2013 13:32:16 GMT

Data flow:

DS -> WODSO -> SDSO -> IC

key fields in WODSO: fiscper, fiscvarnt, fiscper3, fiscyear, Ac_doc_no, Pstng_date, comp_code, chrt_accts, gl_account, item_num

comparing with 0FIGL_O02, in my WODSO I have Posting data in addition.

For the time being I deleted all delta requests in WODSO as they were bringing duplicate records.

Need to figure out solution first and implement it for data load either from PSA to WODSO (wiht DSO setting: don't allow duplicate records) or from WODSO to SDSO (where WODSO setting: allow duplicate records)

I need to use below code but complete is for my scenario.

Sort <internal table>.

delete adjacent duplicates from <internaltable> comparing all fields.

thanks

Re: ABAP code to delete duplicate records

a_ahmad — Mon, 22 Jul 2013 10:01:40 GMT

Any help on this issue

Before loading delta data from WODSO to SDSO, need to write code to check delta data against all data of WODSO and ignore/delete duplicate records that is not letting them go into SDSO.

thanks