Question Re: Selective Data Replication / Filtering in SLT in Technology Q&A

Selective Data Replication / Filtering in SLT

Former Member — Mon, 25 Feb 2013 11:34:51 GMT

Hi SLT experts,

I am trying to use the Selective Data Replication / Filtering feature of SLT.

i.e. I am only interested in loading specific year (2012) data for few tables of ECC into HANA.

Can anyone provide some guideline or details regarding the settings that need to be done in SLT.

Regards,

Sandeep

Re: Selective Data Replication / Filtering in SLT

tobias_koebler — Mon, 25 Feb 2013 13:55:07 GMT

Hi Sandeep,

have you checked the Guide for the Advanced Replication Settings with some useful examples.

You find it here: https://service.sap.com/sap/support/notes/1733714

Best,

Tobias

Re: Selective Data Replication / Filtering in SLT

Former Member — Mon, 25 Feb 2013 14:13:21 GMT

Hi Tobias,

Thanks for the quick response.

Yes, this was my starting point. I am trying to use the Event based rules. As soon as I write any simple code, the table replication errors out.

Do you have any samples codes that you have practically implemented.

Or is there any setting that need to be done in ECC side also which I may be missing?

Regards,

Sandeep

Re: Selective Data Replication / Filtering in SLT

tobias_koebler — Mon, 25 Feb 2013 14:36:03 GMT

Hi Sandeep,

it seems you have an error in the definition of the rule or in the abap code itself. Starting with page 39, the document explain what settings you have to specify and how to define the abap code itself. There you find also some example coding.

There is no need for any adjustements on the source system. The code is only executed at the SLT system.

Best,

Tobias

Re: Selective Data Replication / Filtering in SLT

justin_molenaur2 — Mon, 24 Feb 2014 20:43:12 GMT

Tobias, I am coming to a requirement that requires only replicating values from a given table based on some constant value of a column. In this specific table, the volumes are very large, so it would be ideal to not even have the source system triggers active except for the data we are interested in.

I have used IUUC_ASS_RUL_MAP functions for row level modifications and deletes, but here so much of the data is "throw away" that I don't think it makes sense to even bring it.

For this type of requirement, it seems like the function for IUUC_SPC_PROCOPT would be ideal as you can modify how (or what) data SLT listens to. Unfortunately, the only information available in the document you mention is

"Note that this option is intended for expert users only. We strongly recommend you contact SAP for support for using this feature as incorrect use can result in serious data inconsistencies"

Do you have any more detailed information on the usage you can share? Consider this as contacting SAP

Regards,

Justin

Re: Selective Data Replication / Filtering in SLT

tobias_koebler — Tue, 25 Feb 2014 13:18:08 GMT

Hi Justin,

sure

First an explaination of the fields:

TABNAME	Enter the name of the table in the sender system here.
DBSYS	Enter the database type of the source system.
LINE_NO	You can specify multiple lines if the condition is too complex to be filled in only one line.
LINE	Enter the trigger condition here. Only when the data change fulfills the condition , it will be recorded into the logging table for the SLT replication.

Example: Only replicate if AS4LOCAL is equal N:

DB2	___."field1" = 'value0' AND ___."field2" IN ( 'value1', 'value2' )	___."AS4LOCAL" = 'N'
DB6	___.field1 = 'value0' AND ___.field2 IN ( 'value1', 'value2' )	___.AS4LOCAL = 'N'
MSSQL	field1 = 'value0' AND field2 IN ( 'value1', 'value2' )	AS4LOCAL = 'N'
ORACLE	:___.field1 = 'value0' AND :___.field2 IN ( 'value1', 'value2' )	:___. AS4LOCAL = 'N'

Please note: trigger filtering will only work for replication phase. If it is required to filter the table during the initial load phase, an additional event (or parameter) filter is also necessary or with SP06 you can filter the initial load directly in the source - a how to document will be available soon!

Best,

Tobias

Re: Selective Data Replication / Filtering in SLT

justin_molenaur2 — Tue, 25 Feb 2014 18:56:37 GMT

Thanks Tobias, I already saw your post on filtering. Unfortunately I'll have to wait for a SP06 SLT to implement that.

Until then, I take it by your comments that a combination of event or parameter based filtering PLUS the trigger filtering will be required in conjunction in order to achieve the target result.

Is this going to be backward compatible on upgrades to SP06? ie since the Expert functions will now be available, are the IUUC_SPC_PROCOPT configurations still going to be available?

Regards,

Justin

Re: Selective Data Replication / Filtering in SLT

tobias_koebler — Fri, 28 Feb 2014 15:32:28 GMT

Hi Justin,

IUUC_SPC_PROCOPT will also work with Sp06.

Best,
Tobi

Re: Selective Data Replication / Filtering in SLT

Former Member — Tue, 11 Mar 2014 19:49:50 GMT

Hi Justin,

So finally what approached you followed for making it work both for Initial Load and Replication assuming we dont have SPS6.

I can understand what Tobias mentioned in terms of Trigger based filtering working only for Re[lication phase.

What extra do we need to do to make it work for Initial Load also ?

Thanks & regards,

Jomy

Re: Selective Data Replication / Filtering in SLT

justin_molenaur2 — Tue, 11 Mar 2014 19:58:26 GMT

Hi Jomy, you need to implement both IUUC_REPL_CONTENT filters (I did it at the BOR event) plus the trigger based filtering.

Since there is no initial load filtering available in SP05, you have to read all the data from ECC for the initial load, but it gets dropped in your logic during the BOR event.

Then going forward, only data relevant to your trigger filters in IUUC_SPC_PROCOPT will be replicated (the BOR event won't really filter anything during the replication phase).

Regards,

Justin

Re: Selective Data Replication / Filtering in SLT

Former Member — Tue, 11 Mar 2014 20:03:07 GMT

Thanks Justin!!

Re: Selective Data Replication / Filtering in SLT

Former Member — Thu, 13 Mar 2014 17:09:42 GMT

Hi Jomy,

If you are really dealing with large volumes the there is a way with SP05 to filter the select directly in ECC by using a process that involves entries in table DMC_SELSTRING. However, the process is not pretty 🙂

If you can go to DMIS 2011 SP06 and the latest corrections, then your options are better as described in the blog by Tobias. Else you can proceed as Justin has described, and use filtering in SLT...we had to revert to ECC filters for the initial load due to the volumes of data (4.5 billion records in BSEG) that have to be be stored in DMC_INDXCL...ran out of table space under Oracle and faced time outs. The load worked better under DB2 and we still use the DMC_SELSTRING option for reloads, but will migrate to using DMC_ACSPL_SELECT going forward. I have tested this with COEP and it worked a treat...but only after applying SP06 Correction 5,6 and 7 🙂

Thanks

Kris

Re: Selective Data Replication / Filtering in SLT

Former Member — Thu, 13 Mar 2014 20:09:25 GMT

Thanks Kris for the detailed description!!!

Re: Selective Data Replication / Filtering in SLT

Former Member — Tue, 18 Mar 2014 19:01:01 GMT

Hello Tobias,

Right now, SLT is on DMIS 2011 SP4.

1st Connection from source system(S1) to SLT(T1)

2nd Connection from same source system(S1) to SLT(T1). 2nd connection is having allow multiple usage indicator set.

I thought of only doing the event based filtering and filter the data in SLT itself and hence wrote a BOR event.

I did a local Table Type and new Table Name as well

Now wrote a BOR event for filtering as shown below:

It works fine during load and i get filtered data without any issues but later during the replication phase, i get runtime error and replication goes into error.

Is it mandatory to have the trigger filtering to work in replication phase in DMIS 2011 SP4 ?

error at the below line:

It would be great to have a reply from you on this.

Thanks & regards,

Jomy

Re: Selective Data Replication / Filtering in SLT

tobias_koebler — Wed, 19 Mar 2014 10:45:02 GMT

Hi Jomy,

I do not see any issue in the setup.

Anyway you do not need a trigger filter. Filtering only on SLT server should work with SP04.

You wrote that you defined the multiple usage flag for 2nd configuration. Have you also defined it for the first configuration?

I would open a ticket for further investigation.

Best,
Tobias

Re: Selective Data Replication / Filtering in SLT

Former Member — Wed, 19 Mar 2014 15:53:18 GMT

Hello Tobias,

To your question, I've not marked multiple usage flag for the second configuration.

Based on my question which I once mentioned here:

In this case, We went via the route to have One more configuration with a different configuration name from the same source to the same HANA target, but having the table as VBPA_ZR in the second configuration name and VBPA in the second configuration. The only difference between both configuration is 1st configuration 1:1 and second I've kept it as N:1.

Do we need to keep second one also as 1:1 or does it really matter, if I'm creating two altogether different configuration with configuration name itself.

What I tried now, implementing the Trigger filtering as well to see whether it is getting to any dumps or not.

Till now, I'm not getting dumps and hence closely monitoring the replication as of now.

I'll revert back or create a OSS message if something happens.

Thanks & regards,

Jomy

Re: Selective Data Replication / Filtering in SLT

Former Member — Fri, 21 Mar 2014 13:10:20 GMT

Hello Tobias,

Unfortunately this issue happened in Production now and hence we would be creating OSS as early as possible.

This is critical for customer's Month end reporting standpoint.

Thanks & regards,

Jomy