4 data sources into one import model

sridhar_v6 — Fri, 25 Jun 2021 03:27:11 GMT

Hi ,

we have a requirement to merge data into a model. We are using import connection. Sources are sap ecc, sap bydesign, sap business one, Google drive. Please suggest if we can merge the sources into one model or not. And how delta load works for these sources.

Thanks

Sridhar

Re: 4 data sources into one import model

DebjitSingha — Fri, 25 Jun 2021 04:37:36 GMT

Hello sridhar.v6,

You can add multiple datasources into a single model. System do not put any limitation on creating heterogeneous queries. Only thing to keep in mind is the structure and logic of data coming from different data sources. SAC model treats every dimensions as key fields. Beside you need to make sure there is a common date based delta field available from all 4 system. Tricky part is ByDesign date format. Sometime date fields are not detected as correct format from ByDesign.

SAC offers different modes of delta as well. Building a Query

Update (most common)
Append
Clean and replace selective version data
clean and replace subset of data

Depending upon your underneath data logic you need to choose one suitable for your requirement.

Now say all four system have same set of columns/ dimensions. Then the question comes do you want to overwrite / update/ replace with new data etc.

or you may want to keep data coming form different DS separate, this can be achieve by adding a new dimensions "Source System" and then adding some flag (say ByDesign, ECC..) during data wrangling. Future data loads will consider same set of rules, hence all future loads will get tagged.

If you find above info helpful, feel free to up-vote (arrows on the left) and accept (accept button at the bottom).

Thanks,

Debjit

Question Re: 4 data sources into one import model in Technology Q&A

4 data sources into one import model

Re: 4 data sources into one import model