Data Services with Amazon Redshift.

[former_member244146]

Have seen a lot of posts on this not to many good answers.

Since I am starting a project on this I thought i would detail the project as it happens as it may help others.

We are migrating our Data Warehose to Amazon Redshift. We are using Data Services to load the data and Webbi to run reports from it.

I will start by what I have already done.

First trying to load data into Redshift directly is very slow and I don't recommend it, nor is it the suggested way to do it. You want to incorporate Amason S3 as your starting point.

The latest version of DS 4.2 has a native driver for S3. Set up a bucket then start your load there. We use CDC so we are loading our Changed data to S3. The different between this and the odbc into Redshift was as follow. Maybe a 100 rows a second if your lucky into the odbc. 300,000 to 400,000 per second into S3. Once the data is there you can set it up to copy over to Redshift from within Amazon. If all you want to do is get the data into Redshift then you are done here and it works great. The S3 is not very expensive either.

If you are using Redshift as your Data Warehouse then continue to read.

Once the data is in S3 then copy it to Redshift. We have done this it works well.

I am in the process of getting the SAP ODBC that comes with the BI tools as the native redshift ODBC is not 100% completely compliant(meaning not everything in DS works through it)

Hopefully with the SAP ODBC this will be better. My goal is to be able to push down all the sql so I dont have to host DS on Amazon. I will add to this after I have tested this part.

[former_member215510]

Hi Mark

I am working on a POC to pull data from S/4 Hana to Red Shift. Was there any reason you did not consider Matillion for extraction. Was it because you had SAP BODS already up an running and you wanted to avoid redeveloping the extraction on Matillion.

Can you share the learnings from your project. If you have a part 2 of this integration then, please share it with me.

Poonam

[former_member244146]

Yes one of the great benefits of using Data Services is all the connectors that come out for free. So when you decide you want to change and put your data warehouse on another technology you don't have to rebuild it all over again. That would be very painful. As we already had out Data Warehouse on SQL Server in DS moving it to redshift through S3 was very easy and not a big project.