
An often asked question is how to get data into Hana as there are so many different options. Let me try to differentiate the various methods from a technical point of view.
Standalone products:
and the Hana options:
The first and most important point to consider here is the question itself: How to get data into Hana. If Hana is just one of many sources and targets or there is no Hana in the picture at all, then the standalone products do make more sense. Or to argue into the reverse direction, there are so many different options because of the non-Hana scenarios. Okay, it would be odd if the standalone solutions can load everything but Hana, hence they have that capability as well which creates that confusion.
Example: My task is to integrate various system, the sources are an SAP ERP system, 5 Oracle databases, flat files of various formats, SQL Server,... and to load that into Teradata, Oracle and BW on Hana.
So Hana plays no or a little role in it, sounds like a perfect SAP Data Services scenario. Many sources, many targets and Data Services in the middle.
A few years ago, if a customer wanted to load e.g. flat files into Hana, he had to install Data Services. A full blown ETL tool just to load a few files from Mainframe (Cobol Copybook), CSV files with a non-default format and other sources. Only Data Services as an ETL tool provides reading capabilities of essentially every source system, allows the data to be transformed easily so it can be loaded into the target structures. An installation of a full blown ETL tool just for that.
If realtime is required as well, then SLT or SRS would have to be installed in addition.
In case the source data should be just made available and not copied - the federated/virtual data model use case - SDA would need to be configured as well.
So at the end, for an e.g. SQL Server database as source, three products are needed. All with their own install, with their own connector to the source, totally different look and feel and different capabilities.
For these reasons the Hana EIM Option Smart Data Integration was developed. Take the best concepts of all the products, merge those with existing Hana features and quickly you can develop the most powerful and easy to use product from ground up. Yes, SDI does not reuse any code from the old products.
Essentially, SDI is an extension of SDA which enhances it by
Ideally one source would have one adapter. One Twitter adapter to read Tweets. One Facebook adapter to read Facebook posts. One Oracle adapter to read Oracle data.
Some sources do provide different APIs to get data, especially when it comes to realtime. Take Oracle as a first example. To read from it executing SQL selects via JDBC is sufficient. But JDBC certainly does not support realtime. What are the realtime options for Oracle?
It does not make sense to have one adapter supporting all options or one adapter per realtime method. Hence the SDI OracleLogReader adapter is using the LogMiner technology for realtime plus the JDBC driver for normal reads.
Later we might add more adapters to support other realtime APIs as well, in case these are preferred by a customer.
SAP ERP is another example where multiple technologies could be used. There is certainly no JDBC driver for SAP ERP, so even reading tables is a challenge already.
As the goal of SDI was to provide a one stop solution for all Data Integration problems with Hana, the correct answer should be SDI. Only SDI ...
Also keep in mind, there are certain features missing as of today (SPS11), like Workflows. This is high on the priority list. The product was first delivered with Hana SPS09 which is relatively young compared to all others.
But again, SDI is the supposed optimal solution only because we have reduced the question to "loading into Hana".
There are more than enough use cases, even in the Hana world where other tools have the edge.
For Data Integration from many to many systems with all the requirements related, Data Services is and will be a good choice. Its focus is on batch performance, transformations and connectivity to every possible source/target.
The product's sweet spot is ABAP to ABAP realtime replication. Supported sources and targets are the databases ABAP runs on, Hana being one of them. Unlike the SDI ECCAdapter SLT is adding database triggers on the original source tables to capture the changes, which has downsides but also upsides.
For ABAP savvy users this is certainly the preferred option still. From a technical point of view 90% of the logic of SLT is what you would call an adapter in SDI. That is the reason why adding an SLT adapter to SDI would make so much sense.
Main use case for SRS is a multi source/target replication. Supports the most common databases only.
Had been a simple method to consume Extractors from Hana, completely replaced by SDA ABAPAdapter.
This tool does actually play in another league, it is an Enterprise Application Integration tool, not a Data Integration tool. Of course the difference between the two types is often marginal, e.g. if a customer master record consists of a single row in the application, there is no difference if you move the record (=row) or if you move the master record object (=row). It gets interesting in the complex cases, where one business object consists of many different types of Information.
With SAP BW on Hana the user is in a very comfortable position. For SAP extractors for example SAP BW has native support, why go another route? Hana data can be read as well. And for none-Hana data BW is using the Hana virtual tables, the SDI solution hence.
Therefore BW should not be considered an alternative but instead BW can utilize all other loading options to take advantage of those.
This feature enables the use of virtual tables in Hana. These are Hana objects that point to an external system to blend in the remote data in Hana. For example a SQL Server table CUSTOMER_MASTER can be blended in to Hana as virtual table V_CUSTOMER_MASTER. This virtual table is nothing else than a pointer to the remote table. So it does not store any data on the Hana side, all queries are forwarded to the remote system and the data retrieved.
SDA provides a set of adapters, all running in the Hana IndexServer and do use ODBC drivers. Nowadays the SDI provided adapters are used instead.
This solution is a natural part of Hana and extends Hana SDA. Therefore all Hana products and applications can take full advantage of its capabilities just like SDI does utilize other Hana features.
Hana Smart Data Streaming again does actually compare to the other Data Provisioning options. Although it has adapters as well, its main goal is to aggregate the incoming stream of data, mostly by time slices. A good example is the streaming of WebLog changes. Instead of storing the billions and billions of raw weblog information in Hana, it might make more sense to extract key information from the raw data and store this information instead. How many users have viewed a web page within 10 minutes intervals?
Therefore the SDS provided adapters are more focused on these kinds of source which produce millions of rows per second and not database sources. It does support reading from databases as well but just to lookup master data, not to subscribe to database changes.
You must be a registered user to add a comment. If you've already registered, sign in. Otherwise, register and sign in.
User | Count |
---|---|
13 | |
11 | |
11 | |
10 | |
10 | |
9 | |
8 | |
8 | |
7 | |
6 |