HANA SDI Implementation

[former_member278448]

1. In BODS, we have project, scripts and query transform that are very handy and useful. What are the equivalent objects in HANA SDI(workbench and/or webide)?
2. In BODS, we can right click and import a template table to make it permanent, how can we do the same in SDI(workbench and/or webide)?
3. In BODS, we can create a dataflow and link to another dataflow within a workflow, how to achieve the same process in SDI(workbench and/or webide)?
4. In BODS, we can debug from designer, what is the process to debug a flowgraph from SDI (workbench and/or webide)?
5. In BODS, we have central repo to share and migrate, what is the equivalent in SDI(workbench and/or webide)?
6. Besides the youtube channel is there documentation that goes into the details of how SDI should be implemented (best practices, etc.)?

[jeff_k]

SDI is not Data Services. The functionality is data federation (remote data brought into HANA) first and transformation second. Most of your questions are addressed in the modelling guide, recommend you to check that document: https://help.sap.com/viewer/product/HANA_SMART_DATA_INTEGRATION/2.0_SPS03/en-US

1. All the scripting can be done with procedures in HANA SQL, and there are various transform nodes that approximate the function of DS transforms.
2. In HANA catalog you make virtual tables based on remote tables from your remote sources.
3. Not sure what the real use case is here, but you can probably make a flowgraph with a procedure target and reuse that procedure in other flowgraphs.
4. It will depend on the error you are facing in the flowgraph, but the HANA editor is able to show design time output for many nodes, so the debug will be possible in SDI as well.
5. Can export/import the .hdbflowgraph or .hdbreptask files. Or HANA permissions can be granted on an object or package basis.
6. Best practices for what? We have some recommendations for agent and HANA sizing, but the best practices will change based on what you are doing. One recommendation is to use replication tasks if at all possible, since flowgraphs have much more overhead and performance will usually be a lot better in a replication task.

Regards,
Jeff K

[former_member278448]

Thank you for the detailed response. Really appreciate it.

I agree SDI is not DS in the sense of a complete ETL tool but more of a data federation tool with some ETL capabilities, however, the common belief is that SDI can replace DS in its entirety.

1. Yes, I am aware of scripting with procedures in SQL but DS has a scripting language and script transform that is very useful and wanted to know if SDI has an equivalent transform other than procedural SQL. But I understand your point.

2. Virtual tables and template tables are not the same. Template tables are created at design time with source data structure and is dropped/recreated every time the job is run. Virtual tables are federated tables. We have the same template table concept in SDI as well and it works exactly the same way as DS, the only difference is you can right click and make them permanent tables in DS while in SDI (workbench/ide) you cannot or atleast, I have not seen a way to do it yet.

3. The use case is, in DS, you create a dataflow that does something, then once that dataflow completes execution, you can call another one sequentially as the output of the first one is used as the input to the second one. This is a common DS functionality. I understand your point and that is what I was afraid of the only way to implement.

4. I haven't seen documentation about it in SDI. Hana Studio is very different from workbench/IDE so whatever works in studio may/not work in WB/IDE.

5. Thank you.

6.Thank you.

Do you have recommendation/documentation on when it will be beneficial to use SDI over DS other than real-time? DS real-time is limited while SDI real-time is more expansive.

[Sefan_Linders]

Adding to Jeff, on point 3, if you mean scheduling flowgraphs in sequence, take a look at the Data Warehouse Foundation Scheduler, which provides graphical and more advanced scheduling options. For more information, visit https://help.sap.com/viewer/dcc2ad3d29604f0da9673f8e77ba3a5d/2.0.4.0/en-US/f59aa29683664f6e83e199d16... or watch this video: https://www.youtube.com/watch?v=F7ELG_k1vYY∈dex=5&list=PLkzo92owKnVxZcLRGmfZS-bdlIFYRQ39R

[former_member278448]

Thank you.