SQL Query Push Down to external Source

[amos_mastel]

We have connected several external sources to a HANA database and included them as virtual tables. From this tables we want to include the data into our reporting queries to implement a so called drill down scenario where the user can report on data loaded in the HANA database and then drill down to detail data from the external source.

What we now see is that the HANA database does not push down the sql query, it does read the full content of the data table and then try to run the selection in the HANA database. The sources we connect are hadoop and adls storages and contain billions of records, so this always ends in a timeout. We have created database views with specific data type definitions to abstract the untyped (strings) raw data. Is there any setting we can do that the query is passwed down to the spark cluster we use for the data selection?

We currently cannot use the SAP HANA Spark controller from an architecture perspective.

[vamsiintelgraph]

Amos,

We had similar but not exactly the same issue of SQL push down. I am assuming you are expecting for the where clause to be pushed down to the remote source and its not happening. There are a variety of factors the SQL optimizer considers for filter push down's. The simple thing to try out in your case is to create statistics on the virtual tables. Create a simple statistics and then try running the query and observe if it changes the behaviour.

Best,

Vamsi.