Consuming Data from Databricks to Datasphere

[vignesh3027]

Prerequisites:

• Access: Access to both SAP Datasphere and Databricks.
• Credentials: Have Databricks authentication credentials ready.

Steps:

1. Login to Datasphere:

•  In the home screen of the datasphere, Expand the navigation bar on the left and click “Configuration”.

1. Database Access

• Click database access and then click Database Analysis Users

• Then click “Create” to create a database access user

• Enter the Database Analysis username suffix
• Enable Space Schema Access
• Set Database Analysis user expires in Never option.

• Once created it will generate a Hostname, Port Number and Password and make sure that Enable Space schema access has been enabled, so by this you can be able to access Datasphere’s all spaces and its schemas

1. Login to Databricks

• Go to Workspace, create Cluster

• In the cluster, paste this code to connect to datasphere and Databricks

Python

import org.apache.spark.sql.SparkSession

# Initialize Spark Session

spark = SparkSession.builder()\

    .appName("SAP HANA Connection")\

    .config("spark.jars", "/usr/sap/hdbclient/ngdbc-2.4.62.jar")\

    .getOrCreate()

# JDBC URL

jdbcUrl = "jdbc:sap://zeus.hana.prod.eu-central-1.whatever.cloud.sap:443/?encrypt=true&validateCertificate=false&currentschema=MYSCHEMA"

# Connection Properties

connectionProperties = {

  "user" : "MYUSER",

  "password" : "mypassword",

  "driver" : "com.sap.db.jdbc.Driver"

}

# Query to Select all the Data from MYTABLE

val query = "SELECT * FROM MYTABLE"

# Read Data from JDBC Connection into DataFrame

val df = spark.read.jdbc(jdbcUrl, query, connectionProperties)

• Once code is executed successfully, the connection is established between the Datasphere and Databricks.