FedML SAP Datasphere & Databricks

[Isab3ll]

Hello everyone,

I am interested in applying FedML to Datasphere data in Databricks and subsequently deploying the inference results back into Datasphere, while also deploying the Machine Learning Model to SAP Kyma. Before proceeding, I have several technical questions that I would like clarified:

1) When I call an SAP Datasphere View for consumption in Databricks using SQL, does the data remain solely in SAP Datasphere, or is it also replicated into Databricks via the "select" command?

2) Is the configuration for the DBConnection between SAP Datasphere and Databricks the same as the connection setup between Databricks and SAP Kyma?

3) Can inference be performed on the deployed model within SAP Kyma?

4) Apart from Azure, Amazon, and Google, are there any other hyperscalers supported by Databricks that I might be overlooking?

I am looking forward to your responses. Thank you in advance.

Best regards from Germany.

[Sangeetha_K]

Hello,

Thanks for your questions . 

1) When you use FedML to source data from SAP Datasphere models, the data remains in its sources. You are federating or "reading" the data directly into compute memory for Machine learning purposes and hence not replicating it. 

2) The configuration for connecting to SAP Datapshere is different from connection to Kyma as they are two different services in BTP.  Please refer to documentation here: https://github.com/SAP-samples/datasphere-fedml/tree/main/Databricks.  

3) Yes inference can be performed on the deployed model in SAP Kyma . We have and end-to-end example that shows deployment and inferencing with SAP Kyma  . https://github.com/SAP-samples/datasphere-fedml/blob/main/Databricks/sample-notebooks/SAP%20Datasphe...  

4)The fedml-databricks documentation itself  shows how to use FedML with Databricks on AWS and Azure for eg.  But we have recently released a machine-learning platform independent version of the library that can be used in any machine learning platform not just databricks or hyperscalers in particular,

For further questions please feel free to reach out to us at : paa@sap.com 

Thanks,

Sangeetha