Technology Blogs by SAP
Learn how to extend and personalize SAP applications. Follow the SAP technology blog for insights into SAP BTP, ABAP, SAP Analytics Cloud, SAP HANA, and more.
Showing results for 
Search instead for 
Did you mean: 
Product and Topic Expert
Product and Topic Expert
Learn with this hands-on tutorial how to provide realtime predictions from SAP HANA Machine Learning through SAP AI Core. Set up a REST-API, that uses a previously trained model in SAP HANA Cloud (or SAP Datasphere) to return predictions.

To follow the steps in this blog you need to be familiar with the earlier blog SAP AI Core – Scheduling SAP HANA Machine Learning, in which a Machine Learning model is trained to estimate the price of a used car.

In this blog that trained Machine Learning model is now used in SAP AI Core to create predictions in realtime.

Table of contents


Code and configuration

All code that is used in this blog is available in this repository, to help you implement the scenario yourself.

We pick up from the previous blog (which used SAP AI Core to train and save the Machine Laerning model in SAP HANA Cloud) and jump right into the Python logic, that creates the REST-API and returns the predictions. The code in that file ( creates predictions by:

  • using the Flask framework to create a REST-API with the endpoint /v1/predict

  • retrieving the logon credentials for SAP HANA from a Generic Secret in SAP AI Core

  • establishing a connection to SAP HANA Cloud on startup, and attempting to reuse the existing connection for any future incoming request

  • uploading any incoming data as temporary table to SAP HANA Cloud

  • loading the saved Machine Learning model

  • obtaining a prediction from the loaded Machine Learning model on the temporary table, which has just been created

  • returning that prediction through the REST-API

import os
import json
import pandas as pd
import hana_ml.dataframe as dataframe
from hana_ml.model_storage import ModelStorage
from flask import Flask
from flask import request as call_request
import uuid

# Print external IP address
from requests import get
ip = get('').text
print(f'My public IP address is: {ip}')

# Variable for connection to SAP HANA
conn = None

# Function to establish connection to SAP HANA
def get_hanaconnection():
global conn
conn = dataframe.ConnectionContext(address=os.getenv('HANA_ADDRESS'),
print('Connection to SAP HANA established: ' + str(conn.connection.isconnected()))

# Creates Flask serving engine
app = Flask(__name__)

# You may customize the endpoint, but must have the prefix `/v<number>`
@app.route("/v1/predict", methods=["POST"])
def predict():
print("start serving ...")

# Convert incoming payload to Pandas DataFrame
payload = call_request.get_json()
payloadString = json.dumps(payload)
parsedPayload = json.loads(payloadString)
df_data = pd.json_normalize(parsedPayload)
# Ensure that the connection to SAP HANA is active
if (conn.connection.isconnected() == False):

# Load the saved model
model_storage = ModelStorage(connection_context=conn)
ur_hgbt = model_storage.load_model('Car price regression')

# Save received data as temporary table in SAP HANA
temp_table_name = '#' + str(uuid.uuid1()).upper()
df_remote_topredict = dataframe.create_dataframe_from_pandas(

# Create prediction based on the data in the temporary table
df_predicted = ur_hgbt.predict(data=df_remote_topredict,
prediction = df_predicted['SCORE'][0]

# Delete the temporary table
dbapi_cursor = conn.connection.cursor()
dbapi_cursor.execute('DROP TABLE "' + temp_table_name + '"')

# Send the prediction as response
output = json.dumps({'price_estimate': round(prediction, 1)})
return output

if __name__ == "__main__":
# Establish initial connection to SAP HANA

print("Serving Started")"", debug=False, port=9001)


The Python dependencies for the Docker image are in the requirements.txt file. This file is similar to the one used in the previous blog, when the model was trained. However, note that for creating the REST-API the Flask package is now included. 


The Dockerfile is identical to the one used for training the Machine Learning model. However, the Python code in, which is getting copied onto the image, is very different of course.
# Specify which base layers (default dependencies) to use
# You may find more base layers at
FROM python:3.9
# Creates directory within your Docker image
RUN mkdir -p /app/src/
# Copies file from your Local system TO path in Docker image
COPY /app/src/
COPY requirements.txt /app/src/
# Installs dependencies within you Docker image
RUN pip3 install -r /app/src/requirements.txt
# Enable permission to execute anything inside the folder app
RUN chgrp -R 65534 /app && \
chmod -R 777 /app


Build the docker image.
docker build -t YOURUSERHERE/hana-ml-car-price-restapi-inference:01 .


Push the image to the Docker repository, that is connected to SAP AI Core.
docker push YOURUSERHERE/hana-ml-car-price-restapi-inference:01


Now place this Argo ServingTemplate hana-ml-car-price-inference-rest-api.yaml into the Git repository that is connected to SAP AI Core. If the following values are identical to the one used in the WorkflowTemplate (as used in the previous blog), then AI Core automatically places both Templates into the same Scenario:




Note that the ServingTemplate's does not allow spaces in the name.
kind: ServingTemplate
name: i056450-hana-ml-car-price-inference-rest-api # Executable ID (max length 64 lowercase-hyphen-separated), please modify this to any value if you are not the only user of your SAP AI Core instance. Example: `first-pipeline-1234`
annotations: "HANA ML Car Price" "Estimate the price of a use car with SAP HANA Machine Learning" "HANAMLCarPriceInferenceRESTAPI" # No spaces allowed in the name here "Create estimate"
labels: "hana-ml-car-price" # The scenario ID to which the serving template belongs "0.0.1"
apiVersion: ""
annotations: | concurrency # condition when to scale 1 0
labels: | starter # computing power
spec: |
- name: docker-registry-secret-XYZ # your docker registry secret
minReplicas: 1
maxReplicas: 2 # how much to scale
- name: kserve-container
image: # Your docker image name
- containerPort: 9001 # customizable port
protocol: TCP
command: ["/bin/sh", "-c"]
- "python /app/src/"
- name: HANA_ADDRESS # Name of the environment variable that will be available in the image
name: hanacred # Name of the generic secret created in SAP AI Core for the Resource Group
key: address # Name of the key from the JSON string that is saved as generic secret
- name: HANA_PORT
name: hanacred
key: port
- name: HANA_USER
name: hanacred
key: user
name: hanacred
key: password



With the above steps SAP AI Core now has everything that's required to create the REST-API. In SAP AI Core go into the application from the previous blog, that trains the Machine Learning model. Hit "Sync" on that page so that the application picks up the new content from the above steps. Then click the "Refresh" button and you should see the new ServingTemplate in addition to the existing WorkflowTemplate.


The scenario now also shows both those Executables.


Now create a configuration based on the Inference Executable.


Personally, I found it useful to create two configurations. Both have exactly the same settings, they just have different names. This will be useful later, when updating the REST-API. We will come to that.


Now create the actual REST-API, that can be called to obtained predictions. Click into one of the two new Inference Configurations and hit "Create Deployment". Keep the Standard Duration, which means that the deployment is indefinite. It will keep running, until you stop it.


Review the settings, then create the deployment. After a few minutes the status switches to "RUNNING" and it shows the URL of the REST-API. The code is deployed, it is ready to provide predictions.


Hint: If you want to update the code, that is executed, when the REST-API is called, you can follow these steps. The deployment URL remains the same after the update. Those steps are not required now to follow this blog, but you might find this helpful, whenever you want to change the code behind the REST-API.

  1. Update the code in the Python file (here

  2. Build a new Docker image, which includes that new Python file, and push the image to the Docker repository

  3. In SAP AI Core sync the application so that it becomes aware of the changes (after the Sync hit the "Refresh" button to update the display)

  4. Go into the Deployment that is still running and click "Update". You can then replace the Configuration of the current deployment, with another Configuration, so that the deployment switches to the new code. A running configuration cannot be updated with itself. Hence, earlier on in the blog, two identical configurations were created with different names. You can switch between them to update the Deployment every time you have a change in the code.

Realtime inference

Any program that can call a REST-API can now obtain a prediction from the deployment. We use a Jupyter Notebook here to test this out.

When calling that REST-API you need to provide a Bearer token to authenticate. That token can be created by the Python program by using a few values from your SAP AI Core instance's Service Key. This key can be downloaded from the SAP BTP Cockpit. You might need to ask your Administrator to create that key for you.


In this example the Service Key is saved as the file AICoreServiceKey.json, which we load in the Python code to access the values that are needed to create the Bearer token.
import json
service_key_location = "AICoreServiceKey.json"
file_read = open(service_key_location, "r")
config = json.loads(
uua_url = config['url']
clientid = config["clientid"]
clientsecret = config["clientsecret"]


Create the bearer token.
import requests
params = {"grant_type": "client_credentials" }
resp ="{uua_url}/oauth/token",
auth=(clientid, clientsecret),
token = resp.json()["access_token"]


Specify the Deployment URL of the running Inference Workflow. That is the URL SAP AI Core was showing when the deployment went into "RUNNING" status. Also add the endpoint "'/v1/predict'" as defined in the Python program.
inference_url = 'YOURDEPLOYMENTURLHERE' + '/v1/predict'


Create the header for calling the REST-API, which passes the Bearer token. You also need to specify the name of the Resource Group, in which the deployment is running on SAP AI Core. There is no need to create a new Resource Group. Just enter the name of the Resource Group in which you created the deployment.
headers = {'Content-Type' : 'application/json',
'Authorization': f'Bearer {token}'}


Specify the vehicle's details, whose price you want to estimate.
payload = json.dumps({
"CAR_ID": 1,
"BRAND": "audi",
"MODEL": "a5",
"VEHICLETYPE": "coupe",
"HP": 120,
"FUELTYPE": "benzin",
"GEARBOX": "manuell",
"KILOMETER": 100000


Finally, piece all content together and call the Deployment URL
response = requests.request("POST", inference_url, headers=headers, data=payload)
if response.status_code == 200:
print('Error. Status code: ' + str(response.status_code))


The REST-API should respond with a price estimate for the car that you have provided. This  particular car is estimated to cost € 14.877,20 at the time, when the training data was collected.


You can experiment with different car specifications. If the car's mileage is only 50.000 kilometers for example (instead of the 100.000 above), the price rises to € 16.447.40.

This completes the scenario. You can now use SAP AI Core to:

  • train a Machine Learning model in SAP HANA Cloud

  • schedule that training process to keep the Machine Learning model up to date

  • create a REST-API to integrate realtime-predictions into your business application

Happy predicting!