- SAP Managed Tags:
Welcome to this deep-dive blog series about SAP AI Core template creation. In this series of blogs, I will show some of the details of the WorkflowTemplate and ServingTemplate concept by concept. This blog will focus on ways of using environment variables with your AI Core Workload.
Environment Variables in ML Workflows:
Environment variables play a crucial role in machine learning (ML) training workflows and deployments, providing a flexible and secure method for parameterization and configuration. They are commonly used to specify hyperparameters, such as learning rates and batch sizes, allowing for easy adjustments without modifying code. Environment variables also point to data sources, indicating paths to training datasets, validation datasets, and other essential resources, which simplifies data handling across different environments.
In deployments, environment variables are vital for securely managing credentials and API keys. This approach avoids hard-coding sensitive information in the codebase, enhancing security. For instance, database connection strings, cloud storage access keys, and service credentials are often set as environment variables.
When working with Docker containers, environment variables facilitate the configuration of containerized applications. They allow for dynamic adjustments to container behavior based on the deployment environment, enabling the same image to be used across development, testing, and production with different settings tailored via environment variables. This modularity and security make environment variables indispensable in modern ML workflows and deployments.
Concept in SAP AI Core:
Overall SAP AI Core Workloads are defined in the two types of templates: Workflows and Servings. They themselves leverage two open source API specifications from the Kubeflow ecosystem: Argo Workflows and KFServing. Thus both Template types do have some distinct differences. Although they both do share the YAML syntax and use plenty of artifacts from the underlying Kubernetes environment, like the Containers definition. This means the same scripting syntax is used across the templates.
Overall, there are multiple ways to specify environment variables in SAP AI Core:
- Directly hardcoding them in the template.
- Referencing a configuration parameter from the template.
- Referencing a secret from the template.
- Hardcoding them in the Docker image.
Option 4 is quite independent from AI Core and can be achieved using the simple ENV Keyword in the Dockerfiles, plus it also does have some disadvantages, because we do not necessarily want our secrets and parameters to be bound to the source code. Let's examine how the first three options appear in the template:
WorkflowTemplates:
apiVersion: argoproj.io/v1alpha1
kind: WorkflowTemplate
metadata:
name: trainenvdemo
annotations:
scenarios.ai.sap.com/description: "trainenvdemo"
scenarios.ai.sap.com/name: "trainenvdemo"
executables.ai.sap.com/description: "trainenvdemo"
executables.ai.sap.com/name: "trainenvdemo"
labels:
scenarios.ai.sap.com/id: "trainenvdemo"
executables.ai.sap.com/id: "trainenvdemo"
ai.sap.com/version: "1.0.0"
spec:
entrypoint: trainenvdemo
arguments:
parameters:
- name: "ENV_FROM_CONFIG" # name of the configuration parameter "not" the actual env variable set below
description: "description of the parameter"
templates:
- name: trainenvdemo
container:
image: docker.io/python:latest
env:
- name: ENV_SECRET
valueFrom:
secretKeyRef:
name: felix-secret-1
key: api-key
- name: ENV_HARDCODED
value: "here we can hardcode any values we just want to have in the template"
- name: ENV_FROM_CONFIG
value: "{{workflow.parameters.ENV_FROM_CONFIG}}"
command:
- python3
- '-c'
args:
- |
import os
print("Hello from SAP AI Core ENV Demo")
print(os.environ.get("ENV_SECRET", "not_set"))
print(os.environ.get("ENV_HARDCODED", "not_set"))
print(os.environ.get("ENV_FROM_CONFIG", "not_set"))
We need to set the environment variables directly in the container specification using the env keyword. In there we specify the name of the environment variable under which we can use it in our source code later. The values need to be specified either directly - for example like ENV_HARDCODED, referenced via the replacement syntax to the workflow parameters or by using a secretKeyRef. The replacement syntax refers to the field name of our parameter under spec > arguments > parameters. You can use a different name than the actual environment variable name.
In Action, we need to create a Configuration containing the actual values to be set in those variables - as we can see for this example I did not specify a default, so I do have to set a value to continue:
Creating a Generic Secret in AI Launchpad:
Referencing a Secret can be very helpful to deal with Credentials. In AI Launchpad we can maintain Generic Secrets either on Resource Group Level or on Tenant level.
For the example I created one under my resource group with the name felix-secret-1. This name is referenced in the secretKeyRef > name. The secret content needs to be in JSON format. The secret can contain multiple keys that we reference by the secretKeyRef > key parameter.
Note: We need to encode all the values in our secrets! AI Core does not accept secrets with arbitrary values see the official documentation.
For the example you can do this either key by key in the command line:
> echo -n 'hey this is top secret' | base64
> aGV5IHRoaXMgaXMgdG9wIHNlY3JldA==
Or via a simple python script - encoding all keys in a object:
import base64
secrets = {"secret1": "very_secret", "secret2": "more_secret"}
def base64_encode(value):
encoded_bytes = base64.b64encode(value.encode('utf-8'))
encoded_str = encoded_bytes.decode('utf-8')
return encoded_str
encoded_secrets = {key: base64_encode(value) for key, value in secrets.items()} # Base64 encode all the values in the secrets dictionary
print(encoded_secrets)
Outputting:
{'secret1': 'dmVyeV9zZWNyZXQ=', 'secret2': 'bW9yZV9zZWNyZXQ='}
Demo:
When running the above template, we see the below output.
> Hello from SAP AI Core ENV Demo
> hey this is top secret
> here we can hardcode any values we just want to have in the template
> this is a value from the configuration
All three types of environment variables are successfully created and able to be consumed from our code. Interestingly the base64 encoded secret is also handed over in plain text to the executing code, nice! 😊
ServingTemplates:
Now in the second part, let's look at the example for the serving templates:
apiVersion: ai.sap.com/v1alpha1
kind: ServingTemplate
metadata:
name: serveenvdemo
annotations:
scenarios.ai.sap.com/description: "serveenvdemo"
scenarios.ai.sap.com/name: "serveenvdemo"
executables.ai.sap.com/description: "serveenvdemo"
executables.ai.sap.com/name: "serveenvdemo"
labels:
scenarios.ai.sap.com/id: "serveenvdemo"
ai.sap.com/version: "1.0"
spec:
inputs:
parameters:
- name: "ENV_FROM_CONFIG" # name of the configuration parameter "not" the actual env variable set below
description: "description of the parameter"
default: "some default value for ENV_FROM_CONFIG"
template:
apiVersion: "serving.kserve.io/v1beta1"
metadata:
annotations: |
autoscaling.knative.dev/metric: concurrency
autoscaling.knative.dev/target: 1
autoscaling.knative.dev/targetBurstCapacity: 0
labels: |
ai.sap.com/resourcePlan: starter
spec: |
predictor:
minReplicas: 1
maxReplicas: 5
containers:
- name: kserve-container
image: docker.io/python:latest
ports:
- containerPort: 8080
protocol: TCP
env:
- name: ENV_SECRET
valueFrom:
secretKeyRef:
name: felix-secret-1
key: api-key
- name: ENV_HARDCODED
value: "here we can hardcode any values we just want to have in the template"
- name: ENV_FROM_CONFIG
value: "{{inputs.parameters.ENV_FROM_CONFIG}}"
command:
- python3
- '-c'
args:
- |
import os
print("Hello from SAP AI Core ENV Demo")
print(os.environ.get("ENV_SECRET", "not_set"))
print(os.environ.get("ENV_HARDCODED", "not_set"))
print(os.environ.get("ENV_FROM_CONFIG", "not_set"))
The main difference if we compare the both template, is the place to put the "inputs/configurations" to. In the serving case, we specify them under spec > inputs > parameters. So I replicated the same scenario, having one hardcoded, one via configuration and the secret reference. When defining a default value, we are prompted with it as a help in the AI Launchpad:
Demo:
Other then that, when running the deployment, we will not go into running mode because we did not set up a actual server - but the logs still show, that the code is able to access the environment variables:
> Hello from SAP AI Core ENV Demo
> hey this is top secret
> here we can hardcode any values we just want to have in the template
> some default value for ENV_FROM_CONFIG
Hope you find this blog insightful, feel free to try out the example templates and leave a comment in case of further questions. 🙂
