Generative AI with SAP – Part 5: Introduction to SAP AI Core

[Raja_Gupta]

This blog is part of the series Generative AI with SAP, which is focused on understanding how SAP is leveraging AI and generative AI across its portfolio.

In this series we are learning:

• How SAP is leveraging AI and generative AI across it’s portfolio.
• What SAP Business AI is and how SAP partners and customers can use it?
• What is SAP AI Core? What is SAP AI launchpad? What is Joule? etc.
• What role SAP BTP plays in SAP's AI strategy?
• Hands-on, Real-life examples and many more.

To make it easy to understand, the blog series is divided in small parts. Each blog requires maximum 10–15 minutes to learn.

• Part 1 – Introduction to SAP Business AI 
• Part 2 – Components of SAP Business AI 
• Part 3 – Joule, SAP’s Generative AI Copilot 
• Part 4 – AI Foundation on SAP BTP 
• Part 5 – Introduction to SAP AI Core [Current blog]
• Part 6 – Introduction to SAP AI launchpad
• Part 7 – Get Started with SAP AI Core and SAP AI Launchpad
• part 8 - Hello World using SAP AI Core
• Part 9 – Overview of Generative AI Hub
• Part 10 – SAP HANA Cloud vector engine
• Part 11 - RAG, a real-life use case using SAP AI Core, Generative AI Hub and HANA Vector Engine

This is the 5th blog in the series, where will learn about SAP AI Core.

Note: I will publish the subsequent  blogs soon.

Before understanding SAP AI Core, it might be a good idea to quickly break down few important terminologies to understand SAP AI Core better.

What is an AI Model?

An AI Model is a computer program that has been trained to perform specific tasks by learning from data. The AI models (programs) usually include complex mathematical and computational techniques to process vast amounts of data and extract meaningful insights.

Below image summarizes important points about AI Model.

AI models need huge datasets to get trained. The training data may include text, images, videos, numbers, or any other format of data. Some powerful AI models are trained on entire Internet data.

To know more about AI Models, you may refer to the blog What is AI Model?

What is AI Runtime?

AI Runtime refers to the environment or platform where AI models or algorithms are executed.

It includes the necessary software, tools, and resources needed to run AI models in production, such as hardware (CPUs, GPUs), software frameworks (TensorFlow, PyTorch), and supporting services (like inference servers).

AI Runtimes are used to deploy, manage, and run AI models efficiently. 

Examples of AI Runtime:

• TensorFlow Serving
• Amazon SageMaker
• Google AI Platform
• Azure Machine Learning
• SAP AI Core

What do we mean by AI Workflows?

In general, Workflow is defined as a sequence of tasks that processes a set of data through a specific path from initiation to completion. Workflows are the paths that describe how something goes from being undone to done, or raw to processed.

AI workflows are the workflows which use AI to streamline and improve business processes that are often manual and repetitive.

What is Inference Server?

Large language models, once trained, need to be deployed so they can be used to make predictions or generate text in real-time. This is where inference servers come into play.

An inference server hosts the trained model and handles requests from users or applications to generate outputs based on new input data.  

In other words, we can say - An inference server is a specialized server designed to deploy and run machine learning models, particularly for making predictions or inferences in real-time. 

Example of Inference Server

NVIDIA Triton Inference Server is a scalable and flexible inference server for deploying AI models at scale. Triton Inference Server can host and serve models like GPT-2 and GPT-3.

Now, let’s come to the main topic – SAP AI Core!

What is SAP AI Core?

SAP AI Core is a service available on SAP BTP, which offers a powerful AI Runtime. We can use SAP AI Core to run AI workflows and AI models.

SAP AI Core is designed to handle the execution and operations of AI assets in a standardized, scalable, and hyperscaler-agnostic way.

Some Important point on SAP AI Core

• SAP AI Core is SAP’s runtime for heavy-load AI. It allows us to train and deploy AI models cost-efficiently at scale.
• SAP AI Core is natively integrated with SAP AI Launchpad.
• The SAP AI Launchpad offers an easy-to-use interface to manage AI workflow administration, processes, and tasks.
• SAP AI Core can also be accessed by other supported tools, such as Postman or programmatically using Python.
• SAP AI Core supports full lifecycle management of AI scenarios.

Features and Capabilities of SAP AI Core

The main capabilities that SAP AI Core provides are the orchestration of AI workflows, such as model trainings and batch inference, as well as serving model inference, so that models can make predictions. Below diagram summarizes various features and functionalities of SAP AI Core.

Let’s demystify the above diagram!

Standardized AI Interface

The AI models running on SAP AI Cores can be easily integrated with any business applications with the help of a standardized AI interface. These standardized AI interface (called AI API), helps us integrate AI models as well as managed, monitored, and operated SAP AI Core with SAP AI Launchpad (or even Postman or Python).

What Exactly is AI API?

To connect to the business applications as well as to operate our AI scenarios in AI Launchpad, SAP AI Core provides the standardized AI interface, called AI API.

The AI API is not a product but a unified API definition, which provides a common framework for consuming and operating our AI scenarios. With this unified API definition, we benefit from unified consumption from business applications and unified operations and management in SAP AI Launchpad.

In other words, AI API provides an abstraction layer for managing AI asset, scenarios and workflows in SAP AI Core.  

For more information on AI API, refer this document or check AI API Reference.

Full Lifecycle Management

SAP AI Core provides full lifecycle management support such as content deployment using the GitOps principle. It is built around state-of-the-art open source solutions such as Argo Workflows and KFServing.

Choose your own infrastructure

One of the main benefits of SAP AI Core is that it allows us to choose from a broad range of storage, CPU and GPU service plans.

We can configure SAP AI Core to use different infrastructure resources for different tasks, based on demand. To achieve this, SAP AI Core provides several preconfigured infrastructure bundles called “resource plans” for this purpose. This document provides the list of existing resource plans.

Simplifies AI development using SAP managed deployments

With SAP AI Core, we benefit from SAP-managed model deployments, which hide the complexity of managing deployments. We can just expose an API endpoint that can be integrated and consumed from a business application. SAP AI Core exposes simple API endpoints to embed AI into your business applications.

Perfect balance between costs and performance

With SAP AI Core, on the one hand, we benefit from accelerated performance with GPU support to run most resource-hungry use cases at scale. On the other hand, we can run efficiently while keeping control of the costs by leveraging built-in autoscaling and scale-to-zero as well as by choosing from a broad range of storage, CPU, and GPU service plans.

Note: Scale-to-zero is a concept often used in cloud computing, particularly in the context of AI. It means that an application/service, or AI model is automatically turned off or scaled down to use zero computing resources when it is not in use to save cost. When a request comes in, the system automatically "scales up" by starting up again, processing the task, and then scaling back down to zero once the task is complete.

Easily embed AI capabilities in SAP applications and business processes

With SAP AI Core, we can take advantage of the standardized integration into SAP applications to quickly build and integrate AI use cases into business applications.

CI/CD and Multitenancy Support

To ease the shipment of new AI scenarios, SAP AI Core offers continuous delivery capabilities and ensures tenant isolation with multi-tenancy. To safeguard customer’s costs and to scale on demand, we only need to pay for the resources we use but can tap into scalability and increased performance powered by GPUs.

All of the functionality comes out of the box and is managed by SAP while retaining openness to any AI framework so that customers can ship their AI scenario easily.

Development Tools – Libraries and SDKs

SAP AI Core provides development tools, such as SAP AI Core SDK to simplify the development process.

The SAP AI Core SDK is a Python-based SDK that lets you access SAP AI Core using Python methods and data structures. It provides tools that help you to manage your scenarios and workflows in SAP AI Core.

You can check the SAP AI Core SDK project at https://pypi.org/project/ai-core-sdk/

I hope you have got a clear understanding of what SAP AI Core is and it’s features. For more information on SAP AI Core, you may refer to SAP AI Core.

If you have reached so far, don't forget to give a kudos 😉

What’s Next?

Part 6 – Introduction to SAP AI launchpad [To be published soon]