- SAP Managed Tags:
This blog is part of the series Understanding Joule in 8 Baby Steps In the previous blog #1 – Introduction to Joule, we learnt what Joule is and how customers can use it.
In this post, we will learn how Joule works.
Let’s quickly recap what we learnt so far:
1️⃣Joule is an advanced, generative AI copilot.
2️⃣It is embedded into various SAP applications, for example SAP S/4HANA, SAP SuccessFactors etc.
3️⃣Users may simply ask a question in plain language and receive intelligent and contextualized answers.
Now, let's see how Joule works. Below image summarizes the complete flows starting from user input till the final response from Joule.
Step 1 – User Enters the Prompt
Users start with their queries in the natively integrated Joule client embedded in SAP cloud solutions, such as S/4HANA, SuccessFactors etc.
Step 2 – The request is processed and orchestrated by Joule
1. First, Joule analyses if the prompt is relevant for Joule’s Scenario Catalog.
What is Joule's Scenario Catalog?
Joule's Scenario Catalog is a comprehensive repository that contains metadata of all available scenarios, functions, and skills across SAP cloud applications. When a user interacts with Joule, their input is first analysed against this catalog to determine if it aligns with any predefined scenarios or functionalities. This process ensures that Joule can provide accurate and contextually relevant responses based on the capabilities defined within the SAP ecosystem.
2. Further, Joule conducts an informational filtering based on the Knowledge Catalog. This process is based on RAG and lets the LLM generate an answer which is contextually relevant and precise.
Knowledge Catalog is a repository that contains SAP’s proprietary knowledge and customer owned knowledge via Document Grounding.
3. Joule is aware of the user's context and chat history.
This includes information about which SAP cloud application the respective user is using and which additional SAP application the customer has licensed and activated with Joule.
Also, Joule is aware of the user's role(s) and permissions. This means a user cannot access information or adjust business processes they couldn't do directly in the SAP cloud application without using Joule.
Step 3 – Joule sends an enhanced query to LLM
Joule takes all of this information and provides a much richer query to the LLM.
These LLMs are also available directly via SAP’s Generative AI Hub, which is part of the SAP BTP.
Which LLM is used by Joule?
Joule follows SAP‘s general multi-partner strategy for generative AI. Joule may employ a variety of foundational models to deliver the best performance for each use case. This ensures that customers get benefitted from the most relevant LLM.
Joule leverages only those LLMs which are available for SAP customer and partners within Generative AI Hub in AI Foundation in SAP BTP.
SAP has under strict contractual agreements with LLM providers which prohibits them from training any of their AI-models on any SAP’s customer data.
Step 4 – LLM’s response back to Joule
Next, the LLM provides a grounded response back to the Joule service.
This is:
- Either the response based on its training data and knowledge which could be sent directly to the user
- or there is a need to leverage Joule functions which includes the Joule scenarios of SAP Cloud Applications.
In simple words, if the user’s query requires action within an SAP system or information from SAP system, the LLM identifies the need to call Joule functions, which connect to SAP Cloud Applications (e.g., S/4HANA, SuccessFactors).
Step 5 – Joule Calls the SAP Backend System
If required, Joule leverages Joule functions to call an SAP backend system.
1. Joule sends the request to the relevant SAP backend system.
For this, the respective information is retrieved via Odata calls OR the LLM for dialog management calls another LLM which was used by an SAP Cloud Application to develop a specific scenario (e.g. interview question generation).
2. Joule gets the required information, processes it.
3. The response from SAP is also filtered and validated to ensure it meets enterprise security, data privacy, and responsible AI guidelines. This step ensures that only secure, accurate, and appropriate responses are sent back to the user.
Step 6 - Final Response to User
The user receives a response, knowing that the conversation and output has been processed all with enterprise security, data privacy, and responsible AI considered.
This is in nutshell - How Joule works!
What's Next?
You may also read the blog Joule, SAP’s Generative AI Copilot
1 Comment
