- SAP Managed Tags
What an amazing Sapphire Orlando in 2024 was! It is firing on all cylinders with amazing announcements of new solutions, new AI capabilities, current solutions getting better, new and strengthened partnerships within the ecosystems, and so on and so forth. Obviously, that was made possible because of the amazing customer base (in person, online, and in spirit) and the tenacious SAP staff who are dedicated to bringing the best to our customers.
However, reporting on what happened at SAP Sapphire is not the purpose of this blog post. I want to take you a little into the background to share a use case that we experimented with internally to try to elevate your experience at Sapphire. Yes, we used Generative AI (GenAI) on the SAP Business Technology Platform (SAP BTP) to equip our customer-facing teams to efficiently identify and recommend sessions based on the specific situation in your organization. There were more than 550 sessions over 12 solution areas and seven types of sessions. It's a lot, and building a perfect agenda that gives the best back for your time did need some extensive planning. Also, depending on where your organization is in your own transformation journey, there were some excellent sessions that provided great opportunities for you to learn from, make connections, and carry them back home to delve further into over the next few weeks.
So, in the spirit of being relevant, reliable, and responsible, we created an internal tool to indirectly help you via our field sales teams, especially your account executives. I am going to briefly touch upon some of the technologies that were used in the making and, most importantly, share what we learned from this experiment.
NOTE: The use of this tool was discretionary due to it's experimental nature.
But first, what did the tool do?
The tool is a recommendation engine built using Generative AI techniques (more below), the consumption of which happens using a simple web page that captures relevant information from the user. It was a conscious decision to build a form and not a chat interface (we've got enough of those already!). This way, we took away the need for the user to be an expert prompt engineer. We manage prompt engineering expertise in the backend.
Fig. 1. SAP Sapphire Recommender
Upon filling in the information, the recommendation engine refers to the Sapphire session catalog from a vector database and identifies a list of sessions that will be beneficial for the described situation, landscape, and the role of the organization coming to Sapphire.
Then, based on the identified list of sessions, the tool makes personalized recommendations on why each session was identified and the value that can be obtained in the context of their current situation. Finally, it also wraps it up by writing a personalized email to the role identified, which serves as a cover letter for the recommendations.
Fig. 2(a). Response - personalized email
Fig. 2(b). Response - personalized session recommendations
Now, the Technology & the Technique
The tool uses the popular Retrieval Augmented Generation (a.k.a. RAG) technique, which we used from Generative AI Hub in SAP BTP. The Generative AI Hub provides SAP AI Core, where we enable the LLM models from Azure Open AI service, as well as the Prompt Editor from the SAP AI Launchpad, where we created our prompt template, which was used to generate a well-grounded LLM prompt in runtime depending upon the information supplied via the UI.
Fig. 3. GenAI using RAG on SAP BTP
Sapphire session information, being practically static information for all sets and purposes, was converted into text embeddings and stored in the SAP HANA Cloud Vector engine. The RAG application would play a central role here in receiving user input via the UI, converting it into text embeddings, and using these embeddings to perform semantic search in the SAP HANA Cloud Vector engine. This ensured there was a semantic relation between the situation described by the user and the session descriptions.
Lessons Learnt
So, apart from the fact that we ate our own dog food, we had many lessons to learn from this internal experiment. Some of the critical ones are worth sharing and are listed below:
- Pay special attention to Data source, Data attributes, and Data quality. RAG helps ground LLMs with contextual information. So, the better the context, better is LLM response. Also try experimenting with data format as it inversely impacts token requirements and response quality.
- Test, Test, Test. Prompt template testing is the most crucial part of building GenAI applications. You need to make sure prompts are provided specific goals, tasks, and guardrails.
- Watch-out for hallucinations! Make sure you utilize the 'temperature' settings for LLMs appropriately. In cases where creativity can be detrimental, set lower temperatures. For example, in this tool, LLM generating imaginary sessions would be detrimental, so lower temperatures were set. On the other hand, for content generation, such as emails, blogs, etc., higher values will work best.
- Set and standardize expectations from LLM, specifically the response format. Specify if you want it to provide a response as a table, or as a JSON, or simple text with or without formatting, within specific length, etc.
Biggest Take-away!
There are many low-hanging opportunities around us where technology like GenAI or platforms like SAP BTP can provide quick value. If nothing, these low-hanging opportunities provide perfect opportunities to learn and set up the foundation to manage the next innovation projects. Leveraging select use cases as showcases helps drive new idea generation and incremental utilization of foundation technology investments (stickiness).
If you have incorporated GenAI in any low-hanging opportunities in your organization, we'd love to hear about your experience and approach from you. Stay engaged!
