SAP HANA Meets Gen AI 🤖 | Build Your Own AI Assistant with GPT-4 + Streamlit !

[abhimanyu_sharma]

The Starting Point

In this blog post, I'm sharing my dive into the world of Generative AI.

 Like many, I'm fascinated by what Large Language Models (LLMs) can do - writing code, drafting emails, even creating images. But on their own, they're often disconnected from the live data businesses run on. You can't just re-train them every day; it's too costly and slow. This got me thinking: how do we bridge that gap?

How can LLMs tap into real-time information or query our complex company databases, maybe even those with millions of records and intricate relationships?

This is where I started exploring Retrieval-Augmented Generation (RAG) and AI agents. RAG is great for feeding relevant context to the LLM, whether from documents or other sources. Agents can act on instructions, like querying databases or using APIs.

Agents represent systems that intelligently accomplish tasks, ranging from executing simple workflows to pursuing complex, open-ended objectives.

Building agents involves assembling components across several domains—such as models, tools, knowledge and memory, audio and speech, guardrails, and orchestration.

For the deeper technical aspects, this post assumes a basic understanding of common AI terms such as tokenization, vector embeddings, and prompting methods. However, the main architectural concepts will be explained simply

Why Gen AI Outperforms Traditional Programming?

• Traditional: You usually need specific buttons, forms, dropdowns, or exact commands. If you want to ask for sales in a slightly different way, you often can't – the programmer had to anticipate exactly how you'd ask.
• Gen AI: You can just talk or type naturally! Ask "Show top sellers," "Who sold the most?", or "What products are flying off the shelves?" – the AI is designed to understand the meaning behind your words, even if they're phrased differently each time. This makes it way more intuitive and less rigid.
• Traditional: Code needs precise instructions. If you misspell "Eclairs" as "eclars," a traditional search might just fail unless the programmer specifically added code to handle common typos (which is hard to do exhaustively!)
• Gen AI: It's much better at handling ambiguity, typos, or slightly unclear requests. It can often figure out you meant "Eclairs" or ask you a clarifying question if needed, rather than just giving up. It rolls with the punches better.
• Traditional: If management suddenly wants a new type of report or chart, a developer usually needs to go in and write new, specific code for that exact report format or SQL query.
• Gen AI: You can often just ask for the information in a new way. The LLM, guided by its instructions and tool descriptions, can potentially generate the new SQL or figure out how to combine existing tools differently on the fly, without needing explicit code changes for every single variation.
• Traditional: Best suited for well-defined tasks where the steps are clear (e.g., "Show sales for this specific product ID").
• Gen AI: Can attempt more exploratory questions like "Are there any interesting trends in the sales data for last quarter?" or "Summarize the performance in the UK." It can use its reasoning ability (and the tools you give it) to try and figure out how to approach these less rigidly defined problems.

Note: It's not magic! Gen AI needs clear instructions (your System Prompt), access to the right tools (your Agents), and sometimes makes mistakes (hallucinations or incorrect assumptions). As of now, Traditional programming is still essential for building the reliable tools the AI uses, handling complex business logic precisely, and ensuring security and data integrity.

The real power often comes from combining them – using Gen AI for the flexible, natural language understanding and planning front-end, and traditional programming for the robust, reliable back-end tools and agents that interact with databases and APIs

I wanted to create something where anyone could just ask questions about our sales data (stored in SAP HANA Cloud) in plain English and get answers, charts, or even related info without needing to know SQL or navigate complex dashboards.

Below is the Architectural Diagram for better understanding.

🌐Tech Stack:

💾SAP HANA Cloud

🧠 GPT-4o (OpenAI)

🛠️ Streamlit + Python

🧪 Google Colab + Ngrok

🗃️ Real sales data from Kaggle

Step 1: You Ask a Question

It all starts with you! You type your request into the chat interface of our Streamlit app – maybe something like "Show me sales for Product A" or "What's the weather in Berlin?". This is your Input Prompt.

Step 2: The App Sends Your Request to the Open AI LLM

The Streamlit app (our lightweight user interface) takes your question and sends it off, along with some context and built-in instructions, to the OpenAI LLM (Large Language Model - gpt-4o).

Step 3: The LLM Understands and Plans

This is where the AI magic happens! The LLM uses its powerful Natural Language Processing (NLP) capabilities to understand what you're asking for. It looks at your specific words ("sales", "Product A") and combines that with its knowledge and the specific instructions we gave it (the "System Prompt").

Based on this understanding, it figures out:

• What needs to be done? (e.g., "I need to query the sales database.")
• Which Agent is best for the job? (e.g., "Agent A, the database expert, should handle this.")
• What specific information does that specialist need? (e.g., "I need to generate an SQL query to find 'Product A')

It then generates precise instructions, often including generated code like SQL, formatted in a JSON.

Step 4: The App Delegates to the Right Agent

The Streamlit app receives the JSON instructions back from the LLM. It reads the instruction, sees which agent the LLM selected (like Agent A), and what input it needs (like the SQL query). The app then calls the specific Agent function within our code.

Step 5: The Agent Gets to Work

The activated Agent takes the input provided by the app (as instructed by the LLM) and performs its specialized task:

• Agent A (Database Agent): Connects to the database (SAP HANA or others) and runs the SQL query.
• Agent B (API Agent): Calls an external API (like a weather service or internal company API).
• Agent C (Web Search Agent): Performs a web search to find information.

The agent interacts with its designated backend system to fetch the required data or perform the action.

Step 6: Checking In & Getting Ready for the Next Step

Once the Agent finishes its task, it sends the result (like the data found, the weather report, or a success/error message) back to the Streamlit app.

The app then sends an Observation message back to the LLM. This is super important! It tells the LLM: "Okay, Agent A ran the query, and here's what happened (e.g., 'found 10 rows' or 'encountered an error')." This allows the LLM to:

• Track the progress of your request.
• Handle any errors gracefully.
• Plan the next step if it's a multi-part task (like getting data then creating a chart).

Step 7: Delivering the Final Answer

Once the LLM determines the task is complete (either after one step or multiple steps), it generates the final, user-friendly response. It sends this final response (again, likely in that JSON format with "step": "output") back to the Streamlit app.

The app takes this final text and displays it back to you in the chat window.

And that's how your simple question gets turned into a specific action and a clear answer, all orchestrated by the AI brain and executed by specialized agents!

Demo :

To see how this works in practice, I loaded up some chocolate sales data from Kaggle into our HANA Cloud instance ( you can connect with any DB ) . I also connected to a free weather API and even made a simple agent using Python's OS tools to create files on your local desktop or if you run this in your vs code  , even can generate code files for your project based on instructions ( like mini cursor 😀 )

Here are some of the things I asked it:

Show me the top 3 best-selling products based on total sales amount and display the results clearly.

Result: It correctly figured out it needed to SUM the sales, GROUP by product, ORDER them, and LIMIT to 3. It generated the SQL and showed me the table!

You can even see behind the curtain in the "Execution Details" to see the SQL it wrote.

Can you show me sales of eclars  -  (Deliberate typo!)

• Result: Success! Because we told the AI brain (in the System Prompt) to always use case-insensitive searching (UPPER()), it found the sales data for "Eclairs" even with my typo.

In Other Language : 

Muestra los tres productos más vendidos según el monto total de ventas y muestra los resultados claramente  -

Create a line chart comparing the monthly sales trend for our two key products: '85% Dark Bars' and '99% Dark & Pure'. Title it 'Monthly Sales Trend: Key Products'. – Complex Visualization

Result: This was more complex! It first ran a query to get the monthly sales for both products, then correctly called the charting agent to create a line chart with two distinct lines – perfect for visual comparison.

What was the exact total sales amount generated in the USA for August 2022 and weather of New York?

• Result: This was cool – it seamlessly called the HANA agent for the sales figure and then called the Weather agent for NYC, presenting both pieces of information in the final answer. Two agents working together!

Negative Testing:

Show me all the data in the chocolate sales table.

Result:  The assistant blocked this! It recognized that asking for the entire table without filters is usually a bad idea. This is possible with System Prompt.

While this prototype provides a glimpse of the potential, the underlying agentic architecture is designed to be scalable and can be further developed using industry-standard practices for security, monitoring, and robustness.

So, what does this all mean? By blending the conversational Power of an LLM with focused agents that can securely talk to our real systems (like HANA, APIs, or other tools), we're building more than just a Q&A bot.

We're creating tools that feel natural, breaking down technical barriers and putting powerful data insights directly into more people's hands. It feels like we're just scratching the surface – I truly envision a future where this kind of seamless AI integration becomes the norm, making all our tools smarter, friendlier, and ultimately, more helpful.

Github link : 

https://github.com/abhimanyu-agi/genAI--SAP-HANA-AI-Assistant

Demo Video:

https://www.youtube.com/watch?v=UApzc5pQLXI