On-Premise AI Innovation: SAP HANA EE and Skybuffer AI Deliver Advanced RAG Solutions

[Siarhei]

General Introduction: SAP-Connected On-Premise AI Solution

In our previous article, On-Premise AI Excellence: How SAP HANA EE and Skybuffer AI Deliver Cutting-Edge Solutions, we explored the foundation of on-premise AI solutions powered by SAP HANA Extended Edition (EE) and Skybuffer AI - SAP Certified Product. This pioneering approach highlighted the synergy between SAP HANA EE's robust data management capabilities and Skybuffer AI's intelligent automation framework.

We showcased how combining these technologies enables organizations to leverage on-premise infrastructure for advanced AI applications, ensuring data security, compliance, and seamless integration with existing SAP ecosystems. 

Building on that foundation, this article takes a step further by introducing our Retrieval-Augmented Generation (RAG) solution. Designed specifically for on-premise environments, this new development offers metadata-driven precision, user-friendly interfaces, optimized performance, and stringent access control, all while remaining fully compatible with SAP HANA EE.

Problem Description: Missing Steps in Simple RAG Solutions

While Retrieval-Augmented Generation (RAG) has gained significant traction as a framework for improving AI-driven knowledge retrieval, many implementations face critical limitations when deployed in enterprise environments:

Lack of Metadata Layer for RAG Documents

Simple RAG solutions fail to associate metadata with each document, significantly limiting their ability to provide contextually relevant results. Metadata is essential for categorizing, filtering, and improving document retrieval accuracy, especially in enterprise scenarios where precision and relevance are crucial.

No Access Control Based on Metadata Filtering

A key limitation of many RAG solutions is the absence of robust access control mechanisms based on metadata filtering. This omission creates security vulnerabilities in multi-user environments, where restricting content access by roles, permissions, or business rules is critical for ensuring compliance and protecting sensitive data.

No Vector Engine in SAP HANA On-Premise

Currently, SAP offers vector engine capabilities exclusively for SAP HANA Cloud databases. This limitation restricts the implementation of AI-powered solutions that rely on vector-based retrieval within on-premise environments. As a result, enterprises utilizing SAP HANA Extended Edition (EE) on-premise infrastructure face significant challenges in adopting state-of-the-art RAG solutions optimized for their existing setups.

How Skybuffer AI Addresses On-Premise Challenges with No-Code AI Platform on SAP HANA EE

SAP HANA Extended Edition (EE) is a powerful on-premise platform that enables organizations to run applications on top of a fast, enterprise-level database. It offers an exceptionally secure environment, trusted and utilized by hundreds of thousands of companies worldwide. SAP HANA EE allows organizations to host all necessary applications for an AI platform within its application layer. The only missing component for a comprehensive AI solution is a vector engine. However, this gap is easily addressed.

Skybuffer AI delivers a vector engine for SAP HANA EE as part of its AI platform, making it possible to deploy a Retrieval-Augmented Generation (RAG) solution in a customer's preferred data center, including large language models (LLMs). For use cases like embedding functions and generative AI summarization, smaller models such as Llama 3.1 8B (used in today’s demo) or Llama 3.3 70B perform exceptionally well and do not require high GPU computation power.

Pic. 1 Skybuffer AI - Defining Aliases for RAG Metadata Characteristics

How Skybuffer AI’s RAG Solution Works

Skybuffer AI provides a RAG user interface developed using SAP Fiori UI technologies to automate the following core functions of the RAG solution:

• RAG Upload
  When organizations deal with hundreds or thousands of files for the RAG solution, our user interface allows users to easily upload and manage files. This solution is fast, out-of-the-box, and incurs no development costs.

• RAG Review
  This function enables users to view extracted text directly from documents within the application, helping to assess document quality and decide whether it should be part of the RAG solution.

Pic. 2 Skybuffer AI - RAG Document Content Review

• RAG Metadata
  Metadata is an essential layer that offers flexibility for organizations to use 16 string, 16 numeric, and 16 date-based characteristics that define aliases corresponding to the organization’s document metadata. This ensures seamless integration with the existing data structures in place.

Pic. 3 Skybuffer AI - RAG Documents Filtering by Metadata

• RAG Clean-Up
  The ability to review and delete RAG documents is crucial to maintaining an organized, updated, and efficient document storage system.

How to Connect RAG Metadata with Access Control

Skybuffer AI provides a Conversational AI engine as part of every AI model by default. The engine is based on fast embeddings and does not require complex large language models (LLMs) like BERT or RoBERTa. This makes it lightweight, fast, and ready for use without the need for model training. Users can define intents or entities that can be immediately applied in the system.

Pic.4 Skybuffer AI - Conversational AI Engine

Access Control Integration
Once intents or entities are defined for access control, they can be linked to the RAG search via Skybuffer’s no-code Action Server. This allows organizations to:

• Filter RAG documents based on user permissions and metadata.
• Optimize RAG search performance by limiting the scope of documents being analyzed, resulting in faster responses.

By combining Conversational AI with RAG, users are guaranteed quick, secure access to only those RAG documents they are authorized to view.

No-Code Action Server from Skybuffer AI

With Skybuffer AI’s no-code Action Server, users can create and manage triggers for RAG and Generative AI actions without any coding.

Pic. 5 Skybuffer AI - Action Server

Key capabilities include:

• Intent-based Access Control: Define access rules by simply creating skills and associating them with RAG document filters.
• RAG Search Optimization: Use the defined intents/entities to limit search scope and improve performance.

Pic. 6 Skybuffer AI - RAG Action

Additionally, users can implement Generative AI actions to summarize RAG search results, which can be tagged for future reporting and analysis.

Pic. 7 Skybuffer AI - Generative AI Action

Business Case: Company Policy Assistant with Access Control

Let’s consider a business use case: a company wants to provide a common AI assistant that only accesses information within the Procurement or Staff-related policies.

Pic. 8 Skybuffer AI - AI Models Dashboard

We upload two documents (PDF and Word) to the RAG solution using the enterprise-level user interface.

• During the upload, we add metadata to classify each document by its category and line of business.
• We create two simple intents: one for Procurement and another for Staff. These intents correspond to skills that enable separate RAG searches for each topic, ensuring accurate and secure access to relevant content.
• With the ability to filter documents based on metadata, we can guarantee that only authorized users access the appropriate content.
• To enhance user experience, we can use a small LLM like Llama 3.1 8B for Generative AI summarization, optimizing costs and avoiding the use of large, resource-intensive models.
• Tags are applied to categorize conversations and enable future analysis and reporting.

Quick and Easy with Skybuffer AI

Once configured, the AI Assistant is ready for consumption across various platforms, including:

• Web chat
• MS Teams
• SAP Fiori Launchpad
• Other corporate communication channels

Pic. 9 Skybuffer AI - Web chat channel

You can request free demo tenant access via the SAP Store to experience the full capabilities of Skybuffer AI.

P.S. Don’t forget, Skybuffer AI is fully integrated into SAP ECC and SAP S/4HANA systems out of the box, providing ready-to-consume scenarios for Generative AI interactions with your SAP ERP backend.