- SAP Managed Tags:
Introduction
In my previous post, I showed how to get started with Retrieval-Augmented Generation (RAG) using SAP AI Core’s built-in integration, leveraging out-of-the-box SAP product documentation as the default grounding content—no setup required. But what if you want to ground your LLMs using your own documents?
In this follow-up, I’ll show you how to configure AWS S3 (via the Object Store service on SAP BTP) as a custom data repository source for document grounding. This gives you the flexibility to serve proprietary or domain-specific content into your RAG workflow—all while using the same orchestration and retrieval APIs in SAP AI Core.
If you want to the configuration with Microsoft SharePoint, you can refer to another blog from my colleague @rachel_gan on "Setting up SAP Document Grounding for Chat Apps" or developer tutorial "Orchestration with Grounding Capabilities in SAP AI Core".
Let’s walk through the setup.
Prerequisite
- Subscription to Object Store on SAP BTP - Standard plan (AWS regions)
- Subscription to SAP AI Core - Extended plan
- Subscription to SAP AI Launchpad – Recommended for easier management, and testing (this blog will demonstrate API-based steps with Postman)
Restrictions (as of this blog publication):
- Object Store on SAP BTP is only available with CPEA or BTPEA entitlement. You can find more details about pricing and service plans in the Discovery Center here.
- SAP AI Core free tier is only available in SAP BTP free tier, not SAP BTP trial.
- The Extended Plan for SAP AI Core is only available with CPEA or BTPEA entitlement. You can find more details about pricing and service plans in the Discovery Center here.
Initial Setup
Step 1: Service instance & Service key creation
This guide assumes you're already familiar with subscribing to SAP BTP services via the Service Marketplace, creating service instances, and generating service keys.
You should have the following already set up:
- SAP Object Store (in an AWS region) – Ensure you’ve provisioned the service and obtained the service key. You can follow this cloud foundry command line guide (need only from steps 1 to 5) for reference.
- SAP AI Core (Extended Plan) – A foundational instance should be in place. If not, refer to the official setup documentation under “Enabling the Service in Cloud Foundry”. Be sure to select the extended plan to access generative AI features.
Step 2: Download and import Postman collection
For the SAP Object Store on BTP (AWS), the service provides native AWS S3 buckets as object storage resources. This means there’s no custom API variation—you can interact with the buckets using standard AWS S3 REST APIs and authenticate using the AWS Signature mechanism.
Use the values provided in your Object Store service key to complete the mapping.
Example of REST API that you will be using:
GET - List available objects
PUT - Upload a file object
For SAP AI Core, you can refer to my previous blog post - same section.
Alternatively, you’re free to use your preferred API platform tool—such as Bruno, cURL, or others—based on your workflow and tooling preferences. In this blog, I will be using Postman for demonstration purposes.
Quick Start
REST APIs that we need for this blog are as follows:
- /Object Store > PUT - Upload a file object
- /Object Store > GET- List available objects
- /SAP AI Core /AI Core /Secret > POST /admin/secrets
- /SAP AI Core /Grounding /Retrieval > GET /retrieval/dataRepositories
- /SAP AI Core /Grounding /Retrieval > POST /retrieval/search
- /SAP AI Core /Orchestration /Completion > POST /completion
Step 1: Create a resource group for document grounding
Before we begin, we need a dedicated resource group to organize and manage our document grounding workloads. If you haven’t already created one, please refer to my previous blog post for creating a resource group for document grounding.
Step 2: Create a generic secret
Create a new generic secret at the resource group level. This secret is required to allow the document grounding process to access your Object Store - AWS S3 bucket.
Update the 'AI-Resource-Group' value in the request headers, and include the following JSON payload in the request body. Make sure to add the label key "ext.ai.sap.com/document-grounding" with the value "true", and the label key "ext.ai.sap.com/documentRepositoryType" with the value "S3". These labels are required to enable document grounding capabilities and specify AWS S3 as the repository source for your resource group.
In my example below, I named my generic secret as 'object-store-secret', you can use any unique name of your choice.
{
"name": "<input unique generic secret name, i.e. object-store-secret>",
"data": {
"url": "<Input Base64 encoded value in the format https://s3-<region>.amazonaws.com>",
"description": "<Input Base64 encoded value for the description of the generic secret to be created>",
"authentication": "<Input Base64 encoded value for NoAuthentication, i.e. Tm9BdXRoZW50aWNhdGlvbg==>",
"access_key_id": "<Input Base64 encoded value for access key id>",
"bucket": "<Input Base64 encoded value for AWS S3 bucket name>",
"host": "<Input Base64 encoded value for AWS S3 host>",
"region": "<Input Base64 encoded value for AWS S3 region>",
"username": "<Input Base64 encoded value for AWS S3 credentials>",
"secret_access_key": "<Input Base64 encoded value for AWS S3 credentials>"
},
"labels": [
{
"key": "ext.ai.sap.com/document-grounding",
"value": "true"
},
{
"key": "ext.ai.sap.com/documentRepositoryType",
"value": "S3"
}
]
}Step 3: Upload a PDF document into Object Store - AWS S3 bucket
Use the standard AWS S3 API (e.g., PUT Object) to upload your PDF file to the designated bucket.
Note: You don't need to include /BTP/ in the file path—it's optional. Just make sure your filename includes the correct file extension (e.g., .pdf).
In my example, I’m using a PDF that outlines SAP Joule capabilities as the basis for RAG. If you’d like to follow along using the same document, you can download it here.
Step 4: List all objects in Object Store - AWS S3 bucket (Optional)
To verify the upload or browse contents, use the ListObjects API to retrieve a list of all files stored in the bucket. This helps confirm that your document is ready for grounding.
Step 5: Create a pipeline
This step is essential, as the pipeline defines how content is processed and grounded from your AWS S3 bucket. Also remember to update the value of 'AI-Resource-Group' in the request headers, and use this JSON payload in the request body:
{
"type": "S3",
"configuration": {
"destination": "<enter generic secret name i.e. object-store-secret>"
}
}Once the pipeline is successfully created and triggered, the uploaded PDF will be processed—its content converted into vector embeddings for retrieval. You can verify that the grounding process completed by checking the pipeline execution status.
Step 6: List all data repository objects (Optional)
If you’d like to view all data repository objects available in your SAP AI Core resource group, use the following API: GET /retrieval/dataRepositories
As shown in the response, a new data repository named 'S3' is already provisioned. This built-in integration enables you to run RAG workflows directly on your AWS S3 bucket—providing access to enterprise content without extra configuration. Make sure to set the 'AI-Resource-Group' value in the request headers before making the call. You should see a response similar to the one below.
Step 7: Retrieval search via Vector (Approach #1)
There are two ways to query grounded documents from data repositories—whether the type is 'help.sap.com' or 'vector'. In this scenario, we’ll be working with a 'vector' data repository.
One approach is to use vector retrieval search, which allows you to query the connected repository and receive the most relevant content chunks based on a user-defined query. To do this, use the following API: POST /retrieval/search.
As with previous requests, make sure to set the 'AI-Resource-Group' value in the request headers, and include the following JSON payload in the request body:
{
"query": "<input your query here>",
"filters": [
{
"id": "filter1",
"dataRepositoryType": "vector",
"dataRepositories": [
"*"
]
}
]
}In below example, I’m interested in learning about the capabilities of SAP Joule, so I’ve entered the query: "What are the use cases supported in Joule?" You can also enhance the search with configuration options—for instance, setting "maxChunkCount": 10 to limit the number of returned chunks to 10.
Retrieval search results as follows:
As shown above, the response returns grounded content chunks ready for use. To explore the full grounding schema, you can refer to the official documentation.
Step 8: Retrieval search via Orchestration (Approach #2)
This is another way to query grounded documents via orchestration service. What it does it integrates document grounding configurations and filters to provide precise results.
With the orchestration service, you can interact with LLMs from multiple providers through a unified interface, without changing your application code. Switching models is as simple as updating a configuration parameter, making it easy to compare performance or adapt to different requirements. It also supports advanced features like prompt templating, content filtering, data masking, and grounding for secure and context-aware AI integration.
To get started, you’ll need to create at least one orchestration deployment—or use the default deployment within your resource group. If no deployments are found, please refer to this tutorial for guidance tutorial. Otherwise, use GET /lm/deployments to retrieve list of available deployments.
Next, to perform retrieval search using orchestration service with 'vector' as data repository source, make sure that you update the "deploymentId" URL path variable and update the value of 'AI-Resource-Group' in the request header, then use this example JSON payload in the request body:
Optionally, you can contextualize retrieval further by including metadata in the grounding module’s output. To do this, add the "metadata_params" field in "grounding_module_config" > "config". This field accepts a list of comma-separated strings, allowing you to access specific metadata fields and reference them in your prompt template for more tailored responses. To see available metadata parameters, you can use POST /retrieval/search. Do also note that "metadata_params" not yet available for 'help.sap.com' data repository.
{
"orchestration_config": {
"module_configurations": {
"grounding_module_config": {
"type": "document_grounding_service",
"config": {
"filters": [
{
"id": "filter1",
"data_repositories": [
"*"
],
"search_config": {
"max_chunk_count": 10
},
"data_repository_type": "vector"
}
],
"input_params": [
"groundingRequest"
],
"output_param": "groundingOutput",
"metadata_params": ["title", "source", "webUrlPageNo","webUrl"],
}
},
"llm_module_config": {
"model_name": "<input your llm name, i.e. gpt-35-turbo>",
"model_params": {},
"model_version": "<input your llm version, i.e. latest>"
},
"templating_module_config": {
"template": [
{
"role": "user",
"content": "<input your prompt template here, i.e.: You are a precise and reliable assistant. Using only the provided context, write a concise overall summary, followed by a numbered list of short content summaries with page numbers and URLs as citation. Request:{{ ?groundingRequest }}, Context:{{ ?groundingOutput }}>"
}
],
"defaults": {}
},
"filtering_module_config": {
"input": {
"filters": [
{
"type": "azure_content_safety",
"config": {
"Hate": 2,
"SelfHarm": 2,
"Sexual": 2,
"Violence": 2
}
}
]
},
"output": {
"filters": [
{
"type": "azure_content_safety",
"config": {
"Hate": 2,
"SelfHarm": 2,
"Sexual": 2,
"Violence": 2
}
}
]
}
}
}
},
"input_params": {
"groundingRequest": "<input your search query>"
}
}Some quick explanation:
"input_param": Input variables are dynamic parameters provided to the grounding service and referenced in templates to tailor responses based on user input.
"output_param": The output variable stores retrieved content from the grounding service, which is used to generate relevant and informed responses. "data_repository": You can specify which data repositories to query; if none are selected, all available sources are used by default. In our case, we select all (*).
"template": Templating lets you define prompts with dynamic placeholders (e.g., {{?groundingRequest}}) to customize AI model input during execution.
"llm_model_config": Model configuration allows you to select the generative AI model and set optional parameters in JSON to influence output behavior.
"template_module_config": Configuration for the templating module where you can configure role within the conversation template, e.g., user or assistant and content placeholder for the user prompt.
Orchestration service result as follows:
Essentially what I need is the "message" > "content" from "orchestration_result". If you want to explore the full orchestration schema, refer to this official documentation.
And this is how it looked like using SAP AI Launchpad (please excuse below low-res gif):
Conclusion
With SAP AI Core’s support for document grounding via AWS S3—enabled through the Object Store service on SAP BTP—you can bring your own enterprise content into Retrieval-Augmented Generation (RAG) workflows with minimal setup. This blog walked through the key steps: setting up the Object Store, uploading documents, configuring access, and performing retrieval searches.
By grounding LLMs with your domain-specific content, you enhance the accuracy and trustworthiness of AI responses—laying the foundation for smarter, more context-aware enterprise applications.
For more information on SAP AI Core
- Exchange knowledge: SAP Community
- Learn more: Help Portal | Roadmap | Orchestration
- Explore: Product Information
2 Comments
