Building CAP Agents for Knowledge Work - Announcing Two Vector-Store CDS Plugins

[merbinjanselm]

A CAP Agent can continue a conversation and, with a long-term Memory Store which I made 2 weeks ago, can remember useful information across conversations. But a support agent also needs to answer questions from product guides, troubleshooting notes, or company policies.

That knowledge already exists in documents. The agent needs a way to find the relevant passages and use them when answering a question.

In this post, let's focus on that retrieval layer. I'm introducing two open-source vector store add-ons for CAP-level Agents:

1. @mi8y/cap-agents-aicore-vectorstore (npm package link & source code), backed by SAP AI Core Document Grounding
2. @mi8y/cap-agents-cds-vectorstore (npm package link & source code), backed by CDS entities in the application's database.

The concept itself is not new, RAG has already past the trend. But the use-cases surrounding it have evolved into Agentic RAG to enable building Knowledge Agents. The above two CDS plugins help build such agents with CAP-level Agents.

We'll use the same knowledge-agent pattern for both: ingest a text file, split it into chunks, retrieve relevant passages, and pass those passages with their source information to the agent.

---

Remembering information and finding information

Conversation state, long-term memory, and document retrieval serve different purposes:

Capability What it provides Example

Checkpointer	Conversation state, resumability, HITL, fault tolerance	Resume an interrupted interaction
Memory Store	Remember facts across conversations	Remember a user's communication preference
Vector store	Knowledge grounding from indexed content	Find the procedure described in a support guide

When a user asks a question, vector-store retrieval finds passages that are semantically relevant. The application then supplies those retrieved passages to the model as context. This is the retrieval part of Retrieval-Augmented Generation, or RAG. For AI Agents which rely on knowledge-based task reasoning, vector-stores provide the necessary mechanisms to retrieve and ground relevant information, while working on tasks.

For example:

User: What does company policy say about data retention?
   [Tool]: Retrieves the relevant section of the company policy from ingested documents
Agent: Answers using that section and identifies the source.

Retrieval gives the model evidence to work with. Its usefulness still depends on the content, chunking, retrieval quality, and how the agent uses the results.

Two ways for CAP-level Agentic RAG

Both packages implement LangChain's VectorStore interface. That gives application code familiar methods such as addDocuments(), similaritySearch(), and asRetriever().

The main difference is where the knowledge base lives and who manages the lifecycle of embeddings and the retrieval.

  AI Core Document Grounding CDS vector store

Implementation	AICoreVectorStore	CDSVectorStore
Content and vector storage	SAP AI Core Document Grounding collection	CDS entities in the CAP application's database
Embedding generation	SAP AI Core Doc Grounding service generates and manage embeddings	Configured LangChain embedding implementation
Retrieval	Grounding Retrieval API	CAP's Vector & cosine similarity operation
Initial setup	Existing collection and AI Core connection	Automatically created CDS entities along with embedding configuration
Useful fit	A managed grounding service	Infrastructure owned and operated within a CAP application

Note that, I have not added specific database requirements for the CDS vector store, because CAP handles database interactions and similarity search internally, abstracting away handling the vector operations.

If you are looking for managed grounding services, my recommendation is SAP AI Core Document Grounding. SAP provides collection, document, and retrieval APIs and helps abstract away a lot of generating and managing embeddings & text as well as complex retrieval operations.

The CDS approach is useful (I prefer for standalone agent implementations) if you'd like to keep the knowledge base closely tied to one application (say for quick PoCs or one-off requirements) and keeping it in that application's persistence makes development and operations simpler. It uses CAP's Vector type and database similarity search; See vector embeddings guide.

Both packages are open-source and MIT licensed. They provide retrieval adapters for CAP applications and can be used with @cap-js/agents.

Two Vector Stores for CAP Agents

---

Getting started with a knowledge agent

The following walkthrough uses CAP Node.js 10 or later, LangChain, and SAP AI Core credentials. Configure the agent's model and AI Core connection for your environment using the official CAP Agents documentation and SAP AI SDK connection guide.

See fully functional examples:

1. Example 1: CAP Agents AI Core Vector Store Example
2. Example 2: CAP Agents CDS Vector Store Example

Create an application if you do not already have one:

cds init my-knowledge-agent
cd my-knowledge-agent
cds add nodejs
npm install @cap-js/agents @SAP-ai-sdk/langchain \
  @langchain/core @langchain/textsplitters langchain zod

Choose one of the following vector-store configurations. In either case, save the configuration as srv/lib/vector-store.js. The ingestion handler and agent tool below import this file.

Option 1: SAP AI Core Document Grounding

Install the adapter:

npm install @mi8y/cap-agents-aicore-vectorstore

Create a Grounding collection separately and configure its embedding model. Follow SAP's Vector API and Pipelines API documentation.

Then connect the adapter to the collection:

import { AICoreVectorStore } from "@mi8y/cap-agents-aicore-vectorstore";

export const vectorStore = new AICoreVectorStore({
  collectionId: "<document-grounding-collection-id>",
  aiResourceGroup: "default",
  destination: { // optional if using a specific AI Core destination
    destinationName: "<aicore-destination-name>"
  }
});

The adapter needs no CDS model for vector persistence and no client-side embedding implementation. SAP AI Core Document Grounding as a managed service in BTP completely take care of storing the supplied chunks, generates embeddings according to the collection configuration, and performs retrieval.

This example (see repository here) prepares chunks in the CAP application and writes them through the Vector API.

Option 2: CDS persistence with SAP AI Core embeddings

Install the CDS adapter and generate its persistence entities:

npm install @mi8y/cap-agents-cds-vectorstore
cds add agent-cds-vectorstore

The command creates db/agent-cds-vectorstore.cds with Documents and DocumentMetadata entities. Deploy them as part of the regular CAP database lifecycle.

Configure the store with an embedding client:

import { CDSVectorStore } from "@mi8y/cap-agents-cds-vectorstore";
import { AzureOpenAiEmbeddingClient } from "@sap-ai-sdk/langchain";

const embeddings = new AzureOpenAiEmbeddingClient({
  modelName: "text-embedding-3-small",
});

export const vectorStore = new CDSVectorStore(embeddings, {
  name: "knowledge-base",
  threshold: 0,
});

Here the SAP AI SDK generates embeddings through SAP AI Core, while content, metadata, and vectors are stored in the CAP application's database.

By default, the CDS entity model uses Vector(1536) column for storing embeddings, matching the default output of text-embedding-3-small. But for higher-dimensional embeddings required for text-embedding-3-large models, you need to adjust the column dimensions accordingly i.e. Vector(3072). Point here is - keep the stored dimension and embedding model configuration consistent. If you change the model or its output dimensions, update the model and re-index existing content as needed. Document and query embeddings must use compatible configurations.

The name separates logical stores in the same tables. This helps to keep different collections logically distinct.

For production, use a supported database configuration such as SAP HANA Cloud or Postgres. For local development, CAP supports vector operations in SQLite too.

Prepare and ingest the documents

For AI Core Document Grounding-based Vector Store setup using the Pipeline API, the API handles the ingestion process, including chunking, embedding generation, and storage for a connected document source such as Microsoft Sharepoint, Object Storage, SAP Document Management service or other supported repositories. Hence, this is applicable only while using Vector API approach for Document Grounding service as well as CDS-based Vector Store.

Your application takes care of extracting/converting to plain text before splitting it into chunks for embedding. Then the vector store takes care of generating embeddings, storing and managing the embeddings.

import { Document } from "@langchain/core/documents";
import { RecursiveCharacterTextSplitter } from "@langchain/textsplitters";

// create a text splitter to divide large documents into smaller chunks
const splitter = new RecursiveCharacterTextSplitter({
  chunkSize: 1000, // adjust based on your requirement
  chunkOverlap: 200,
});
const chunks = await splitter.createDocuments([buffer.toString("utf-8")]);

// map individual text chunk into 'Document' type
const documents = chunks.map(
  (chunk, index) =>
    new Document({
      id: `${fileId}:${index}`,
      pageContent: chunk.pageContent,
      metadata: {
        sourceId: fileId,
        source: file.fileName,
        chunk: index,
      },
    }),
);

// finally embed and add the documents to the vector store
await vectorStore.addDocuments(documents);

Chunk size and overlap are starting values. Evaluate and adjust them based on whether a retrieved passage contains enough information to answer a real question without including too much unrelated text.

For CDS storage, each LangChain document is persisted as a vector document. For Grounding, each non-empty addDocuments() call creates one parent Grounding document containing the supplied chunks. Keep the returned parent document IDs if you need to delete that content later.

Add RAG tool to your agent

Then register a retrieval tool through buildTools:

import cds from "@sap/cds";
import { tool, context } from "langchain";
import { z } from "zod";
import { vectorStore } from "./lib/vector-store.js";

const searchKnowledgeBase = tool(
  async ({ query, limit }) => {
    const documents = await vectorStore.similaritySearch(query, limit);
    if (documents.length === 0) {
      return "No relevant passages were found in the knowledge base.";
    }

    return documents
      .map(
        (document) => context`
      Source: ${document.metadata.source ?? "Unknown source"}
      Passage: ${document.pageContent}
    `,
      )
      .join("\n\n");
  },
  {
    name: "search_knowledge_base",
    description: "Find passages in the knowledge base relevant to a question.",
    schema: z.object({
      query: z.string().min(1),
      limit: z.number().int().min(1).max(10).default(4),
    }),
  },
);

export class AgentService extends cds.ApplicationService {
  init() {
    this.on("buildTools", async (req, next) => {
      const tools = await next();
      return [...tools, searchKnowledgeBase];
    });
    return super.init();
  }
}

The CAP Agent keeps its normal runtime and adds this search tool. The tool code is the same for both configurations because it uses their common text-search interface.

Using CDS-based Vector Store

Using SAP AI Core Document Grounding Vector Store

Note: The adapters are not identical in every operation. Document Grounding approach uses native retrieval filters and server-side embeddings. CDS approach supports its own metadata-filter operators and maximum marginal relevance search. Avoid assuming that scores, filters, or deletion arguments are interchangeable.

---

Choosing an approach for your application

I would start with Document Grounding when a managed grounding service is part of the architecture, especially when the team wants the knowledge base operated separately from individual application databases - imagine you have a Microsoft Sharepoint, and you want to ingest once in a while the agent to search documents stored there without importing them into your CAP application's database.

I would consider CDS storage when the content belongs to one CAP application, the team already operates its persistence, and a direct model of documents, metadata, and embeddings fits the requirement. It is a quick way to get started for a PoC or one-off usage, but its usefulness is not limited to demonstrations. It supports similarity search, metadata filtering, custom persistence entities, configurable vector dimensions, and maximum marginal relevance retrieval.

The agent needs useful evidence at the point where it answers a question. These add-ons provide two ways to retrieve that evidence while keeping the agent integration familiar: SAP AI Core Document Grounding for managed grounding, or CDS persistence for a knowledge base operated with your CAP application.

You can explore the source code and examples, try the approach that fits your application, and share feedback or issues in the repository.