Revolutionizing CAP Applications with AI: Introducing the SAP CAP LLM Plugin

[AbhijeetK]

In the evolving landscape of enterprise application development, the integration of artificial intelligence (AI) is no longer just an option but a necessity. SAP's CAP LLM Plugin is at the forefront of this revolution, offering SAP developers a powerful tool to enhance the SAP Cloud Application Programming (CAP) model with generative AI while ensuring the utmost data privacy.

Key Features of the CAP LLM Plugin:

1. Robust Data Anonymization: Utilizing SAP HANA Cloud’s advanced data anonymization techniques, the plugin enables developers to protect sensitive data seamlessly. With just a few annotations, such as @anonymize, developers can implement algorithms like k-Anonymity and l-Diversity, safeguarding data integrity and compliance.

2. Enhanced AI-Powered Functionalities: The plugin connects directly to SAP AI Core, facilitating the generation of vector embeddings and performing similarity searches. This capability is crucial for applications that demand precise and insightful data analysis.

3. Simplified Access to Large Language Models (LLMs): Through SAP AI Core, the plugin offers streamlined access to LLMs for tasks such as chat completion. This feature allows applications to deliver contextually appropriate AI-generated responses, enhancing user interaction.

4. Efficient Retrieval-Augmented Generation (RAG): The plugin optimizes the RAG process by integrating functionalities of both SAP AI Core and SAP HANA Cloud Vector Engine into a single, streamlined method. This integration enables dynamic data fetching and utilization in AI operations, significantly boosting efficiency.

Practical Use Cases:

The CAP LLM Plugin is incredibly versatile, suitable for a wide range of applications across various industries. Whether automating customer service responses, optimizing HR recruitment processes, or ensuring compliance through data anonymization, the plugin provides the tools necessary for developing sophisticated, AI-driven solutions.

Getting Started:

For developers interested in leveraging this technology, SAP offers extensive resources, including documentation and sample cases. These are available in the SAP Samples repository, providing a solid foundation for integrating the plugin into your projects.

Conclusion:

The SAP CAP LLM Plugin is more than just a development tool—it's a game-changer for SAP application development. By marrying AI capabilities with stringent data protection standards, SAP is setting a new standard for enterprise applications. Join us in embracing the future of intelligent and secure enterprise solutions with the CAP LLM Plugin.

For detailed documentation and to dive deeper into the capabilities of the CAP LLM Plugin, visit the SAP Discovery Center.

[Cocquerel]

Thanks a lot for this CAP LLM Plugin, it really simplifies the development.
I would suggest few improvements for function getRagResponse:

- I would prefer the search for similar vectors to be based on the full user chat history and not the last question only. Indeed, let suppose the first question is "What is XXXX ?" and the second is "Where to find it ?". If you only take into account the last question, you will not consider that "it" is referring to "XXXX" and the result is not appropriate.

- I would have preferred that the getRagResponse function to send back in additionalContents, not only the columns score and pageContent but also the column pagemetadata. In my use case, it contains the source URL to the origin documents and I wanted to provide those links to the user together with LLM answer.

 - last point would be to exclude from the vector search, the page contents that were already provided  in previous messages of the conversation. I would prefer to provide the LLM with additional context information instead of repeating the same.

[adamharkus]

I've had to modify the response from the sim search. Doing this make the reponses much cleaner and more useful.

let results = similaritySearchResults;
        results.forEach(item => {
             delete item.TEXT_CHUNK;
             delete item.ID;
             delete item.EMBEDDING;
             item.METADATA_COLUMN = item.METADATA_COLUMN.toString('utf-8');
         });

[kai_merklein]

Thanks for sharing. I wonder whether this plugin is officially provided by SAP, how well it fulfills SAP product standards and whether this open source or closed source. In case it is open source, where is the main repository?

Thanks, Kai

[adamharkus]

I'm thinking the same.

The npm repo cap-llm-plugin - npm (npmjs.com) hasn't been updated for a month.

Other than that, this appears to be very bare bones at the moment, and I've noticed a bug whereby running the rag or sim search needs an addition cds.connect.to('db') in the oninit() class of the plugin.

(see here CAP LLM Plugin needs cds.connect.to('db'). - SAP Community)