Hey Folks, it's Navya again!
Navigating the vast amount of information in the business world requires both precision and efficiency. Extracting essential insights by summarizing extensive textual data/ latent data can significantly enhance decision-making and operational strategies.
Langchain offers a robust framework to streamline this process, especially when integrated with custom models and Generative AI Hub via the CAP LLM Plugin for anonymizing sensitive data.
In this blog post, we will explore the technical details of using Langchain’s summarization chain, incorporating custom model class, and integrating seamlessly with Generative AI Hub via the cap-llm-plugin. By understanding these tools and techniques, businesses can enhance their intelligence efforts and make more informed decisions.
In the above diagram, we utilize Langchain's summarization chain with its three techniques: stuff, map-reduce, and refine for generating summaries. It supports various chat models such as Open AI, Anthropic, and Llama. However, for handling business data, we need to connect to the Generative AI Hub to utilize LLM models. To achieve this, a custom chat model is implemented where in the "cap-llm-plugin" library is utilized, to directly connects to the Generative AI Hub.
Summarization plays a crucial role in natural language processing (NLP) by distilling extensive volumes of text into concise summaries. Langchain's summarization chain supports 3 techniques i.e. Stuff, Refine, Map-Reduce.
Each technique offers unique advantages and limitations, rendering them appropriate for various use cases.
Let’s delve deeper into each of these techniques:
Pros & Cons:
The chain updates a rolling summary by iterating over the documents in a sequence. In each iteration, the current document and the previously generated summary are passed as prompt for summarization.
Pros & Cons:
Summarization chains can be a game-changer for efficiently extracting key insights by summarizing large volumes of text. LangChain provides implementation support for summarization chain in both Python and JavaScript library.
Langchain offers 3 different chain for previously mentioned summarization techniques: StuffDocumentsChain, MapReduceDocumentsChain, RefineDocumentsChain . It also provides a convenient method “loadSummarizationChain” which returns corresponding chain based on type.
Let’s explore how to implement a summarization chain specifically tailored for SAP environments.
To efficiently manage business data within SAP, connecting to the Generative AI Hub to utilize LLM chat models is essential. The Generative AI Hub offers a robust platform for accessing and leveraging advanced large language models, enhancing our ability to process and summarize extensive business data effectively.
Problem Statement
The summarization chain provided by Langchain connects directly with Open LLM chat models like OpenAI, Anthropic, Llama, and others. However, In SAP BTP context, to access these LLMs in our scenario, we need to go through the Generative AI Hub. The solution is to create a custom model that connects to the Generative AI Hub and use it in Langchain’s summarization chains.
Solution Overview
For this scenario I’ll be creating a custom chat model in JavaScript to connect to the Generative AI Hub using the cap-llm-plugin library. This custom model will enable us to make LLM calls via the Generative AI Hub.
Below is an example of how to create a custom chat model:
const { SimpleChatModel } = require("@langchain/core/language_models/chat_models");
const cds = require("@sap/cds");
class GenAIHubChatModel extends SimpleChatModel {
constructor(params) {
super(params);
}
async _call(messages) {
try {
const capllm = await cds.connect.to("cap-llm-plugin");
let chatResponse = await capllm.getChatCompletion({
messages: this._convertToChatMessages(messages),
});
return chatResponse.content;
} catch (err) {
console.log(err);
return "Error in calling GenAI Hub Chat Model";
}
}
_convertToChatMessages(messages) {
const chatMessages = [];
messages.forEach((message) => {
switch (message.toDict().type) {
case "human":
chatMessages.push({
role: "user",
content: message.toDict().data.content,
});
break;
case "system":
chatMessages.push({
role: "system",
content: message.toDict().data.content,
});
break;
case "ai":
chatMessages.push({
role: "assistant",
content: message.toDict().data.content,
});
break;
}
});
return chatMessages;
}
_llmType() {
return "GenAI Hub Chat Model";
}
async *_streamResponseChunks(messages, options, runManager) {
throw new Error("Streaming of response is not supported!");
}
}
module.exports = { GenAIHubChatModel };
Explanation
To learn more about the cap-llm-plugin library, refer to the following blog: documentation.
To create a custom chat model, you can also follow Langchain’s documentation on “How to create a custom chat model class”
For a hands-on experience of these Summarization chain by creating a custom model in a NodeJs-based CAP application and accessing LLMs via GenAI Hub, you can refer to the following repository: cap-summarization-chain.
const { RecursiveCharacterTextSplitter } = require("@langchain/textsplitters");
const { loadSummarizationChain, SummarizationChainParams } = require("langchain/chains");
const { PromptTemplate } = require("@langchain/core/prompts");
const fs = require("fs");
const utilGenAIHub = require("./utilGenAIHub");
const stuffConfig = { type:"stuff",
prompt:new PromptTemplate({inputVariables: ["text"], template: 'Write a short and concise summary of the following:\n\n\n"{text}"\n\n\nCONCISE SUMMARY:'}) };
const refineConfig = { type:"refine",
refinePrompt:new PromptTemplate({inputVariables: ["existing_answer","text"], template: 'Your job is to produce a final summary\nWe have provided an existing summary up to a certain point: \"{existing_answer}\"\nWe have the opportunity to refine the existing summary\n(only if needed) with some more context below.\n------------\n\"{text}\"\n------------\n\nGiven the new context, refine the original summary\nIf the context isnt useful, return the original summary.\n\nREFINED SUMMARY:'}),
questionPrompt:new PromptTemplate({inputVariables: ["text"], template: 'Write a short and concise summary of the following:\n\n\n"{text}"\n\n\nCONCISE SUMMARY:'}) };
const reduceConfig = { type:"map_reduce",
combinePrompt:new PromptTemplate({inputVariables: ["text"], template: 'Write a short and concise summary of the following:\n\n\n"{text}"\n\n\nCONCISE SUMMARY:'}),
combineMapPrompt:new PromptTemplate({inputVariables: ["text"], template: 'Write a short and concise summary of the following:\n\n\n"{text}"\n\n\nCONCISE SUMMARY:'}) };
const summarize = async function(textDocuments, type) {
let configParams;
switch (type) {
case "stuff":
configParams = stuffConfig;
break;
case "refine":
configParams = refineConfig;
break;
case "map_reduce":
configParams = reduceConfig;
break;
}
const chatModel = new utilGenAIHub.GenAIHubChatModel({});
const summaryChain = loadSummarizationChain(chatModel, configParams);
const textSummary = await summaryChain.invoke({ input_documents: textDocuments });
return textSummary;
}
const getTextDocuments = async function(filePath) {
const textContent = fs.readFileSync(filePath, "utf8");
const textSplitter = new RecursiveCharacterTextSplitter({ chunkSize: 3000 });
const textDocuments = await textSplitter.createDocuments([textContent]);
return textDocuments;
}
module.exports = { summarize, getTextDocuments}
Example:
{
"pageContent": "The Marvel Cinematic Universe (MCU) is a media franchise and shared universe centered around a series of superhero films and television series produced by Marvel Studios. It features interconnected stories based on characters from Marvel Comics, including iconic heroes like Iron Man, Captain America, and Spider-Man.",
"metadata":{"lines": { "from":1, "to":3 } }
}
When working with documents, you might need to transform them to fit your application better. A common transformation is splitting a long document into smaller chunks that can be processed by map-reduce or refine summarization chain. Langchain offers several built-in document transformers to facilitate this, allowing you to split, combine, filter, and manipulate documents as needed.
In this blog post we explored how to streamline business insights using Langchain’s advanced summarization techniques, custom models, and Generative AI Hub integration. By leveraging Langchain’s summarization chain—encompassing stuff, map-reduce, and refine techniques—and connecting with Generative AI Hub via the CAP LLM Plugin, businesses can efficiently extract and secure essential insights from extensive data. This approach not only enhances decision-making and operational strategies but also ensures the effective anonymization of sensitive information, empowering organizations to make more informed and strategic decisions.
At last, I would like to extend my heartfelt gratitude to my colleagues whose insights, support, and collaboration have been invaluable throughout this PoC. So Special thanks to @Ajit_K_Panda for his guidance and @Aryan_Raj_Sinha for his valuable contributions in PoC development and @PVNPavanKumar for their leadership support.
You must be a registered user to add a comment. If you've already registered, sign in. Otherwise, register and sign in.
User | Count |
---|---|
10 | |
10 | |
9 | |
9 | |
8 | |
8 | |
6 | |
6 | |
5 | |
5 |