Introduction to Vector Similarity in RAG
Retrieval-Augmented Generation (RAG) techniques significantly improve the contextual relevance of large language model (LLM) outputs by incorporating real-time database lookups. SAP HANA Cloud vector engine plays a key role here, enabling efficient retrieval of pertinent data through vector similarity measures, thereby augmenting the performance of RAG tasks.
This tutorial provides a comprehensive walkthrough, using Python, to showcase the utilization of SAP HANA cloud vector engine in a RAG-based LLM application.
Setting Up: Connecting to SAP HANA
Start by establishing a secure connection to your SAP HANA instance.
from hdbcli import dbapi
# Establish a connection. Replace placeholders with actual credentials.
connection = dbapi.connect(
address="HANA_DB_ADDRESS", # Your database address
port="HANA_DB_PORT", # Typically 443 for HTTPS
user="YOUR_USERNAME", # HANA DB username
password="YOUR_PASSWORD", # Corresponding password
charset="utf-8", # Ensures correct encoding
use_unicode=True # Allows for Unicode characters
)
Creating an Embedding Table
For RAG to work, we need to store document vector representations. This involves creating a table in SAP HANA for embeddings.
cursor = connection.cursor()
# SQL statement to create an embedding table
create_table_sql = """
CREATE COLUMN TABLE embeddings_table (
id INTEGER PRIMARY KEY, -- Unique identifier for each document
document NVARCHAR(5000), -- Text content of the document
embedding REAL_VECTOR(768) -- Vector representation of the document
)
"""
# Executing the table creation SQL command
cursor.execute(create_table_sql)
connection.commit()
cursor.close()
Populating the Table with Document Embeddings
Before retrieval, our table needs populating with document embeddings. This involves defining documents, simulating embedding generation, and batch inserting these into SAP HANA.
Defining Documents
# A curated list of documents for embedding
documents = [
"What is natural language processing?",
"How do vector embeddings work?",
"Examples of machine learning applications.",
"Understanding deep learning for text analysis.",
"The impact of artificial intelligence on society."
]
Generating Embeddings
Simulate LLM calls for embedding generation. In practice, replace this with actual model interactions.
def get_embedding(document):
# Placeholder function to simulate embedding generation
return [float(i) for i in range(768)] # Returns a fixed-size (768) dummy vector
Batch Insertion into SAP HANA
Efficiently insert the document embeddings into the database.
cursor = connection.cursor()
# SQL command template for inserting document embeddings
insert_sql = """
INSERT INTO embeddings_table (id, document, embedding) VALUES (?, ?, TO_REAL_VECTOR(?))
"""
# Iteratively inserting each document and its embedding
for i, document in enumerate(documents, start=1):
embedding_str = str(get_embedding(document)).replace(" ", "")
# Execute insert command for each document
cursor.execute(insert_sql, (i, document, embedding_str))
connection.commit()
cursor.close()
Executing Vector Similarity Searches
With the database prepared, perform a similarity search to find relevant documents for a given query.
Generating Query Embedding
First, generate an embedding for the query.
query = "What is an example of vector similarity search?"
query_embedding = get_embedding(query)
Conducting Similarity Searches
Use L2 distance and cosine similarity measures to find the most relevant documents.
cursor = connection.cursor()
# L2 Distance search
l2_query = """SELECT TOP 5 id, document FROM embeddings_table ORDER BY L2DISTANCE(embedding, TO_REAL_VECTOR(?))"""
cursor.execute(l2_query, (str(query_embedding).replace(" ", ""),))
l2_results = cursor.fetchall()
# Cosine Similarity search
cosine_query = """SELECT TOP 5 id, document FROM embeddings_table ORDER BY COSINE_SIMILARITY(embedding, TO_REAL_VECTOR(?)) DESC"""
cursor.execute(cosine_query, (str(query_embedding).replace(" ", ""),))
cosine_results = cursor.fetchall()
cursor.close()
The l2_results and cosine_results variables in the code snippet above contain the top 5 outcomes of similarity searches performed on the database, ordered from the most relevant to the least relevant.
Conclusion
This tutorial demonstrates how SAP HANA cloud's vector engine can be utilized in a RAG-based LLM application. The approach enhances LLM responses by ensuring that the generated outputs are informed by the most relevant data.
Reference
