Use Amazon OpenSearch Serverless for RAG

In the dynamic world of AI advancements, Retrieval Augmented Generation (RAG) is a breakthrough that’s reshaping how we interact with Large Language Models (LLMs). If you’re new to the concepts of LLMs or RAG, it’s worth exploring how this innovative approach can revolutionize applications in various fields, such as media, finance, and healthcare, just to name a few. RAG combines the deep learning capabilities of LLMs like GPT-4 or models like Claude 2, Llama 2, and many more that are available through Amazon Bedrock, with an extensive external database, offering a level of contextual understanding and depth that the models are lacking out of the box.

For a newspaper or media company, staying ahead with accurate, comprehensive, and timely information is crucial. RAG, leveraging models like those available on Amazon Bedrock, for example, can enhance content creation and fact-checking processes. Journalists can access historical data, past articles, and reference material, ensuring contextually relevant and factually correct information in their reporting. In finance, RAG can be a game-changer by creating highly accurate market analysis reports. Similarly, in the healthcare industry, RAG can significantly enhance medical research. Integrating the latest clinical trial data with existing medical literature aids in producing more informed and accurate research outcomes, a critical factor in medical advancements.

In the context of RAG, a vector database becomes essential. Unlike traditional databases, vector databases are built for storing high-dimensional data that could be easily “consumed” by LLMs. In the realm of vector databases, there are multiple options out there like Pinecone, Milvus, Chroma, or Amazon OpenSearch, just to name a few. I would like to highlight Amazon OpenSearch with its Serverless capability. It stands out for its scalability and ease of management, making it a practical choice for various industries. Its serverless nature reduces maintenance burdens, allowing organizations to concentrate more on their core operations. It inherits the robust security and reliability features of AWS, ensuring data is securely stored and reliably accessible, which is paramount for sensitive applications. Another huge plus is it’s compatible with a wide range of AWS services, making it easier to integrate into existing AWS-based architectures. This compatibility is particularly useful for businesses already invested in the AWS ecosystem.

While there are a lot of examples and tutorials out there on how you can utilize OpenSearch as your Vector DB to build question answering bot, serverless RAG demo with Amazon Bedrock, and many more, the purpose of this quick solution is to allow you to quickly spin up AWS infrastructure and play around in a Jupyter notebook indexing and querying an OpenSearch Serverless Vector DB feature. In this notebook, we will explore an advanced architecture known as Retrieval Augmented Generation (RAG) for handling complex enterprise-level question-answering tasks. This approach involves processing large documents, creating numerical vector representations of document chunks, and then indexing these for efficient retrieval.

Deployment

Resources to be provisioned within your account by following this example:

• • Amazon OpenSearch Serverless collection
  • Amazon SageMaker Jupyter Notebook instance
  • IAM role with S3 Full Access and SageMaker Full Access, Bedrock Full Access, access to the OpenSearch collection that was provisioned for SageMaker Jupyter Notebook and an API BatchGetCollection call against OpenSearch Serverless collections.

 

Impotant Note: Jupyter notebook as well as assets related to it (like images and dependencies) that are included and being used in this example were cloned and modified from the official aws-samples repository (see original notebook here).

This tutorial assumes some prerequisites in the form of installed dependencies so make sure you install them before you proceed (if you don’t have them already installed on your machine):

1. node >= 16.0.0
2. Python >= 3.11
3. pip >= 23.3.1
4. AWS CLI >= 2.0.0
5. AWS CDK >= 2.66.1
6. AWS CLI credentials setup with Administrator Access

Clone this GitHub repo by running the following command in your terminal/shell:

From the root folder execute the following command to create a virtual environment:

Note: Windows users, skip the next command.

If you are a MacOS/Linux user, execute the following command to activate the virtual environment:

Note: MacOS/Linux users, skip the next command.

If you are a Windows user, execute the following command to activate the virtual environment:

Execute the next command to install the required dependencies:

If you are deploying your infrastructure for the first time, bootstrap your account:

Finally, deploy the infrastructure:

Due to the lengthy time it takes to spin up the OpenSearch collection and Notebook Instance, the whole deployment can take anywhere around 10–15 minutes.

After everything is successfully provisioned, navigate to the Amazon SageMaker console (make sure you select the right region where you provisioned your infrastructure). On the left panel and under Notebook section select Notebook instances.

 

In the opened window select Open JupyterLab next to the instance that was created. It should be named in the form SagemakerNotebookConstruct with a random appendix after it.

You should be redirected to the JupyterLab environment inside your Amazon SageMaker Notebook Instance. On the left panel open the opensearch-embeddings-cdk folder. Inside that folder open the notebook folder.

 

You should see a Jupyter Notebook named RAG_Claude_Opensearch.ipynb. Double-click on the file to open a notebook. You should be prompted to select a Kernel that would be used for your notebook. Select conda_python3 as a Kernel.

 

 

Follow the instructions in the notebook (link to the notebook that was cloned) where you will create an embedding for the question, find relevant document chunks by comparing embeddings, and use these chunks to augment the prompt sent to a language model, in this case, Anthropic Claude V2 through Amazon Bedrock. This method allows for contextually rich answers based on extensive documentation, such as IRS documents, making it suitable for users without specialized knowledge in the field. The implementation leverages the LangChain framework, integrating services like Amazon Titan Embeddings and Amazon OpenSearch, to build a comprehensive system for effective question-answering.

Conclusion

To stop incurring any charges and remove all of the provisioned infrastructure, run the following command in the root directory of a cloned repo:

I hope this demonstration showed how you can utilize Amazon OpenSearch Serverless as your Vector DB of choice for use cases that require RAG implementation. If you have any questions or just want to chat about it, feel free to reach out to me on LinkedIn or in the comments. Happy 2024 everyone!

I work for AWS but all of the opinions, ideas and solution implementations in my blogs are purely of my own.