Building Powerful Applications with Weaviate and Red Hat OpenShift: A Retrieval-Augmented Generation Workflow

In the field of artificial intelligence (AI) and machine learning (ML), the goal is to generate applications that give human-like responses to human interaction. One promising approach to achieving this is retrieval-augmented generation (RAG), which combines the strengths of existing methods (specifically, retrieval and generation-based models) to produce accurate and contextually relevant output from a large language model (LLM). This allows an organization’s private data sources to drive the behavior and performance of an LLM as real-time data becomes available. Leveraging the capabilities of Weaviate, an open source, cloud-native vector database, in conjunction with Red Hat OpenShift, your organization can harness the power of a RAG workflow to improve the quality of your products and customer experiences. 

Understanding the retrieval-augmented generation workflow

A retrieval-augmented generation workflow begins by ingesting a pre-existing private knowledge base, typically in the form of unstructured data, into Weaviate. An embeddings model is used to encode the private data into vectors before it is stored in the database. Application queries written in a natural language must also be encoded before performing a similarity search using Weaviate’s SDK. This is all necessary to prepare a context relevant prompt which is then used to perform a generative search using an LLM.

This hybrid approach enhances the quality of generated outputs, grounds them contextually and keeps them aligned with the user's intent. The RAG workflow is appealing to enterprises and allows organizations to take advantage of the power of generative AI while their confidential data sources remain confidential. Additionally, this path avoids retraining or fine-tuning an LLM, a time consuming and often cost prohibitive process.

Features of Weaviate

At the heart of the retrieval-augmented generation workflow lies Weaviate, an open source vector database specializing in understanding and connecting heterogeneous data through semantic relationships. Weaviate's ability to organize data into spatial relationships enables it to capture complex connections between entities, making it an ideal foundation for building intelligent applications that require contextual understanding and semantic relevance.

Key features of Weaviate include:

• Built-in hybrid search: Merge different search algorithms and re-rank the results accordingly
• Fast advanced filtering: Apply complex filters across large datasets in milliseconds
• Native multi-tenancy and Scalability: Weaviate's cloud-native design allows for horizontal scaling and efficient resource consumption, allowing it to handle large volumes of data and user requests easily
• Configurable backups: Back up as often as you need, with zero downtime
• Vector index compression: Improve the memory footprint of large datasets

Key features of Red Hat OpenShift

To deploy and manage intelligent applications built on Weaviate, you can leverage Red Hat OpenShift, the industry’s leading hybrid cloud application platform powered by Kubernetes, that provides a more consistent environment with stronger security features for developing, deploying, and scaling containerized AI workloads.

• Container orchestration: OpenShift simplifies the deployment and management of containerized applications and provides a robust platform to support Weaviate’s distributed architecture options, including replication and sharding for high availability and throughput
• Developer experience: OpenShift offers a more seamless developer experience with integrated CI/CD pipelines, GitOps workflows, and developer tools, enabling rapid iteration and innovation. Developers can take advantage of Weaviate’s robust SDKs that combine semantic and generative search queries needed for creating RAG applications
• Security and compliance: OpenShift includes built-in features such as pod security policies, network policies, and encryption, providing confidence that applications deployed on the platform have an enhanced security posture and are compliant with industry regulations
• Accelerated computing: OpenShift supports hardware acceleration provided by the latest GPUs and advanced CPU SIMD instructions. Weaviate uses these features to speed up distance calculations, allowing your applications to run at peak performance

Use cases for Weaviate and Red Hat OpenShift

By combining Weaviate's and Red Hat OpenShift capabilities, organizations can build powerful applications that leverage the RAG workflow to deliver contextually relevant and intelligent responses to user queries. Whether it's developing virtual assistants, chatbots, recommendation systems, or knowledge management tools, the synergy between Weaviate and OpenShift enables organizations to unlock new possibilities in AI-driven application development.

Here are just a few common use cases for Weviate on OpenShift:

• Enterprise knowledge management: RAG can be a powerful tool for enterprise knowledge management, allowing organizations to efficiently organize, discover, and utilize internal knowledge resources
• E-commerce recommendation systems: By analyzing user behavior and product attributes within a vector database, RAG applications can drive personalized recommendations in e-commerce applications, enhancing the customer shopping experience
• Healthcare data integration: In the healthcare sector, RAG can facilitate the integration of diverse healthcare data sources, enabling comprehensive patient insights and supporting clinical decision-making processes

Get started with an example

Want to learn more?  I've assembled a simple RAG demonstration deployable on a modest OpenShift cluster (similar to the Developer Sandbox for Red Hat OpenShift). The example application ingests a small number of text files containing categories, questions, and answers into Weaviate. The Weaviate SDK handles the vector embeddings and generative search in just a few lines of Python code to complete the RAG application. The source code, sample data, and instructions are available at https://github.com/redhat-na-ssa/demo-ai-weaviate.

Leverage open source AI

The convergence of the RAG workflow with the capabilities of Weaviate and Red Hat OpenShift presents a significant opportunity for organizations to propel their AI-driven applications to new heights.

Whether optimizing enterprise knowledge management, enhancing recommender systems, or facilitating proprietary data integration, the harmony between Weaviate and OpenShift empowers organizations to unlock the full potential of AI-driven innovation, paving the way for a future where intelligent applications thrive.