With generative AI capturing the imagination of users worldwide, enterprises are looking at infusing AI into their products. IBM watsonx is an enterprise-focused artificial intelligence (AI) and data platform.
The watsonx platform enables customers to scale and accelerate the adoption and impact of AI with a full technology stack, AI, trusted data and governance, and flexibility across clouds and on-premises environments by being built on top of Red Hat OpenShift. The watsonx platform has three core components:
- watsonx.ai: An enterprise-grade AI studio that helps AI builders innovate with all the APIs, tools, models and runtimes to simplify and scale the development and deployment of AI applications
- watsonx.data: A fit-for-purpose data store that is open, hybrid and governed – built on open lake house architecture to scale analytics and AI workloads with all your data
- watsonx.governance: An end-to-end toolkit for AI governance across the entire AI lifecycle to accelerate responsible, transparent, and explainable AI for Generative AI and ML models and workflows to ensure organizational accountability for risk and investment decisions
![Image showing how OpenShift, OpenShift AI, watsonx.ai offer the whole tech stack to provide an AI centric portfolio](/rhdc/managed-files/styles/wysiwyg_full_width/private/infusing-ai-into-applications-using-ibm-watsonx-ai-with-red-hat-openshift-ai-img2.png.webp?itok=3GFSxtWN)
Benefits of OpenShift AI
OpenShift AI provides a unified platform for data scientists, application developers and IT administrators for training, serving, monitoring and managing the lifecycle of AI/ML models and applications. It simplifies the iterative process of integrating models into software development processes, production rollout, monitoring, retraining and redeployment.
OpenShift AI leverages the hybrid cloud capabilities of OpenShift, allowing it to be deployed in a hybrid configuration. This means you can deploy your AI platform closer to your data, and overcome the struggles of data gravity. Do you have personal identifiable information (PII) forcing your organization to store some data on-prem? OpenShift AI can easily be deployed on-prem in an air gapped or disconnected configuration, allowing you to build your models where needed while also keeping your data secure. Once completed, your model can be served on-prem, in the cloud, or at the edge.
Because watsonx.ai leverages AI accelerator enhancements in OpenShift AI, Linux kernel solutions have been developed in conjunction with the top AI accelerator companies. Thanks to work by Nvidia, AMD, and Intel, a watsonx AI workload can run at the highest efficiency.
The graphic below explains how OpenShift AI supports a full AI model lifecycle, and provides a detailed view of the technologies used for each key feature.
![Image showing technologies in OpenShift AI](/rhdc/managed-files/styles/wysiwyg_full_width/private/infusing-ai-into-applications-using-ibm-watsonx-ai-with-red-hat-openshift-ai-img1.png.webp?itok=_DG5TWsh)
Workbench
You can interact with your data and train models within a notebook IDE powered by Jupyter Notebook, R Studio or Visual Studio Code. These workbenches come with a variety of popular data science libraries, such as PyTorch or TensorFlow. Need to add your own libraries? No problem. Within OpenShift AI, you can customize the image used for your workbench and install Python libraries or Linux packages within your image. These workbenches can interact with different data storage locations (like S3) and take advantage of distributed training libraries such as CodeFlare and KubeRay.
KServe
A variety of different model runtimes are available within OpenShift AI, powered by the model server KServe. This allows you to bring the runtime you need without being locked into a specific runtime environment. The custom foundational model approach (in which you bring your own foundational model) of OpenShift allows you to serve a variety of predictive and generative AI model types. This means your OpenShift AI cluster can manage and serve any model, allowing it to function as your organization's AI platform. The custom foundational model approach has become a core component of watsonx.ai V2, and it's the KServe model server on OpenShift AI that powers it.
Monitoring and security
OpenShift AI leverages many of the existing functionality of the base OpenShift platform. This includes the monitoring capabilities of the open source Prometheus tool and the RBAC controls that limit the ability of users to services that have been designated to them. These features have been enhanced further to include the ability to monitor the quality and model drift of the models running on OpenShift AI.
Benefits of watsonx.ai
IBM watsonx.ai is an enterprise-grade AI studio for AI builders that helps simplify and scale the development of AI applications with easy-to-use tools and system prompts for building, deploying, and monitoring AI applications. It provides AI developers and model builders with tools, templates, and frameworks to streamline the application development process using high-quality data across the AI application lifecycle.
The watsonx.ai components include:
- Foundational models: Choose from IBM-developed Granite models, open source and third-party models, or bring your own custom foundation model
- Prompt Lab: Experiment with foundation models and build prompts
- Tuning Studio: Tune your foundation models with labeled data
- Data Science and MLOps: Build machine learning models automatically with model training, development, visual modeling and synthetic data generation
Foundational models
Watsonx.ai offers a selection of models encompassing IBM-developed Granite AI models, open source models, models sourced from third-party providers like Mistral AI and Meta’s llama models, as well as custom foundational models.
Granite is IBM’s flagship brand of both open and proprietary large language model (LLM) foundation models, spanning multiple modalities. IBM has released a family of core Granite Code, Time Series, Language and GeoSpatial models as open source software. These are available on Hugging Face under a permissive Apache 2.0 license that enables broad, unencumbered commercial usage. All Granite models are trained on carefully curated data, with industry-leading levels of transparency about the data that went into them. Watsonx.ai offers the ability to bring your own foundational models and deploy them using these instructions.
Prompt Lab
Prompt Lab is a key feature of watsonx.ai that allows AI builders to develop effective prompts for foundation models using prompt engineering techniques. Within the Prompt Lab, users can interact with foundation models in the prompt editor using the Chat, Freeform or Structured mode. These multiple options allow you to craft the best model configurations to support a range of Natural Language Processing (NLP) type tasks including question answering, content generation and summarization, text classification and extraction.
![Image showing sample prompts in watsonx.ai Prompt Lab](/rhdc/managed-files/styles/wysiwyg_full_width/private/infusing-ai-into-applications-using-ibm-watsonx-ai-with-red-hat-openshift-ai-img4.png.webp?itok=uPOMnOWD)
Watsonx.ai Tuning Studio is a tool designed to help users customize and optimize foundation models for specific business needs using techniques like prompt-tuning to help tune your foundation models with labeled data for better performance and accuracy.
Data Science and MLOps
IBM watsonx.ai provides all the tools, pipelines, and runtimes that a business needs to support building ML models automatically. You can automate your entire AI model lifecycle, from development to deployment, with connections to a variety of application programming interfaces (API), software development kits (SDK) and libraries.
Install watsonx.ai on OpenShift
The OpenShift AI repository has detailed instructions for a quick installation of watsonx.ai on OpenShift AI for a proof-of-concept demonstration. You can have a model like Meta’s llama-3 series, Mistral AI, IBM's Granite model, or other commercial or open source or custom models up and running on watsonx.ai on OpenShift AI.
![Image showing pod running IBM Granite series model on OpenShift AI](/rhdc/managed-files/styles/wysiwyg_full_width/private/infusing-ai-into-applications-using-ibm-watsonx-ai-with-red-hat-openshift-ai-img3.png.webp?itok=XXwiz9ri)
Sample application leveraging watsonx.ai
Retrieval-augmented generation (RAG) is an AI framework for improving the quality of LLM generated responses by grounding the model on external sources of knowledge to supplement the LLM’s internal representation of information. You can see a sample application that uses RAG in the example repository contributed on GitHub by Ruslan Magana Vsevolodovna of IBM. In our sample, we used IBM granite-13b-chat-v2 foundational model, but you can use other foundational models in watsonx.ai (such as Meta’s llama-3-70b-instruct, Mistral AI’s Mistral Large 2 model, and so on).
The foundational model can run on Red Hat OpenShift, or can be accessed using the SaaS option. For our demonstration, we had an instance of PostgreSQL with the pgvector extension running on OpenShift. This can be used as a vector store. One of the approaches to integrate large language models into your applications is by using LangChain.
Here's an example response to a RAG query:
from langchain.chains import RetrievalQA qa = RetrievalQA.from_chain_type(llm=watsonx_granite, chain_type="stuff", retriever=db.as_retriever()) query = "What is a vector database?" qa.run(query) "A vector database is a database that can store vectors (fixed-length lists of numbers) along with other data items.”
Learn more about OpenShift AI and download solution patterns at here. If you're interested in learning more about watsonx.ai, you can try watsonx.ai for free in a trial or demo experience.
This post was written with the help of Ruslan Magana Vsevolodovna (IBM).
References
- https://www.redhat.com/en/technologies/cloud-computing/openshift/openshift-ai
- https://www.redhat.com/en/blog/the-power-of-ai-is-open
- https://www.ibm.com/products/watsonx-ai
- https://github.com/rh-aiservices-bu/llm-on-openshift/blob/main/pgvector_deployment/README.md
- https://research.ibm.com/blog/retrieval-augmented-generation-RAG
- https://newsletter.nocode.ai/p/guide-retrieval-augmented-generation
- https://newsletter.nocode.ai/p/tutorial-chat-documents
- https://cdrdv2-public.intel.com/821840/OpenShift-AI-Sol-Brief-V3.pdf
저자 소개
With his experience of being a customer of OpenShift as well as prior experience of working on IBM software, Santosh helps customers with their Hybrid Cloud adoption journey.
Christopher Nuland is a cloud architect for Red Hat services. He helps organizations with their cloud native migrations with a focus with the Red Hat OpenShift product.
채널별 검색
오토메이션
기술, 팀, 인프라를 위한 IT 자동화 최신 동향
인공지능
고객이 어디서나 AI 워크로드를 실행할 수 있도록 지원하는 플랫폼 업데이트
오픈 하이브리드 클라우드
하이브리드 클라우드로 더욱 유연한 미래를 구축하는 방법을 알아보세요
보안
환경과 기술 전반에 걸쳐 리스크를 감소하는 방법에 대한 최신 정보
엣지 컴퓨팅
엣지에서의 운영을 단순화하는 플랫폼 업데이트
인프라
세계적으로 인정받은 기업용 Linux 플랫폼에 대한 최신 정보
애플리케이션
복잡한 애플리케이션에 대한 솔루션 더 보기
오리지널 쇼
엔터프라이즈 기술 분야의 제작자와 리더가 전하는 흥미로운 스토리
제품
- Red Hat Enterprise Linux
- Red Hat OpenShift Enterprise
- Red Hat Ansible Automation Platform
- 클라우드 서비스
- 모든 제품 보기
툴
체험, 구매 & 영업
커뮤니케이션
Red Hat 소개
Red Hat은 Linux, 클라우드, 컨테이너, 쿠버네티스 등을 포함한 글로벌 엔터프라이즈 오픈소스 솔루션 공급업체입니다. Red Hat은 코어 데이터센터에서 네트워크 엣지에 이르기까지 다양한 플랫폼과 환경에서 기업의 업무 편의성을 높여 주는 강화된 기능의 솔루션을 제공합니다.