Red Hat OpenShift AI Expands Predictive and Generative AI Flexibility Across the Hybrid Cloud

Red Hat, Inc., the world's leading provider of open source solutions, today announced advances in Red Hat OpenShift AI, an open hybrid artificial intelligence (AI) and machine learning (ML) platform built on Red Hat OpenShift that enables enterprises to create and deliver AI-enabled applications at scale across hybrid clouds. These updates highlight Red Hat’s vision for AI, bringing Red Hat’s commitment to customer choice to the world of intelligent workloads, from the underlying hardware to the services and tools, such as Jupyter and PyTorch, used to build on the platform. This provides faster innovation, increased productivity and the capacity to layer AI into daily business operations through a more flexible, scalable and adaptable open source platform that enables both predictive and generative models, with or without the use of cloud environments.

Customers are facing many challenges when moving AI models from experimentation into production, including increased hardware costs, data privacy concerns and lack of trust in sharing their data with SaaS-based models. Generative AI (GenAI) is changing rapidly, and many organizations are struggling to establish a reliable core AI platform that can run on-premise or on the cloud.

According to IDC¹, to successfully exploit AI, enterprises will need to modernize many existing applications and data environments, break down barriers between existing systems and storage platforms, improve infrastructure sustainability and carefully choose where to deploy different workloads across cloud, datacenter, and edge locations. To Red Hat, this shows that AI platforms must provide flexibility to support enterprises as they progress through their AI adoption journey and their needs and resources adapt.

Red Hat’s AI strategy enables flexibility across the hybrid cloud, provides the ability to enhance pre-trained or curated foundation models with their customer data and the freedom to enable a variety of hardware and software accelerators. Red Hat OpenShift AI’s new and enhanced features deliver on these needs through access to the latest AI/ML innovations and support from an expansive AI-centric partner ecosystem. The latest version of the platform, Red Hat OpenShift AI 2.9, delivers:

• Model serving at the edge extends the deployment of AI models to remote locations using single-node OpenShift. It provides inferencing capabilities in resource-constrained environments with intermittent or air-gapped network access. This technology preview feature provides organizations with a scalable, consistent operational experience from core to cloud to edge and includes out-of-the-box observability.
• Enhanced model serving with the ability to use multiple model servers to support both predictive and GenAI, including support for KServe, a Kubernetes custom resource definition that orchestrates serving for all types of models, vLLM and text generation inference server (TGIS), serving engines for LLMs and Caikit-nlp-tgis runtime, which handles natural language processing (NLP) models and tasks. Enhanced model serving allows users to run predictive and GenAI on a single platform for multiple use cases, reducing costs and simplifying operations. This enables out-of-the-box model serving for LLMs and simplifies the surrounding user workflow.
• Distributed workloads with Ray, using CodeFlare and KubeRay, which uses multiple cluster nodes for faster, more efficient data processing and model training. Ray is a framework for accelerating AI workloads, and KubeRay helps manage these workloads on Kubernetes. CodeFlare is central to Red Hat OpenShift AI’s distributed workload capabilities, providing a user-friendly framework that helps simplify task orchestration and monitoring. The central queuing and management capabilities enable optimal node utilization, and enable the allocation of resources, such as GPUs, to the right users and workloads.
• Improved model development through project workspaces and additional workbench images that provide data scientists the flexibility to use IDEs and toolkits, including VS Code and RStudio, currently available as a technology preview, and enhanced CUDA, for a variety of use cases and model types.
• Model monitoring visualizations for performance and operational metrics, improving observability into how AI models are performing.
• New accelerator profiles enable administrators to configure different types of hardware accelerators available for model development and model-serving workflows. This provides simple, self-service user access to the appropriate accelerator type for a specific workload.

In addition to Red Hat OpenShift AI underpinning IBM’s watsonx.ai, enterprises across industries are equipping themselves with Red Hat OpenShift AI to drive more AI innovation and growth, including AGESIC and Ortec Finance.

The cloud is hybrid. So is AI.

For more than 30 years, open source technologies have paired rapid innovation with greatly reduced IT costs and lowered barriers to innovation. Red Hat has been leading this charge for nearly as long, from delivering open enterprise Linux platforms with RHEL in the early 2000s to driving containers and Kubernetes as the foundation for open hybrid cloud and cloud-native computing with Red Hat OpenShift.

This drive continues with Red Hat powering AI/ML strategies across the open hybrid cloud, enabling AI workloads to run where data lives, whether in the datacenter, multiple public clouds or at the edge. More than just the workloads, Red Hat’s vision for AI brings model training and tuning down this same path to better address limitations around data sovereignty, compliance and operational integrity.  The consistency delivered by Red Hat’s platforms across these environments, no matter where they run, is crucial in keeping AI innovation flowing.

Red Hat Summit

Join the Red Hat Summit keynotes to hear the latest from Red Hat executives, customers and partners:

• The cloud is hybrid. So is AI. — Tuesday, May 7, 8-10 a.m. MDT (YouTube)
• Optimizing IT for the AI era. — Wednesday, May 8, 8:30-9:30 a.m. MDT (YouTube)

Supporting Quotes

Ashesh Badani, chief product officer and senior vice president, Red Hat

“Bringing AI into the enterprise is no longer an ‘if,’ it’s a matter of ‘when.’ Enterprises need a more reliable, consistent and flexible AI platform that can increase productivity, drive revenue and fuel market differentiation. Red Hat’s answer for the demands of enterprise AI at scale is Red Hat OpenShift AI, making it possible for IT leaders to deploy intelligent applications anywhere across the hybrid cloud while growing and fine-tuning operations and models as needed to support the realities of production applications and services.”

Thomas Taroni, CEO, Phoenix Technologies

“Red Hat’s vision for AI aligns perfectly with our goal to provide organizations with a reliable and sovereign AI solution. With Red Hat OpenShift AI, our kvant AI customers benefit from unparalleled flexibility, scalability and security. With kvant AI in combination with Red Hat OpenShift AI, organizations are perfectly equipped to integrate predictive and generative models effortlessly, empowering everyone to create AI applications with confidence and agility.”

Jeff Fonke, Sr. Practice Manager AI & Data, World Wide Technology

"As legacy IT models wrestle with the complexities of AI, WWT is committed to helping our clients make the right technology decisions faster when it comes to AI adoption. WWT’s AI Proving Ground is a testament to this. With Red Hat OpenShift AI as its foundational platform, the AI Proving Ground enables clients to test, train, validate, and deploy secure AI solutions that deliver real-world business value."

Joris Cramwinckel, Technologist, Ortec Finance

“Red Hat has helped us successfully serve machine learning models for residential housing valuations on Azure Red Hat OpenShift for the past two years and we are excited to implement Red Hat OpenShift AI to replace our maintenance efforts and proprietary code for AI model serving in our applications. Combining the power of OpenShift AI and Azure Red Hat OpenShift will enable our research and development team to accelerate our customer’s transition to AI-based solutions, helping us to more efficiently manage risks and improve investments.”