Red Hat SummitDevelop, train, test, and deploy ML across the hybrid cloud
Red Hat® OpenShift® AI is an artificial intelligence (AI) platform that allows you to develop, train, and deploy AI models and applications at scale across hybrid cloud environments.
Product overview
OpenShift AI offers organizations an efficient way to deploy an integrated set of common open source and 3rd-party tools to perform artificial intelligence and machine learning (AI/ML) modeling. Adopters gain a collaborative open source toolset and a platform for building experimental models without worrying about the infrastructure. They can then extend that base platform with partner tools to gain increased capabilities. Models can be served to production environments in a container-ready format, consistently, across cloud, on-premise, and edge environments.
OpenShift AI is a part of Red Hat AI. It provides IT operations with an environment that is simple to manage, with straightforward configurations on a proven, scalable, and security-focused platform.
Available as a traditional software product or managed cloud service, OpenShift AI supports popular generative AI (gen AI) foundation models, letting you fine tune and serve these pretrained models for your unique use cases and with your own data. You can even distribute workloads across multiple Red Hat OpenShift clusters, independent of their location. The platform is layered on top of OpenShift and makes it simpler to exploit AI hardware acceleration, supporting central processing unit (CPU) and graphic processing unit (GPU)-based hardware infrastructure, including NVIDIA, AMD, and Intel accelerators.
For those just getting started with gen AI models, OpenShift AI includes the components of Red Hat Enterprise Linux® AI—a foundational model platform to develop, test, and run Granite family large language models (LLMs) to power enterprise applications. In addition to the Granite models provided with Red Hat OpenShift AI, the framework supports models from HuggingFace, Stability AI, and other model repositories.
Table 1. Features and benefits of Red Hat OpenShift AI
Highlights
Lower costs when moving from experiment to production by scaling and automating infrastructure.
Advance AI/ML operational efficiency across teams with a consistent user experience that empowers data scientists, data engineers, application developers, and DevOps teams.
Gain hybrid cloud flexibility by building, training, deploying, and monitoring AI/ML workloads on-premise, in a cloud, or at the edge.
Features | Benefits |
| Model development tooling | Provides an interactive, collaborative interface based on JupyterLab for exploratory data science and model training, tuning, and serving. Data scientists have continuous access to core AI/ML libraries, widely used frameworks, and an extensive array of predefined and customer-provided images and workbenches to accelerate model experimentation. |
| Model training projects | Allow users to organize model development files, data connections, and other artifacts needed for a given project, simplifying experimentation and enhancing collaboration. |
| Model training distributed workloads | Use multiple cluster nodes simultaneously to more efficiently train and tune predictive and gen AI models, providing the scalability to handle tasks that might otherwise be computationally infeasible. |
GPUs and accelerators
| Provide self-service of GPU access by ITOps personnel to predefine their GPU resource environment, both on-premise and in the cloud, for their data scientists and application developers to simplify the selection of configurations for their project tasks. |
| Data science pipelines | Allow data scientists and AI engineers to automate the steps to deliver and test models in development and production. Pipelines can be versioned, tracked, and managed to reduce user error and simplify experimentation and production workflows. |
| Model serving | Serves models from providers and frameworks like Hugging Face, ONNX, PyTorch, TensorFlow, and others as well as popular serving runtimes like vLLM. Cluster resources, such as CPUs and GPUs, can be scaled across multiple nodes as workloads require. |
| Model monitoring | Tracks metrics such as the number of successful and failed inference requests, average inference response times, and compute uses to proactively adjust resources if needed. |
Drift detection | Monitors changes in input data distributions for deployed ML models to detect when the live data used for model inference significantly deviates from the data upon which the model was trained. |
Bias detection | Provides tooling to monitor whether their models are fair and unbiased, based on the training data but also for fairness during real-world deployments. |
AI Guardrails | Filters out hateful, abusive, or profane speech, personally identifiable information, competitive information, or other domain specific constraints. |
Model evaluation | Provides important information on the quality of the LLM model and benchmark the performance of LLM models across a variety of tasks. |
Model registry | Offers a central place to view and manage registered models, helping data scientists share, version, deploy, and track predictive and gen AI models, metadata, and model artifacts. |
Disconnected environments and edge | Simplify the deployment of disconnected clusters on a restricted network, often behind a firewall, supporting industries where security and regulatory compliance require air-gapped deployments. |
In addition to the capabilities provided in OpenShift AI, many technology partner products have been integrated into the user interface (UI). These include Starburst for distributed data access across diverse data sets, HPE for data lineage and versioning, NVIDIA for performance management of GPUs, Intel for high performance inference on Intel hardware, and Elastic for vector database with Retrieval Augmented Generation (RAG) applications.
Next steps:
Find out more about Red Hat OpenShift AI and watch the informative video.
