Red Hat Unites Builders and Operators on the Agentic Future with Major Advancements to Red Hat AI

Red Hat, the world’s leading provider of open source solutions, today announced significant advancements across the Red Hat AI portfolio to help bridge the gap between AI experimentation and production-grade operational control. By delivering a unified, metal-to-agent platform, Red Hat AI 3.4 simplifies the development and deployment of agentic workflows, allowing organizations to move beyond pilots to scalable AI across their entire infrastructure.

By providing a consistent framework for both builders and operators, Red Hat provides a foundation for organizations to scale autonomous systems while maintaining the control, security capabilities and hardware efficiency required by the modern enterprise. 

What is Red Hat AI 3.4? 

Red Hat AI 3.4 is a comprehensive platform that delivers the architectural foundation and operational tools necessary to scale models and agentic workflows across the hybrid cloud. Central to this release is the delivery of Model-as-a-Service (MaaS), which provides a single, governed interface for developers to access curated models while enabling administrators to track consumption and enforce policies. This builds on a foundation of high-performance distributed inference, powered by vLLM and llm-d, to maintain optimized and efficient model serving across a wide range of environments.

While AI agents drive exponential demand for inference, Red Hat AI provides the capabilities for organizations to deploy and manage agents at scale, regardless of agent framework. Newly introduced AgentOps tools manage agents from development to production with integrated tracing, observability, cryptographic identity and lifecycle management. 

To integrate enterprise data with models and agents, Red Hat AI 3.4 introduces prompt management - treating prompts as first-class data assets - and evaluation hub for assessing model and agent accuracy, quality and safety. These capabilities are powered by MLflow, which provides integrated experiment tracking and artifact management for both generative and predictive AI use cases. The platform empowers users to validate model and agent safety with automated safety testing and red-teaming for models and agents, using technology from Chatterbox Labs and the Garak project to provide a security-forward path from experimental pilots to production-ready enterprise utility.

Why does Red Hat AI 3.4 matter? 

The transition from experimental chatbots to production-grade autonomous systems requires a fundamental shift in how IT teams collaborate. Many organizations now recognize the need to move from being merely “token consumers” to "token providers" to better manage costs and power private, sovereign AI use cases. However, the friction between builders and infrastructure administrators remains a primary hurdle to adoption. Without a unified approach that aligns these two roles, infrastructure access barriers slow innovation while "shadow AI" shortcuts introduce ungoverned risks and unpredictable costs.

Red Hat AI 3.4 helps resolve this tension by providing an enterprise foundation for scalable inference and autonomous agent deployments, delivering the transparency and control required to meet rigorous risk and governance standards. Because agents operate with a level of independence, the lack of visibility into their decision-making creates a critical security risk. Red Hat AI addresses this by providing the infrastructure to trace actions, reasoning steps, and tool calls, making it possible to audit how an agent arrived at an outcome. By integrating cryptographic identity management, the platform ties actions to a verified identity, helping identify which entity performed the task. Together, these capabilities move organizations beyond disconnected pilots to treat AI as a scalable, predictable, and, most importantly, accountable enterprise utility.

What Red Hat and partners are saying

“The agentic era represents an evolution of our platform from running traditional applications to powering intelligent, autonomous systems,” said Joe Fernandes, vice president and general manager, AI Business Unit, Red Hat. “We are defining the open standard for how the enterprise executes AI. By providing a hardened, metal-to-agent foundation for AI inference, MaaS and AgentOps, Red Hat provides the operational assurance organizations need to innovate at scale while maintaining rigorous control.”

“CoreWeave's collaboration with Red Hat is grounded in a shared commitment to openness and delivering a high-performance inference foundation that allows enterprises to scale their most complex AI workloads,” said Urvashi Chowdhary, Vice President of Product Management - AI Services at CoreWeave. "Together, we’ve delivered a deployment blueprint for Red Hat AI Inference on CoreWeave Kubernetes Service to run the same inference stack on-prem and in the cloud, with Kubernetes-native control and production-grade performance. This enables enterprise AI teams in regulated industries to focus on the important work: building and scaling AI, not retooling their stack for every new environment.”

"Autonomous, long running agents in the enterprise demand a new level of infrastructure control and security to ensure trustworthy operations at scale," said John Fanelli, Vice President, Enterprise Software, NVIDIA. "Red Hat AI Factory with NVIDIA provides a unified, open source-driven foundation that gives developers and operators the governance and confidence necessary for the agentic future."

Key takeaways

• Scalable, high-performance inference meets governed model access: High-efficiency model inference remains the core of production-ready AI. By combining the vLLM inference server and llm-d distributed inference engine with MaaS, Red Hat AI 3.4 provides a reliable and performant foundation for model inference while simplifying governed model access for users and agents.
• Streamlined AgentOps for the autonomous application lifecycle: Red Hat AI 3.4 introduces comprehensive AgentOps capabilities to help operationalize agents at scale. This includes integrated tracing, observability and evaluations, alongside agent identity and lifecycle management to move agents from development to production.
• Connecting data to models and agents:  Enterprise data is the fuel that powers models and agents. Red Hat AI 3.4 adds prompt management, treating prompts as first-class data assets, and an evaluation hub for managing evaluations across quality, accuracy, safety, and risk. These capabilities are powered by MLFlow, which also provides integrated experiment tracking and artifact management for both gen AI and predictive AI/ML use cases.
• Integrated safety and security for models and agents: To help protect the entire AI stack, Red Hat AI delivers a layered security posture that extends from the operating system to the agentic logic. By providing automated safety testing and red-teaming, organizations can take a data-driven approach to model and guardrail selection and configuration, helping to better protect AI workloads against evolving threats.

Deeper details

• Advanced inference and model optimization: Red Hat AI Inference adds request prioritization to its distributed inference capabilities, allowing interactive and background traffic to share the same endpoint while latency-sensitive requests are processed first under load. Red Hat AI Inference also extends beyond Red Hat OpenShift to additional Kubernetes services, including CoreWeave and Azure, providing organizations a consistent inference stack across environments.  Speculative decoding support, now generally available, improves response speeds by 2x–3x with minimal quality impact while lowering the cost per interaction.
• Governed Model-as-a-Service (MaaS): This feature enables platform engineers to deliver curated, validated models through security-enhanced API endpoints using standard OpenAI-compatible interfaces. This allows for unified governance of both internal models and external APIs, integrated with identity provider (IDP)-based authentication.
• Integrated prompt management: The platform provides unified tools for building and managing prompts as first-class data assets. Storing the inputs driving models and agents in a central registry provides a single source of truth for both developers and administrators.
• Automated evaluations for models and agents: Red Hat AI 3.4 introduces evaluation hub, a framework-agnostic unified AI evaluation control plane for evaluating large language models (LLMs), AI applications and agents. This replaces fragmented testing methods with a unified approach to benchmarking quality, accuracy and risk.
• Multi-layered safety: Automated adversarial scanning is now integrated directly into the development lifecycle. Leveraging technology from Chatterbox Labs, the Red Hat AI platform uses Garak to screen models and agentic systems for risks such as jailbreaks, prompt injections and bias, paired with NVIDIA NeMo Guardrails for run-time safety.
• Production-ready observability: The integration of MLflow provides visibility into agent execution, enabling end-to-end tracing of LLM calls, reasoning steps, tool execution, model responses, and token usage via OpenTelemetry. This creates a transparent audit trail for the entire lifecycle of prompts, embeddings and RAG configurations to support debugging and auditing. MLFlow also provides integrated experiment tracking and artifact management for gen AI and predictive AI use cases.
• Identity-based governance: Using cryptographic identity management (SPIFFE/SPIRE), Red Hat AI enables organizations to replace static hardcoded keys with short-lived tokens. This supports least-privilege operations for autonomous agents across the stack and helps confirm that agentic actions are tied to a verified identity.
• Automated experiences: Tools like AutoRAG and AutoML automate complex AI tasks, ranging from selecting the most effective retrieval strategies for specific datasets to building and optimizing traditional predictive models.
• Hardware flexibility and managed clouds: Red Hat AI 3.4 delivers day-zero support for NVIDIA Blackwell GPUs and AMD MI325X architectures. By extending this unified platform architecture to run natively on third-party managed clouds - including through the new Red Hat AI Inference on IBM Cloud - Red Hat provides operational consistency across a wide range of hardware and cloud providers.

Availability

Red Hat AI 3.4 is expected to be available later this month.

Red Hat Summit

Join the Red Hat Summit keynotes live on YouTube to hear the latest from Red Hat executives, customers and partners:

• The next platform is choice — Tuesday, May 12, 8:30-10 a.m. EDT
• The AI-ready enterprise is here — Wednesday, May 13, 9-10 a.m. EDT