Red Hat at NVIDIA GTC 2024

Speech recognition delivers significant value in the areas of automotive, customer services, and smart home devices. Achieving a high level of accuracy in speech recognition can be challenging. NVIDIA Riva is a GPU-accelerated SDK for building speech AI applications that can be customized to deliver real-time performance. Riva offers pre-trained speech models in NVIDIA NGC that can be fine-tuned on a custom dataset. Models can be exported, optimized, and deployed as a speech service on-premises or in the cloud with a single command using Helm charts. Riva’s high-performance inference is powered by NVIDIA TensorRT optimizations and served using the NVIDIA Triton Inference Server, which are both part of the NVIDIA AI platform. NVIDIA Riva, part of the NVIDIA AI platform, is a GPU-accelerated speech AI SDK for building and deploying fully customizable, real-time AI pipelines that deliver world-class accuracy in all clouds, on-premises, at the edge, and on embedded devices. Our white paper describes a solution for automated speech recognition and inference with NVIDIA GPUs leveraging NVIDIA Riva services on Red Hat OpenShift, along with Dell PowerFlex software-defined infrastructure. (P62076)

Video inference systems are in demand in various use cases, such as smart cities. Energy efficiency gets more important as the number of video streams, the resolution and frame rate of the video, and the amount of calculation required by AI increase. NTT, Fujitsu, Red Hat, and NVIDIA have been addressing this issue based on the “innovative optical and wireless network (IOWN)” concept. We've developed an efficient pipeline utilizing hardware-acceleration technologies and its workload management on geographically-distributed nodes. Through our pipeline, transmission and reception of video data, decoding, and inference can be executed in a way where all data-plane operations are offloaded from CPUs to hardware accelerators such as GPUs, smart NICs, and DPUs. We'll describe the configuration of our efficient pipeline in terms of hardware and software, and show the advantages of our pipeline over a conventional one through experimental evaluations. (P63156)

This session will guide you through the journey of deploying cutting-edge RAG-based applications using NVIDIA AI Enterprise and Red Hat OpenShift AI. Combine the proven capabilities of Red Hat OpenShift AI and Red Hat OpenShift in a single enterprise-ready AI application platform that brings teams together. Outcome: How to leverage Red Hat and NVIDIA's comprehensive suite of AI tools for efficient and scalable LLM application deployment. (EXPT63176)

Erwan Gallen,

Senior Principal Product Manager - Hybrid Platforms, Red Hat

This session dives into the revolutionary combination of Red Hat OpenShift and NVIDIA Grace-Hopper Superchip servers, a groundbreaking AI platform to streamline AI operations and empower organizations to tackle complex AI challenges, and drive innovation faster. Join us for an insightful exploration into the groundbreaking convergence of NVIDIA GH200 Grace Hopper Superchip architecture and OpenShift, the leading open-source container platform. This session delves into the unparalleled performance and productivity gains achievable for giant AI workloads. We’ll explain how you can productize LLM model training and serving with Red Hat OpenShift on the Grace-Hopper Superchip certified server. OpenShift provides a unified platform for deploying, managing, and optimizing AI applications. It offers enhanced performance, scalability, and automation, enabling organizations to streamline and accelerate AI operations. (S63178)

Explore the future of AI infrastructure with a panel of experts from Run:AI, Red Hat, Dell, and NVIDIA. Panelists will share their insights on current and emerging trends in AI infrastructure, emphasizing aspects such as scalability, security, and the integration of AI technologies across various industries. (S63105)