Skip to main content
Redhat Developers  Logo
  • Products

    Featured

    • Red Hat Enterprise Linux
      Red Hat Enterprise Linux Icon
    • Red Hat OpenShift AI
      Red Hat OpenShift AI
    • Red Hat Enterprise Linux AI
      Linux icon inside of a brain
    • Image mode for Red Hat Enterprise Linux
      RHEL image mode
    • Red Hat OpenShift
      Openshift icon
    • Red Hat Ansible Automation Platform
      Ansible icon
    • Red Hat Developer Hub
      Developer Hub
    • View All Red Hat Products
    • Linux

      • Red Hat Enterprise Linux
      • Image mode for Red Hat Enterprise Linux
      • Red Hat Universal Base Images (UBI)
    • Java runtimes & frameworks

      • JBoss Enterprise Application Platform
      • Red Hat build of OpenJDK
    • Kubernetes

      • Red Hat OpenShift
      • Microsoft Azure Red Hat OpenShift
      • Red Hat OpenShift Virtualization
      • Red Hat OpenShift Lightspeed
    • Integration & App Connectivity

      • Red Hat Build of Apache Camel
      • Red Hat Service Interconnect
      • Red Hat Connectivity Link
    • AI/ML

      • Red Hat OpenShift AI
      • Red Hat Enterprise Linux AI
    • Automation

      • Red Hat Ansible Automation Platform
      • Red Hat Ansible Lightspeed
    • Developer tools

      • Red Hat Trusted Software Supply Chain
      • Podman Desktop
      • Red Hat OpenShift Dev Spaces
    • Developer Sandbox

      Developer Sandbox
      Try Red Hat products and technologies without setup or configuration fees for 30 days with this shared Openshift and Kubernetes cluster.
    • Try at no cost
  • Technologies

    Featured

    • AI/ML
      AI/ML Icon
    • Linux
      Linux Icon
    • Kubernetes
      Cloud icon
    • Automation
      Automation Icon showing arrows moving in a circle around a gear
    • View All Technologies
    • Programming Languages & Frameworks

      • Java
      • Python
      • JavaScript
    • System Design & Architecture

      • Red Hat architecture and design patterns
      • Microservices
      • Event-Driven Architecture
      • Databases
    • Developer Productivity

      • Developer productivity
      • Developer Tools
      • GitOps
    • Secure Development & Architectures

      • Security
      • Secure coding
    • Platform Engineering

      • DevOps
      • DevSecOps
      • Ansible automation for applications and services
    • Automated Data Processing

      • AI/ML
      • Data Science
      • Apache Kafka on Kubernetes
      • View All Technologies
    • Start exploring in the Developer Sandbox for free

      sandbox graphic
      Try Red Hat's products and technologies without setup or configuration.
    • Try at no cost
  • Learn

    Featured

    • Kubernetes & Cloud Native
      Openshift icon
    • Linux
      Rhel icon
    • Automation
      Ansible cloud icon
    • Java
      Java icon
    • AI/ML
      AI/ML Icon
    • View All Learning Resources

    E-Books

    • GitOps Cookbook
    • Podman in Action
    • Kubernetes Operators
    • The Path to GitOps
    • View All E-books

    Cheat Sheets

    • Linux Commands
    • Bash Commands
    • Git
    • systemd Commands
    • View All Cheat Sheets

    Documentation

    • API Catalog
    • Product Documentation
    • Legacy Documentation
    • Red Hat Learning

      Learning image
      Boost your technical skills to expert-level with the help of interactive lessons offered by various Red Hat Learning programs.
    • Explore Red Hat Learning
  • Developer Sandbox

    Developer Sandbox

    • Access Red Hat’s products and technologies without setup or configuration, and start developing quicker than ever before with our new, no-cost sandbox environments.
    • Explore Developer Sandbox

    Featured Developer Sandbox activities

    • Get started with your Developer Sandbox
    • OpenShift virtualization and application modernization using the Developer Sandbox
    • Explore all Developer Sandbox activities

    Ready to start developing apps?

    • Try at no cost
  • Blog
  • Events
  • Videos

Why GPUs are essential for AI and high-performance computing

November 21, 2022
Audrey Reznik Troy Nelson Kaitlyn Abdo Christina Xu
Related topics:
Artificial intelligenceData Science
Related products:
Red Hat OpenShift AI

Share:

    Graphics processing units (GPU) have become the foundation of artificial intelligence. Machine learning was slow and inadequate for many of today's applications. The inclusion and utilization of GPUs made a remarkable difference to large neural networks. Deep learning discovered solutions for image and video processing, putting things like autonomous driving or facial recognition into mainstream technology.

    The connection between GPUs and Red Hat OpenShift does not stop at data science. High-performance computing is one of the hottest trends in enterprise tech. Cloud computing creates a seamless process enabling various tasks designated for supercomputers, better than any other computing power you use, saving you time and money.

    How GPUs work

    Let’s back up and make sure we understand how GPUs do what they do.

    The term, graphics processing unit, was popularized in 1999 when Nvidia marketed its GeForce 256 with the capabilities of graphics transformation, lighting, and triangle clipping. These are math-heavy computations, which ultimately help render three-dimensional spaces. The engineering is tailored towards these actions, which allows processes to be increasingly optimized and accelerated. Performing millions of computations or using floating point values creates repetition. This is the perfect scenario for tasks to be run in parallel.

    GPUs can dominate dozens of CPUs in performance with the help of caching and additional cores. Imagine we are attempting to process high-resolution images. For example, if one CPU takes one minute to process a single image, we would be stuck if we needed to go through nearly a million images for a video. It would take several years to run on a single CPU.

    Scaling CPUs will linearly speed up the process. However, even at 100 CPUs, the process would take over a week, not to mention adding quite an expensive bill. A few GPUs, with parallel processing, can solve the problem within a day. We made impossible tasks possible with this hardware.

    The evolution of GPUs

    Eventually, the capabilities of GPUs expanded to include numerous processes, such as artificial intelligence, which often requires running computations on gigabytes of data. Users can easily integrate high-speed computing with simple queries to APIs and coding libraries with the help of complementary software packages for these beasts.

    In November 2006, NVIDIA introduced CUDA, a parallel computing platform and programming model. This enables developers to use GPUs efficiently by leveraging the parallel compute engine in NVIDIA’s GPUs and guiding them to partition their complex problems into smaller, more manageable problems where each sub-problem is independent of the other's result.

    NVIDIA further spread its roots by partnering with Red Hat OpenShift to adapt CUDA to Kubernetes, allowing customers to develop and deploy applications more efficiently. Prior to this partnership, customers interested in leveraging Kubernetes on top of GPUs had to manually write containers for CUDA and software to integrate Kubernetes with GPUs. This process was time-consuming and prone to errors. Red Hat OpenShift AI simplified this process by enabling the GPU operator to automatically containerize CUDA and other required software when a customer deploys OpenShift with a GPU server. 

    What companies create GPUs?  NVIDIA is the most widely recognized name and also holds the majority of the market share.  But AMD and Intel (Habanna) have also entered the GPU card arena.  

    GPUs in Red Hat OpenShift AI

    Red Hat OpenShift AI (RHOAI) expanded the mission of leveraging and simplifying GPU usage for data science workflows. Now when customers create a project workbench on RHOAI, they have the option to customize the number of GPUs required for their workflow along with Pytorch, TensorFlow and/or CUDA GPU-enabled images.  

    Note:  currently we only support the addition of NVIDIA GPUs, but we are working with AMD to add AMD GPUs.  

    You may be able to select 1 or more GPUs, depending on the GPU machine pool added to your cluster. Customers have the power to use GPUs in their data mining and model processing tasks. 

    Need to train a large dataset or model that may tax a GPU?  Then consider using Distributed Workloads with your GPUs.  ‘Workload distribution refers to the process of evenly allocating tasks or requests among multiple servers or components to optimize performance and efficiency in a computer system.’    

    Why use distributed workloads?

    1. You can iterate faster and experiment more frequently because of the reduced processing time.
    2. You can use larger data sets, which can lead to more accurate models.
    3. You can use complex models that could not be trained on a single node.
    4. You can submit distributed workloads at any time, and the system then schedules the distributed workload when the required resources are available.2

    1 AI generated definition based on: Architecting High Performing, Scalable and Available Enterprise Web Applications, 2015

    For more information on how to configure distributed workloads with Red Hat OpenShift AI check out our Configure distributed workloads instructions.

    Last updated: November 8, 2024

    Related Posts

    • Introduction to machine learning with Jupyter notebooks

    • GPU enablement on MicroShift

    • Boost OpenShift Data Science with the Intel AI Analytics Toolkit

    • More machine learning with OpenShift Data Science

    • 4 reasons you'll love using Red Hat OpenShift Data Science

    Recent Posts

    • How to encrypt RHEL images for Azure confidential VMs

    • How to manage RHEL virtual machines with Podman Desktop

    • Speech-to-text with Whisper and Red Hat AI Inference Server

    • How to use Splunk as an event source for Event-Driven Ansible

    • Integrate vLLM inference on macOS/iOS with Llama Stack APIs

    What’s up next?

    Use the Stable Diffusion model to create images with Red Hat OpenShift AI running on a Red Hat OpenShift Service on AWS cluster with an NVIDIA GPU enabled.

    Use Stable Diffusion to create images on Red Hat OpenShift AI on a ROSA cluster with GPU enabled

    Start learning
    Red Hat Developers logo LinkedIn YouTube Twitter Facebook

    Products

    • Red Hat Enterprise Linux
    • Red Hat OpenShift
    • Red Hat Ansible Automation Platform

    Build

    • Developer Sandbox
    • Developer Tools
    • Interactive Tutorials
    • API Catalog

    Quicklinks

    • Learning Resources
    • E-books
    • Cheat Sheets
    • Blog
    • Events
    • Newsletter

    Communicate

    • About us
    • Contact sales
    • Find a partner
    • Report a website issue
    • Site Status Dashboard
    • Report a security problem

    RED HAT DEVELOPER

    Build here. Go anywhere.

    We serve the builders. The problem solvers who create careers with code.

    Join us if you’re a developer, software engineer, web designer, front-end designer, UX designer, computer scientist, architect, tester, product manager, project manager or team lead.

    Sign me up

    Red Hat legal and privacy links

    • About Red Hat
    • Jobs
    • Events
    • Locations
    • Contact Red Hat
    • Red Hat Blog
    • Inclusion at Red Hat
    • Cool Stuff Store
    • Red Hat Summit

    Red Hat legal and privacy links

    • Privacy statement
    • Terms of use
    • All policies and guidelines
    • Digital accessibility

    Report a website issue