A deep dive into deploying Red Hat OpenShift on Oracle Cloud Infrastructure

In my previous blog post, I introduced the integration of Red Hat OpenShift with Oracle Cloud Infrastructure (OCI) and outlined the minimum requirements and key details. In this post, we'll take a closer look at the steps to deploy OpenShift on OCI using the agent-based installer.

Let's dive deep into each of these steps.

Step 1: Get the latest agent-based installer

To begin, be sure you have the latest OpenShift installer binary (version 4.14). Download the latest OpenShift-installer binary from Red Hat Hybrid Cloud Console – for example, here is the agent-based installer to install OpenShift on bare metal locally with agent: https://console.redhat.com/openshift/install/metal/agent-based

Downloading the agent-based installer

Use this procedure to download the agent-based installer and the command line interface (CLI) needed for your installation:

1. Log in to the OpenShift web console using your login credentials.
2. Navigate to Datacenter.
3. Click Run Agent-based Installer locally.
4. Select the operating system and architecture for the OpenShift Installer and command line interface
5. Click Download Installer to download and extract the install program
6. You can either download or copy the pull secret by clicking on Download pull secret or Copy pull secret
7. Click Download command-line tools and place the openshift-install binary in a directory that is on your PATH

Important: Using the Agent-based Install to install a Red Hat OpenShift cluster on OCI is a Developer Preview feature. For more information Developer Preview overview, see developer and Technology Previews: How they compare

You can Request Access to Red Hat OpenShift on Oracle Cloud Infrastructure in Developer Preview form.

You can familiarize yourself with OpenShift with the official documentation:

Installation and update | Architecture | Red Hat OpenShift 4.14

Step 2: Prepare agent-config.yaml and install-config.yaml

Once your infrastructure is in place, prepare the necessary configuration files for the installation process. Modify the "agent-config.yaml" and "install-config.yaml" files to match your specific environment. Be sure that you set the rendezvousIP in "agent-config.yaml" to an IPv4 address from your VCN CIDR, if you are using IPV4 networking. Here's a minimal example for a three-node compact connected IPv4 cluster configuration:

# agent-config.yaml
apiVersion: v1alpha1
metadata:
 name: <CLUSTER_NAME>
 namespace: <CLUSTER_NAMESPACE>
rendezvousIP: <MUST_BE_AN_IP_FROM_MACHINE_NETWORK_VCN_CIDR>
# install-config.yaml
apiVersion: v1
metadata:
 name: <CLUSTER_NAME>
baseDomain: <BASE_DOMAIN>
networking:
 clusterNetwork:
 - cidr: 10.128.0.0/14
   hostPrefix: 23
 networkType: OVNKubernetes
 machineNetwork:
 - cidr: <MACHINE_NETWORK_VCN_CIDR>
 serviceNetwork: 
 - 172.30.0.0/16
compute:
 - architecture: amd64
   hyperthreading: Enabled
   name: worker
   replicas: 0
controlPlane:
 architecture: amd64
 hyperthreading: Enabled
 name: master
 replicas: 3
platform:
 external:
   platformName: oci
   cloudControllerManager: External   
sshKey: <PUBLIC_SSH_KEY>
pullSecret: '<PULL_SECRET>'

Step 3: Prepare cloud control manager custom manifests

Create a new directory named "openshift" and download two Oracle cloud control manager (CCM) custom manifests custom_manifests/manifests/oci-ccm.yml and custom_manifests/openshift/machineconfig-ccm.yml from the oracle-quickstart/oci-openshift GitHub repository into the "openshift" directory. Update "oci-ccm-04-cloud-controller-manager-config.yaml" with your specific settings, including region, compartment OCID, VCN OCID, and subnet OCID. These manifests deploy Oracle’s CCM during cluster installation, so that OpenShift can connect with the external Oracle Cloud Infrastructure platform.

Step 4: Prepare container storage interface custom manifests

CSI manifests allow Oracle's CSI driver to be deployed at install time. CSI custom manifests are under development. For more information, please contact Oracle on their oracle-quickstart/oci-openshift GitHub page or open a support case on the Red Hat Customer Portal.

Step 5: Generate a minimal ISO using the Red Hat OpenShift agent-based installer

With all the configuration and manifest files in place, you can now generate the agent ISO. Assuming you have the "openshift-install" binary downloaded and all files prepared, run the following command:

./openshift-install agent create image --log-level debug

This command generates the agent ISO along with an auth directory containing kubeconfig and kubeadmin passwords and a rendezvousIP file with the IP set in the agent-config.yaml.

Step 6: Create a custom image in OCI

Create a custom image in Oracle Cloud Infrastructure following the steps outlined in the Oracle documentation. You need to upload the agent ISO to a storage bucket and import it as a custom image. Be sure that you configure the image to boot using UEFI.

Step 7: Set up OCI

Before you can install an OpenShift cluster in OCI, you'll need to prepare the required infrastructure in OCI. This includes creating a compartment, setting up virtual cloud networks (VCNs), network security groups, load balancers, DNS records, instances and storage buckets, etc. Make sure that you have appropriate policies and permissions to manage these resources.

Here are some helpful Oracle documentation for these tasks:

• Create a Compartment
• Creating a VCN
• Creating Network Security Groups
• Creating a Dynamic Group and a Policy for Self-Managed Nodes
• Create Policies
• Creating a Load Balancer
• Adding a Record to a DNS Zone
• Creating compute instances

Create the compute instances in OCI based on your desired cluster topology (Single Node OpenShift, Highly Available, or Compact). Be sure to check the memory and disk requirements and select the custom image you created in Step 7. Also, make sure that at least one compute instance has the same IP address as mentioned in the agent-config.yaml file under rendezvousIP.

Once the instances are booted, you can monitor the cluster status using the kubeconfig generated during image creation. Set the KUBECONFIG environment variable and use oc commands to check the cluster nodes and operators.

oc get nodes -A

NAME                                      STATUS    ROLES                           AGE     VERSION
master-0.private.agenttest.oraclevcn.com  Ready     control-plane, master, worker   15m     v1.27.9+e36e183
master-1.private.agenttest.oraclevcn.com  Ready     control-plane, master, worker   15m     v1.27.9+e36e183
master-2.private.agenttest.oraclevcn.com  Ready     control-plane, master, worker   15m     v1.27.9+e36e183

oc get co

Truncated output
NAME            VERSION    AVAILABLE   PROGRESSING   DEGRADED   SINCE    MESSAGE
authentication  4.14.10   True        False         False      6m18s    
baremetal       4.14.10   True        False         False      2m42s    
network         4.14.10   True        True          False      5m58s    Progressing: …

Conclusion

This deep dive provides a comprehensive guide to deploying Red Hat OpenShift on Oracle Cloud Infrastructure using the agent-based installer. Following these steps will install OpenShift in your OCI environment, whether you need a single-node cluster or a highly available multi-node setup.

Learn more

• Red Hat OpenShift agent-based installer support for Oracle Cloud Infrastructure
• Installing a Red Hat OpenShift cluster with the Agent-based Installer
• Red hat OpenShift 4.14 release notes
• Using the Agent-based Installer to install a cluster on OCI