Using hosted image builder via its API

Image builder is a new hosted Red Hat service for building customized cloud images. In this blog post, I’ll show you how to use it from the command line.

Introducing image builder

Image builder is a hosted service in Red Hat Hybrid Cloud Console allowing you to quickly assemble a customized image. Hybrid cloud solutions can be developed quickly using the service because it can simultaneously upload your image to all the most popular hyperscalers — AWS, Azure and Google Cloud. You can read more about the image builder service in our blog post announcing its general availability. To access it, you can use the no-cost developer subscription — which you can read about in this blog post.

Why would you want to use image builder programmatically?

Although we made the graphical interface as easy to use as possible (see Figure 1), if you are planning to make tens or even thousands of images, you should consider using the image builder API to automate all of your image building needs.

Figure 1: User interface of image builder

Authentication

Image builder uses OAuth 2.0 for authorization. First, you need to generate an offline token for Red Hat APIs on this page. This token cannot be used directly with image builder, so as the second step, you need to exchange it for an access token. You can do that simply using `curl`; just remember to save your offline token to a variable named OFFLINE_TOKEN prior to running this command:

$ OFFLINE_TOKEN=”YOUR_OFFLINE_TOKEN”
$ curl --silent \
    --request POST \
    --data grant_type=refresh_token \
    --data client_id=rhsm-api \
    --data refresh_token=$OFFLINE_TOKEN \
    https://sso.redhat.com/auth/realms/redhat-external/protocol/openid-connect/token \
  | jq .

When you run this command, it should output something like this:

{
  "access_token": "oiZjo1Mjhk...",
  "expires_in": 900,
  "refresh_expires_in": 0,
  "refresh_token": "eyJhbG...",
  "token_type": "bearer",
  "not-before-policy": 0,
  "session_state": "f0dbb8d4-4e4e-4654-844c-6f3704c84422",
  "scope": "offline_access"
}

You can use jq (if you don’t have it installed, run sudo dnf install -y jq) to get the actual access token from the JSON payload and save it in a variable using the following snippet:

$ access_token=$( \
    curl --silent \
      --request POST \
      --data grant_type=refresh_token \
      --data client_id=rhsm-api \
      --data refresh_token=$OFFLINE_TOKEN \

https://sso.redhat.com/auth/realms/redhat-external/protocol/openid-connect/token \
    | jq -r .access_token \
  )

Note that the access token has an expiration time. If image builder returns an authorization error, just rerun the command to get a new one.

More details about Red Hat APIs and OAuth can be found in this article.

Getting the API documentation

To build images, you will use the image builder API. Red Hat Hybrid Console handily embeds documentation for all of its APIs. Image builder’s documentation is available on this page.

This blog post is focused on the CLI though, so let’s also introduce a simple curl command to retrieve the API documentation from the command line:

$ curl --silent \
    --header "Authorization: Bearer $access_token" \
    https://console.redhat.com/api/image-builder/v1/openapi.json \
  | jq .

This command prints the API documentation in the form of an OpenAPI 3 specification encoded in JSON.

Red Hat’s code is open and so is image builder’s, which means that you can also get the specification in YAML format from our upstream repository.

Building a simple KVM guest image

Now we are ready to start building some images. Let’s start with something basic: building an up-to-date RHEL 9.0 guest image for x86_64 CPU architecture that can be used with libvirt or in OpenStack.

Firstly, you need to create a compose request:

$ cat >request.json <<END
{
  "image_name": "My up-to-date guest image",
  "distribution": "rhel-90",
  "image_requests": [
    {
      "architecture": "x86_64",
      "image_type": "guest-image",
      "upload_request": {
        "type": "aws.s3",
        "options": {}
      }
    }
  ]
}
END

The most important fields are the distribution, the CPU architecture and the image type — they are set to build a RHEL 9.0 x86_64 guest image as promised before. The upload request’s type is set to aws.s3, as required by the API. While the image can also be named to your liking, this example names the image “My up-to-date guest image”.

Now that you have the request body prepared, you can send a request to the image builder API:

$ curl --silent \
    --request POST \
    --header "Authorization: Bearer $access_token" \
    --header "Content-Type: application/json" \
    --data @request.json \
    https://console.redhat.com/api/image-builder/v1/compose

If everything went well, you should see output similar to this:

{"id":"fd4ecf3c-f0ce-43dd-9fcc-6ad11208b939"}

Since you are going to need the ID to check the status of the image build, I suggest putting it directly into a variable with a small jq helper:

$ compose_id=$( \
    curl --silent \
      --request POST \
      --header "Authorization: Bearer $access_token" \
      --header "Content-Type: application/json" \
      --data @request.json \
      https://console.redhat.com/api/image-builder/v1/compose \
    | jq -r .id \
  )

If you now look in the Red Hat Hybrid Cloud Console, you should be able to see that your image is building, as shown in Figure 2.

Figure 2: Red Hat Hybrid Cloud Console showing a in-progress image build

Image builds usually take a few minutes, and you can check the status using the following call:

$ curl \
    --silent \
    --header "Authorization: Bearer $access_token" \
    "https://console.redhat.com/api/image-builder/v1/composes/$compose_id" \
  | jq .

When you run this immediately after you created your compose, you will see the call output the following message:

{
  "image_status": {
    "status": "building"
  }
}

After some time, the returned message will change to:

{
  "image_status": {
    "status": "success",
    "upload_status": {
      "options": {
        "url": "https://image-builder-service-production.s3.amazonaws.com/composer-api-76...-disk.qcow2?e42..."
      },
      "status": "success",
      "type": "aws.s3"
    }
  }
}

Hurray, your image is ready! You can now download it using the following command:

$ curl --location --output guest-image.qcow2  \
    “https://image-builder-service-production.s3.amazonaws.com/composer-api-76...-disk.qcow2?e42...”

You can run it locally using libvirt, or upload it to OpenStack. Note that the S3 link currently has an expiration time of six hours, so don’t be surprised if it’s gone the next day.

Building a customized AWS image

The previous example is useful, as it always gives you a fresh and up-to-date image. Let's try something more complex this time though: let’s target AWS, add extra packages into the image, customize the partition layout and embed an activation key so you don't need to worry about subscriptions.

You should already know everything about authentication and the OpenAPI specification, so let's skip directly to the request:

$ cat >request.json <<EOF
{
  "image_name": "My customized AMI",
  "distribution": "rhel-86",
  "image_requests": [
    {
      "architecture": "x86_64",
      "image_type": "aws",
      "upload_request": {
        "type": "aws",
        "options": {
          "share_with_accounts": [
            "438669297799"
          ]
        }
      }
    }
  ],
  "customizations": {
    "packages": [
      "nginx"
    ],
    "filesystem": [
      {
        "mountpoint": "/var",
        "min_size": 10737418240
      }
    ],
    "subscription": {
      "organization": 123456789,
      "activation-key": "toucan",
      "server-url": "subscription.rhsm.redhat.com",
      "base-url": "http://cdn.redhat.com/",
      "insights": true
    }
  }
}
EOF

Let's go over the changes to the request.

Firstly, the distribution was switched to RHEL 8.6 to showcase that RHEL 8.x is also supported. In the image request, the image type was changed to aws which is also reflected in the upload request. Hosted image builder uploads the images to Red Hat's AWS account and then it shares the image with you, so the share_with_accounts field needs to be filled to tell image builder with which account to share the image. Note that the image will go away after 14 days, so if you want to use it for longer, you have to make a copy.

In comparison to the previous request, a big customizations object was added:

• nginx will be preinstalled in the image.

• The root partition will be 4 GiB big. /var will be on a separate, 10 GiB big partition.

• Instances spun up from the image will be automatically subscribed using the given activation key. If you don’t know how to create one, there’s an article about the process.

Let’s send the request:

$ compose_id=$( \
curl --silent \
   --request POST \
   --header "Authorization: Bearer $access_token" \
   --header "Content-Type: application/json" \
   --data @request.json \
   https://console.redhat.com/api/image-builder/v1/compose \
   | jq -r .id \
  )

and start checking its status:

$ curl --silent \
      --header "Authorization: Bearer $access_token" \
      "https://console.redhat.com/api/image-builder/v1/composes/$compose_id" \
  | jq .

Once the image build is finished (it can take a bit longer as import to AWS tends to take a few minutes), you will get the following response:

{
  "image_status": {
      "status": "success",
      "upload_status": {
      "options": {
      "ami": "ami-01f2c869485288e5e",
      "region": "us-east-1"
      },
      "status": "success",
      "type": "aws"
      }
  }
}

If you open the same image in the Red Hat Hybrid Cloud Console, you will see something similar to Figure 3:

Figure 3: Red Hat Hybrid Cloud Console showing a finished AWS image build

Launching your new image in AWS

You can launch your new image using the link in the console. This article is about the command line, though, so let’s use AWS CLI to launch an instance and find its public IP address:

$ aws --region us-east-1 \
    ec2 run-instances \
    --image-id ami-01f2c869485288e5e \
    --key-name KEY_NAME
$ aws --region us-east-1 \
    ec2 describe-instances \
    --instance-ids INSTANCE_ID \
    --query 'Reservations[*].Instances[*].PublicIpAddress' \
    --output text
3.83.39.87

When you SSH into the instance using ssh ec2-user@3.83.39.87, you can confirm that the instance has the requested disk layout, nginx is installed and new packages can be immediately installed because the machine is subscribed:

$ lsblk
NAME              MAJ:MIN RM  SIZE RO TYPE MOUNTPOINT
xvda              202:0    0   15G  0 disk 
|-xvda1           202:1    0    1M  0 part 
|-xvda2           202:2    0   14G  0 part 
| |-rootvg-rootlv 253:0    0    4G  0 lvm  /
| `-rootvg-varlv  253:1    0   10G  0 lvm  /var
`-xvda3           202:3    0  512M  0 part /boot
xvdc              202:32   0  896M  0 disk [SWAP]
$ rpm -q nginx
nginx-1.14.1-9.module+el8.0.0+4108+af250afe.x86_64
$ sudo dnf install tmux
Updating Subscription Management repositories.
Red Hat Enterprise Linux 8 for x86_64 - BaseOS (RPMs)                                  6.7 MB/s |  47 MB     00:07    
Red Hat Enterprise Linux 8 for x86_64 - AppStream (RPMs)                                10 MB/s |  44 MB     00:04    
Last metadata expiration check: 0:00:03 ago on Fri May 27 12:25:55 2022.
Dependencies resolved.
=======================================================================================================================
 Package             Architecture          Version                  Repository                                    Size
=======================================================================================================================
Installing:
 tmux                x86_64                2.7-1.el8                rhel-8-for-x86_64-baseos-rpms                317 k

Transaction Summary
=======================================================================================================================
Install  1 Package

Total download size: 317 k
Installed size: 770 k
Is this ok [y/N]:

Note that this section about AWS CLI might not work for you if your default VPC, default subnet or default security group are configured differently than mine.

Using Ansible to configure the EC2 instance

After your instance is running, you can use Ansible to configure it beyond the capabilities of image builder. In the following example, you will add a simple cron job and set a new hostname. Firstly, you need to create an inventory file (with the IP address you got in the previous section) and a playbook:

$ cat >hosts <<EOF
[webservers]
3.83.39.87
EOF
$ cat >playbook.yml <<EOF
---
- name: Set up web servers
  hosts: webservers
  remote_user: ec2-user

  tasks:
    - name: Log free space
      ansible.builtin.cron:
       name: "log free space"
       hour: "3"
       job: "df -h"

    - name: Set hostname
      ansible.builtin.hostname:
         name: nginx-webserver
      become: true
EOF

Then, you can run the playbook using the following command:

$ ansible-playbook -i hosts playbook.yml

PLAY [Set up web servers] *******************

TASK [Gathering Facts] **********************
ok: [3.83.39.87]

TASK [Log free space] ***********************
changed: [3.83.39.87]

TASK [Set hostname] *************************
changed: [3.83.39.87]

PLAY RECAP **********************************
3.83.39.87           : ok=3 changed=2

As you can see, it’s very simple to start using Ansible with instances deployed from image builder. If you need more information about Red Hat Ansible Automation Platform, check out our documentation here.

Another way to configure your image is via `cloud-init`, and comprehensive documentation for this tool can be found in Red Hat Enterprise Linux documentation.

Conclusion

Hosted image builder can very simply be used via its API. You just need a few curl commands and you quickly get up-to-date and customizable RHEL images directly into a cloud of your choice. Start using it today.