Digitally signing Ansible Content Collections using private automation hub

Red Hat Ansible Automation Platform can manage and execute automation made from many different origins, coming from Red Hat product teams, ISV partners, community and private contributors.

Here is a typical makeup of an automation play that is launched from automation controller:

1. A job template is executed by automation controller and is a playbook.
2. The playbook runs inside of an automation execution environment by the automation controller.
3. The automation execution environment is made using the execution environment builder (ansible-builder tool).
4. When ansible-builder creates the execution environment, it includes dependencies.
5. The dependencies are Ansible Content Collections and their requirements.
6. Collections and their dependencies can be private, community-based, or supplied by Red Hat or its ISV partners.

Previously, there was no way to verify that a Collection downloaded from either Ansible automation hub (console.redhat.com) or private automation hub was developed and released by its original Collection maintainer. This is a potential security issue and breaks the supply chain from creator to consumer.

Providing security-focused features in Ansible Automation Platform 2 continues to be a priority, to enable the execution of certified and supported automation anywhere in your hybrid cloud environment. New in Ansible Automation Platform 2.2  is digital content signing in technology preview, which is the genesis of a more holistic chain-of-custody security feature going forward. It establishes a new framework for a chain of custody for Ansible Content Collections, with a goal for end-to-end digital content signing and distribution. This feature helps ensure that automation being executed in your enterprise is certified and compliant, even if it originates from varied content sources. While this is starting with digitally signing Collections, similar signing capabilities may be available for execution environments in future releases.

In the following sections, we will explain how to enable and consume this new digital content signing feature using Ansible Automation Platform 2.2.

Ansible Automation Platform installer configuration

To successfully sign and publish Ansible Content Collections, private automation hub needs to be configured for signing. The inventory configuration for the installer with signing enabled requires extra key-value pairs in the [all:vars] section like below:

[all:vars]
.
.
.
automationhub_create_default_collection_signing_service = True
automationhub_auto_sign_collections = True
automationhub_require_content_approval = True
automationhub_collection_signing_service_key = /abs/path/to/galaxy_signing_service.gpg
automationhub_collection_signing_service_script =  /abs/path/to/collection_signing.sh

Notice the two keys called automationhub_collection_signing_service_key and automationhub_collection_signing_service_script

• automationhub_collection_signing_service_key- This represents the absolute path to the  private key file from the GPG keypair.
• automationhub_collection_signing_service_script- This represents the absolute path to the signing script that accepts a filename as the only argument. The script needs to generate an ascii-armored detached GPG signature for that file, using the key specified via the PULP_SIGNING_KEY_FINGERPRINT environment variable. The script should then print out a JSON structure with the following format.

{"file": "filename", "signature": "filename.asc"}

How to generate the GnuPG keypair

Below is a sample set of bash commands to set up a key pair and export the associated public and private keys. You can use these directly or change the individual parameters based on your requirements.

cat >gpg.txt <<EOF
%echo Generating a basic OpenPGP key
Key-Type: default
Key-Length: 4096
Subkey-Type: default
Subkey-Length: default
Name-Real: Joe Tester
Name-Comment: with no passphrase
Name-Email: joe@foo.bar
Expire-Date: 0
%no-ask-passphrase
%no-protection
# Do a commit here, so that we can later print "done" :-)
%commit
%echo done
EOF
gpg --batch --gen-key gpg.txt
gpg --output ~/galaxy_signing_service.gpg --armor --export-secret-key
gpg --output ~/galaxy_signing_service.asc --armor --export
rm -rf ~/.gnupg

Sample digital content signing script

Below is a signing script that adheres to the requirements, which you can use as the same script in your setup.

#!/usr/bin/env bash
FILE_PATH=$1
SIGNATURE_PATH="$1.asc"
          
ADMIN_ID="$PULP_SIGNING_KEY_FINGERPRINT"
PASSWORD="password"
          
# Create a detached signature
gpg --quiet --batch --yes --passphrase \
   $PASSWORD --homedir ~/.gnupg/ --detach-sign --default-key $ADMIN_ID \
   --armor --output $SIGNATURE_PATH $FILE_PATH
          
# Check the exit status
STATUS=$?
if [ $STATUS -eq 0 ]; then
   echo {\"file\": \"$FILE_PATH\", \"signature\": \"$SIGNATURE_PATH\"}
else
   exit $STATUS
fi

You can create the above signing script and key-value pair on the installer node for your Ansible Automation Platform and provide the absolute path of the script and private key to the installer as mentioned in the above section. The public key can be used to verify the downloaded collections to initiate a secure chain of custody from the private automation hub.

With the above changes complete,  you can now run the Ansible Automation Platform installer to install your Ansible Automation Platform cluster with private automation hub and digital content signing enabled.

Private automation hub UI/UX changes

Once you deploy a private automation hub with signing enabled to your Ansible Automation Platform cluster, you will see some new UI additions when you interact with Collections. We used two new keys in the deployment configuration as explained in the above sections.

automationhub_auto_sign_collections = True
automationhub_require_content_approval = True

These two configuration parameters solidify that the Collections will get signed and will require approval once they get uploaded to a private automation hub. So, as you upload Collections through the private automation hub UI or via the ansible-galaxy CLI, you will see the Collections gettings queued for approval under the “Approval” tab and the button will now say “Sign and Approve”. Below are some screenshots from the UI.

Once you click the button, you can see the Collection show up with a signed flag on the UI.

You can also pull the Red Hat Ansible Certified Content Collections from the Ansible automation hub at console.redhat.com using the “Repository Management” under the “Collections” tab available on the private automation hub. After a sync, the Collections will show up as unsigned, but you will also get options for Collections and namespaces to sign them from the UI.

Note: Sign community Collections from galaxy.ansible.com using a similar method.

Installing signed Collections from private automation hub

Using the ansible-galaxy CLI (available with ansible-core v2.13), installed with Ansible Automation Platform 2.2, can now download and verify signed Collections from the private automation hub. You can check the upstream documentation for more information on how to use signatures with ansible-galaxy CLI.

Using our example above, we will explain how you can use the public key to verify signed Collections from private automation hub.

To add the public key to a local non-default keyring:

gpg --import --no-default-keyring --keyring ~/keyring.kbx galaxy_signing_service.asc

Use the keyring created in the last step with ansible-galaxy CLI to install and verify the signed Collections.

ansible-galaxy collection install community.lab_collection --keyring ~/keyring.kbx -c -vvvv

You will see the signature verification happening to validate the source of this Collection. Verification can also be done on the installed Collections to check if they have been tampered with. All these features enable a secure content supply chain for Collections from private automation hub to a user’s system. We have also created a self-paced lab that explains the digital content signing and verification workflow in more detail.

Create automation execution environments with signed Collections

With ansible-builder version 1.1.0 that comes with Ansible Automation Platform 2.2, it is possible to supply the keyring as explained in the previous section. If this option is not provided, no signature verification will be performed. If it is provided, and the version of Ansible in the base execution environment is not recent enough (ansible-core < 2.13), an error will occur in the image build process.

ansible-builder create --galaxy-keyring=/path/to/pubring.kbx
ansible-builder build --galaxy-keyring=/path/to/pubring.kbx

Conclusion

• Digital content signing is a technology preview feature in private automation hub in Ansible Automation Platform 2.2
• The ansible-galaxy CLI performs Collection signature verification from private automation hub, which helps build a chain-of-custody for automation consumers in your organization.
• The ansible-builder tool bundled with Ansible Automation Platform 2.2 can also verify signatures based on the version of Ansible present in the base automation execution environment.
• Execution environment signing and signature verification in the automation controller is anticipated for future platform releases.

Next Steps

Collection signing self-paced lab

Try the self-paced lab designed by us on digitally signing content collections with a private automation hub, which explains the concepts touched upon in this blog in detail by using working examples.

More Ansible Automation Platform 2.2 resources

Read the blog on Ansible Automation Platform 2.2. You can get a rundown of what’s new in this checklist, What’s new: Ansible Automation Platform 2.2. For additional context, including a recap of Ansible Automation Platform 2 releases to date and a look ahead at 2.3, check out the free, on-demand webinar, Ansible Automation Platform 2.2: next generation platform enhancements.

Take a video tour 

This eight-minute overview video highlights the components and features in the latest version of Ansible Automation Platform, and how they come together to deliver a comprehensive enterprise automation experience.

Try Ansible Automation Platform

Get hands-on with our other self-paced, on-demand labs. These interactive learning scenarios provide a preconfigured Ansible Automation Platform environment where you can experiment on how the platform can help you solve real-world challenges.

Sign up for an Ansible Automation Platform 60-day trial to try it in your environment.

Plan your upgrade

If you are still operating Ansible Automation Platform 1.2, it is time to start planning your upgrade. 

• Get started with this checklist, “5 ways to prepare for migration to Ansible Automation Platform 2.” 
• Register for a free webinar, “Migrating to the next generation IT automation platform.”