Managing Automatic Certificate Management Environment (ACME) in Identity Management (IdM)

The Automatic Certificate Management Environment (ACME) protocol allows automated interactions between certificate authorities and your servers. This means you can automate the deployment of your public key infrastructure at a low cost, with relatively little effort. ACME provides automated identifier validation and certificate issuance, and its goal is to improve security by providing certificates with a short lifespan (3 months by default, in line with the Let’s Encrypt specification), and by avoiding manual (and error-prone) processes from certificate lifecycle management. 

The Let’s Encrypt public Certificate Authority (CA) is by far the most used ACME server. It's a free publicly-trusted CA, and supports a majority of client implementations (they recommend certbot). There are other CAs that implement ACME, including the Dogtag CA, provided by Red Hat Identity Management (IdM). This is a Technology Preview since RHEL 8.4 in IdM, but the upstream project FreeIPA has several articles on the topic. Because the current support level is Technology Preview, we recommend against relying on this feature in production environments. The objective of this article is to introduce the management of ACME with IdM and Red Hat Enterprise Linux (RHEL) clients with mod_md for Apache httpd (the only ACME client implementation completely supported by Red Hat). I also cover new aspects of this feature coming in mod_md in RHEL 9.5 and in the meantime on IdM CA. 

ACME components

As a starting point, I have an IdM server in RHEL 9.4, and a client also in 9.4 joined with the default options:

As an introduction to the protocol, the ACME service provided by IdM CA uses a challenge and response authentication mechanism to prove that a client has control of an identifier. This challenge is a proof of ownership used to obtain a certificate. Specifically, I discuss the http-01 and the dns-01 challenge. The client implementation mod_md implements the http-01, tls-alpn-01 and dns-01 challenge, and IdM understands http-01 and dns-01, so these are the only options I have for the client to prove control of the identifier. This challenge requires the client to provision an HTTP resource. The identifier is authorized when sufficient challenges (usually one for a single identifier) have been validated. Then the client finalizes the order, causing the CA to issue a new key and certificate pair. Finally, the client configures the issued certificate to be used by the application automatically. In this article, I discuss mod_md, the ACME client module for Apache httpd.

In RHEL, the ACME service uses the Red Hat Certificate System (RHCS) PKI ACME responder. The RHCS ACME subsystem is automatically deployed on every certificate authority (CA) server in the IdM deployment, but it does not service requests until the administrator enables it. Additionally, enabling or disabling the ACME service affects the entire IdM deployment. Turning the ACME service on or off is a deployment-wide operation because the configuration is in the replicated LDAP database. The ipa-acme-manage command controls the feature for the whole deployment.

Also, it is only possible to enable ACME on fresh installations of IdM (specifically in IdM RHEL greater than or equal to 9.2, with Random Serial Numbers v3 (RSNv3) enabled. This is documented in the prerequisites, and is due to the pruning mechanism for expired certificates in the Dogtag CA database. ACME runs as a separate service within Apache Tomcat. ACME configuration files are stored in /etc/pki/pki-tomcat/acme, and PKI logs ACME information to /var/log/pki/pki-tomcat/acme/.

Managing ACME in IdM

By default, the service is disabled. When you enable it, the service runs on any and all IdM CA server in your deployment. First, verify the status of the service:

[root@idm ~]# ipa-acme-manage status
ACME is disabled
The ipa-acme-manage command was successful

Enable it:

[root@idm ~]# ipa-acme-manage enable
The ipa-acme-manage command was successful
[root@idm ~]# ipa-acme-manage status
ACME is enabled
The ipa-acme-manage command was successful

It's important to configure a timespan for removal of expired certificates. You can do this with a cron job (for example, on the first day of every month at midnight). The command to do that, and also a typical error, can be the following:

[root@idm ~]# ipa-acme-manage pruning --enable --cron "0 0 1 * *"
Certificate pruning requires random serial numbers
The ipa-acme-manage command failed.

This means that you have not enabled the random serial numbers when installing your IdM on RHEL 9.2 or later. To perform the installation with RSNv3:

[root@idm ~]# ipa-server-install --random-serial-numbers --setup-dns

Or RHEL 9.2 and later, you can only enable the pruning if you installed the IdM with this option. If this is not enabled, the certificates accumulate on the server and the performance is degraded. The feature can still be used without automatic pruning and RSNv3, but be aware that if you issue a large number of certificates, you can manually prune expired certificates from the database to maintain performance.

Here's an example of a fresh installation of an IdM server without random serial numbers:

Instead, if you enable random serial numbers, this is the result:

And with the installation according to RSN, the pruning command works fine:

[root@idmserver ~]# ipa-acme-manage pruning --enable --cron "0 0 1 * *"
Status: enabled
Certificate Retention Time: 30
Certificate Retention Unit: day
Certificate Search Size Limit: 1000
Certificate Search Time Limit: 0
Request Retention Time: day
Request Retention Unit: 30
Request Search Size Limit: 1000
Request Search Time Limit: 0
cron Schedule: 0 0 1 * *
The CA service must be restarted for changes to take effect
The ipa-acme-manage command was successful

The Certificate Retention Time: 30 property is the retention period before pruning an expired certificate. After enabling pruning, you must restart the CA service:

[root@idmserver ~]# systemctl restart pki-tomcatd@pki-tomcat.service

For illustrative purposes, I'm using sequential ascending serial numbers, so that the certificate serial number is easily identified, and I can easily track the lifecycle of certificates.

Managing certificates

Our IdM server is now set up and ready to issue certificates through the ACME protocol. In a future post, I will look at how to configure mod_md in a client to automatically generate a key and acquire a certificate, how to alter the certificate profile in IdM to modify the expiration lifetime of a certificate issued with ACME, and how revoking a certificate automatically causes a reissue of a new certificate.

Because IdM is included in your standard RHEL subscription, you can try to replicate this content in your lab environment without any additional subscription to set up your own ACME environment. If you are not already a RHEL subscriber, get a no-cost trial from Red Hat.