The experience of bringing OpenSSL 3.0 into Red Hat Enterprise Linux and Fedora

Red Hat Enterprise Linux 9 (RHEL 9) ships with OpenSSL 3.0, a core operating system (OS) library that has been in the making for quite a while. This was a long and involved process for a variety of reasons.

What’s the big deal, why wasn’t this just a normal package update?

The previous major stable version was OpenSSL 1.1 and there is no 2.0 in the middle. The OpenSSL team decided to make an explicit jump in numbering to highlight that this new version included major structural, and more importantly, application programming interface (API) and application binary interface (ABI) changes compared to previous OpenSSL versions.

These changes affect applications that use the library in direct ways, sometimes by breaking compilation, and often in subtle ways by changing how the library behaves in some situations.

Due to these differences, changing from OpenSSL 1.1 to OpenSSL 3.0 is not a simple upgrade.

In the Linux ecosystem, OpenSSL is generally considered a core OS library. It is linked directly or indirectly by the majority of the packages of any distribution and this means that the distribution needs to be fully rebuilt to make such a transition.

Why did we break the rawhide first approach?

OpenSSL 3.0 was introduced in CentOS 9 Stream (c9s) before it landed in Fedora rawhide. Ideally we should have rebased to OpenSSL 3.0 in Fedora-rawhide first before the CentOS Stream split, but this was not possible due to the timelines between OpenSSL upstream and Fedora/c9s/RHEL not aligning.

OpenSSL 3.0 development was not suitable for a stable Fedora until OpenSSL upstream released the beta version (which is where ABI is considered stable). The only option left was to introduce OpenSSL 3.0 directly in CentOS 9 Stream / RHEL 9 in parallel to Fedora. Furthermore, between alpha and beta1 versions, there were ABI breakages introduced by upstream which further delayed our ability to land OpenSSL 3.0 in rawhide.

Another factor was that since Fedora is a much larger distribution than CentOS/RHEL, a lot more components would have needed a rebuild. Because OpenSSL 3.0 was still in beta, few upstream projects had patches ready to support a switch to 3.0.

We were glad that we did not rebase a less stable version of OpenSSL 3.0 into rawhide and cause unnecessary pain to Fedora users and maintainers. Although the whole point of Rawhide is to push changes early and get them tested, it is generally frowned upon to push unstable upstream snapshots not intended for release.

The time required to introduce OpenSSL 3.0 and adjust all user space in a distribution was definitely longer than the six month cadence of Fedora, making it particularly challenging to try for a swap starting from scratch in Rawhide.

OpenSSL was brought to fedora-devel after the most critical packages had patches ready to go and already tested in the RHEL 9 alpha/beta builds. 

Why was releasing RHEL-9 with OpenSSL 1.1.1 not an option?

The OpenSSL upstream community indicated that OpenSSL 1.1.1 support would be discontinued relatively quickly after the OpenSSL 3.0 release. Supporting a cryptography and TLS library on our own for the whole lifecycle of RHEL 9 would be a major task with significant staffing requirements.

It would also effectively be a fork — OpenSSL 3.0 changes internal structures considerably, so backports to 1.1.1 are essentially rewrites. With OpenSSL 1.1.1 we can only sustain activities around critical Fixes and Critical CVEs once upstream EOLs.

Due to these considerations, we thought that releasing RHEL 9 with OpenSSL 1.1.1 wasn’t a sustainable course of action. We also had to consider the additional requirements of the new FIPS-140-3 certification.

FIPS-140-3 certifications

On September 31, 2021, the FIPS-140-2 program stopped accepting new modules for validation. That means RHEL 9 will be able to obtain FIPS certifications only under the new FIPS-140-3 program. The new certification requirements are onerous and require several changes in the crypto libraries.

OpenSSL 3.0 changes allow us to reduce the FIPS boundary from the whole of libcrypto and libssl libraries (as currently done in RHEL 8), to just the fips.so provider, a much smaller module that exclusively contains cryptography. Changes to this module are also less disruptive to applications as it sits beneath the OpenSSL public API.

We assessed that applying the necessary changes and certifying the OpenSSL 3.0 fips.so provider would be significantly easier than an OpenSSL 1.1.1 based FIPS-140-3 certification.

API Compatibility and application changes

OpenSSL 3.0 strives to be as API compatible as possible with version 1.1.1, but it is ABI incompatible. While, in most cases, a rebuild is sufficient, several applications need code changes to be fully compatible with OpenSSL 3.0. Language bindings are the most affected as they tend to use a much wider spectrum of the API. To address this, we notified the maintainers of packages that were expected to be most significantly impacted well in advance and provided pre-release builds to assist with porting.

We expected that most of the upstream communities depending on OpenSSL 3.0 would be receptive to patches to add compatibility with OpenSSL 3.0, and that we would need to maintain downstream patches in RHEL 9 only for a relatively short period. We also expected all packagers to contribute those patches upstream as soon as they can be accepted in order to minimize any differences between RHEL 9 packages and upstream versions. We estimated it unlikely that the changes needed in most packages would cause any API or ABI deviation from upstream such that downstream patches would have to be maintained for the lifetime of RHEL 9.

We were right! Most upstream communities accepted patches from maintainers gracefully and ensured that their packages built well with OpenSSL 3.0. In a time frame of about six to nine months (June 2021 - March 2022), almost all packages in RHEL 9 were compatible with OpenSSL 3.0. RHEL 9 beta and RHEL 9 general availability (GA) were both successfully built with OpenSSL 3.0.

Backwards compatibility with OpenSSL 1.1.1

Customers should not be significantly affected, as we provide a compatibility package for binary compatibility with applications built against OpenSSL 1.1.1. This helps customers that build binaries for multiple RHEL versions, and also helps layered products that need time to transition to RHEL 9 and use binary builds that run on multiple product versions.

Note that we do not support building against OpenSSL 1.1.1 in RHEL 9 as no development headers will be provided for the compat package. The compat package is provided for binary compatibility only with binaries built against RHEL 8 and using a limited set of libraries.

For customers that build applications that use OpenSSL we expect that by the time they move to RHEL 9 there will be significant amounts of available documentation and experience built up in open source communities such that guidance to make changes to build their applications against OpenSSL 3.0 will be abundant, readily available and not a significant barrier to adoption.

The name of the compat package is compat-openssl11. It is rebased to the latest (at time of writing) OpenSSL 1.1.1k version. The compat package will be introduced in the buildroot together with OpenSSL 3.0 package. The compat package is available here: [compat-openssl11].

Bottom line

Changing a component like OpenSSL — a dependency for a large number of packages in a distribution — is not an easy task when the change involves API and ABI incompatible changes. Field days tend to be messy and side rebuilds are very work intensive. It may also take a substantial amount of time to go from start to finish — enough to make it difficult to achieve in a single iteration for fast paced distributions like Fedora, unless there is some substantial preparation work done elsewhere.

In this case we did the preparation work during RHEL 9 early development (alpha and beta phases) giving us time to patch most packages and still then have time before GA for final stabilization. In total, it took more than a year to get everything in place, and substantial planning and upstream involvement were needed to successfully achieve our goal of having OpenSSL 3.0 in RHEL 9 and in Fedora.

A big thanks goes to all people that helped the RHEL Crypto team achieve this result — it wouldn’t have been possible without the collaboration of many other teams and their work to implement the changes needed in their packages.

To read more about how OpenSSL 3.0 will affect your migration to RHEL 9, consult the RHEL 9 release notes section on OpenSSL.