What you need to know about the first-ever DNSSEC root key rollover on October 11, 2018

DNSSEC is a system of digital signatures that prevent DNS spoofing. Using DNSSEC, it does not matter where your DNS answers came from, since the DNS resolver or application can verify the DNSSEC signatures to ensure the DNS data is not tampered with..

DNS is hierarchical, which means that the parent zone vouches for the cryptographic key used by its children via Delegation of Signing (DS) records. At the top of the hierarchy stands the DNSSEC Root Key. This key was first deployed on July 15, 2010, and it is scheduled to be replaced with a fresh new key on October 11, 2018 at 16:00 UTC.

What do you need to know?

If all goes well, end users and operators will notice absolutely nothing. The DNS community coordinated with the Internet Engineering Task Force (IETF), Internet Corporation for Assigned Names and Numbers (ICANN), DNS vendors, operating system vendors and DNS operators to ensure this change will be as uneventful as possible.

But there might be a few old forgotten and unmaintained servers, virtual machines, or containers that will run into issues if these servers had enabled DNSSEC more than a year ago and were not updated since that time.

How do DNS software and DNS services pick up the new key?

It already has! Properly working software should have already picked up this new key. To update the DNSSEC Root Key, a process defined in RFC 5011 is used. It involves pre-publishing the new key signed by the current key and when you have seen this new key for more than 30 days, trust the new key as much as the current key.

You can see the procedure as implemented by ICANN in this PDF about their Operational Implementation plan.

Red Hat has taken steps to pre-configure DNS software we ship with the new key. This means you would not need to wait 30 days before your machine trusts the new key. So, if you are using bind, unbound, knot, or dnsmasq - and your machine is up to date - you do not need to do anything. You don't even need to restart any DNS service.

Check if you have the new DNSSEC Root Key installed

The current DNSSEC Root Key (also called KSK-2010) has Key ID 19036. The new DNSSEC Root key (also called KSK-2017) has Key ID 20326. You can see if the configuration includes the new DNSSEC Root Key:

- bind - see /etc/named.root.key

- unbound / libunbound - see /var/lib/unbound/root.key

- dnsmasq - see /usr/share/dnsmasq/trust-anchors.conf

- knot-resolver - see /etc/knot-resolver/root.keys

Note that this has been included for at least a year now, so you do not need to restart your DNS service unless your DNS server has been running for more than a year and does not support RFC 5011. The only known DNS software that matches these criteria is dnsmasq. ICANN has documented in great detail on how to check your DNS resolver's current trust anchors.

Fallback plan in case of unexpected problems

Again, it is not expected that any DNS issues will happen. But if they do, it is recommend first to simply try restarting your DNS server. Then try to resolve something with DNSSEC, for example by using dig +dnssec dnskey . and if that works, you should be good, although you might want to keep monitoring the situation for a little while longer.

If you still see that DNS is not working properly you can temporarily switch to a public DNS operator. These DNS operators run DNSSEC-enabled public resolvers. You can switch to one of these services, or one of your preference, by configuring these public DNS services in /etc/resolv.conf. We don’t endorse any of these in particular, but they are well-known public DNS providers that support DNSSEC and may be useful if you need a working DNS service quickly.

Note that if you block DNS queries to the internet, you might have to add a firewall rule so these queries can reach the public DNS servers. Ensure to add both UDP and TCP port 53, since TCP is needed to reliably receive larger DNSSEC replies.

Installations that might fail DNS after the rollover

The IETF and ICANN worked on measuring the (pre)adoption rate of the new Root Key. RFC 8145 describes a method, implemented in common DNS software such as bind and unbound, that will report to the root zone which DNSSEC Root Keys are accepted whenever it needs to refresh root zone DNS data.

This shows that about 5% of DNS (recursive) query source IPs have been updated recently to include support for RFC 8145 while not having the new DNSSEC Root Key enabled. You can check if any of your IP addresses are in this bad DNS servers IP list.

It is not known why this happens at all, as any software with RFC 8145 support also ships with the current and new DNSSEC Root Key. There are a few theories and it is worth thinking about whether any of your deployments might fall in this category.

Custom old configuration files with up to date software

If this is on a RHEL, CentOS, or Fedora machine, check for .rpmsave or .rpmnew files. If shipping VM images or containers with specific (year old) configuration files, update these container images as soon as possible.

If you have a named.conf configuration, with a trusted-keys{} statement you must do two things. First, replace trusted-keys{} with a managed-keys{}. This will enable RFC 5011 trust key rollover support for the future. But for this rollover you no longer have 30 days for the hold timer, so you must manually add the new DNSSEC Root Key to the managed-keys{} section and/or add the key to /etc/named.conf. You can check the .rpmnew file or grab an up to date /etc/named.conf from another system. Without manual intervention, these systems will start failing DNS on October 11.

Incomplete RFC 5011 process

Another theory is that there are containers or virtual machines with old configuration files and new software, and on launch these machines would only have the current (soon to be removed) DNSSEC Root Key. This is logged via RFC 8145 to the root nameservers which provides ICANN with the above statistics. Then the server updates the key via RFC 5011, but it is shut down before the 30 day hold timer has been reached. Or the hold timer is reached and the configuration file cannot be written to. Or the file could be written to, but the image is destroyed on shutdown/reboot and a restart of the container starts again from the old golden image that does not contain the new key.

If these machines have been up for 30 days on October 11 2018, they will trust the new key and DNS will keep working. But as soon as these machines are rebooted to the old golden image they will start failing because the bootstrap can no longer work, since the only DNSSEC Root Key these machines trusted will have been removed from the root zone on October 11, 2018.

Current information during the rollover event

To get the latest information published by ICANN, see their Rollover Resources Page. That page will be updated during the event in case of unexpected issues.