DNS problems with .de domains: DENIC provides initial explanation

Contents

On May 5th at 9:43 PM, the German part of the internet experienced issues: Anyone trying to access addresses with the .de extension received only an error message – at least if the DNS server used validated DNSSEC signatures. The problem was not fully resolved until after 1 AM. DENIC eG, which manages these domains, is responsible for the DNSSEC configuration of the .de zone. They have now provided explanations for the issues.

From DENIC's blog post, it is evident that the DNS servers use the software Knot, an open-source server service maintained by the Czech domain administration organization CZ.NIC, administrator of the .cz domain. At DENIC, Knot runs with “in-house developments in combination with Hardware Security Modules (HSMs)”. In April 2026, this infrastructure was upgraded to the third generation.

Key Collision

On May 2nd, the scheduled rotation of the Zone Signing Key began. A new public key with ID 33834 was published – 3 days before it was first used for signing. What no one suspected at the time: An error in the self-developed part of the code caused three key pairs with the tag 33834 to be generated on the servers, but only one public key was published under this number. When this key was first used to sign SOA records, the peculiar error pattern emerged, which can be traced with a tool like dnsviz.com: Only some of the six DENIC name servers used the correct private key that matched the public key, while others consistently delivered invalid signatures. The SOA record is frequently changed because the serial number in this record is modified with every change to the zone. This aligns with the back-and-forth that we already observed in a detailed analysis of the available data.

DENIC officials state that a code section in the custom-built software “was not fully covered by the test scenarios and therefore was not detected as defective during test runs or in 'cold' parallel operation before commissioning” and further emphasize that they apply three checking tools to the zone, all of which signaled issues – “however, the messages were not processed correctly.”

The explanation does not yet provide complete clarity. The code for the in-house developments is not open-source, the Hardware Security Module used is not named, and the conditions under which the key collision occurred exclusively in the production system and not in test environments are not explained. At least they promise to provide further information once the investigation is complete.

All Domains Affected

In the blog post, DENIC also corrects the statement from the night that only domains with DNSSEC active were affected. This claim did not align with observations during the outage. It is correct that, in addition to the SOA record, NSEC3 records were also affected. NSEC3 prevents an attacker who can intercept traffic from easily circumventing DNSSEC by sending the response “DNSSEC is deactivated for this domain.” NSEC3 is a cryptographic proof of non-existence. Without valid NSEC3 records, the DNSSEC check failed for all .de domains.

Conclusion

The case is not yet closed, as DENIC itself writes. In addition to a precise explanation of the problem (ideally including code), DENIC has not yet published a list of planned measures on how such a problem could be detected in the future. Such findings would benefit not only operators of DNSSEC infrastructure for domains but also other TLD operators. Problems with key collisions at the Toplevel Domain level are not new: in 2024, the Russian operator of the .ru TLD experienced an outage, which was caused by a key collision at the time.

(jam)