cloudflare

A broken DNSSEC rollover took down .AL. Now 1.1.1.1 tells you when validation is bypassed (opens in new tab)

A failed DNSSEC key rollover at Albania’s .AL registry caused validating resolvers to return errors for every .AL domain. Cloudflare temporarily installed a Negative Trust Anchor (NTA) to restore access, suspending DNSSEC validation while the registry fixed the issue. To make this bypass visible, 1.1.1.1 began returning a new Extended DNS Error (EDE) code alongside affected responses.

What Happened to .AL

  • Around 14:15 UTC on July 3, the registry published a new DNSKEY and stopped serving the old key.
  • The root zone’s DS record still referenced the old key (id=26319), breaking the DNSSEC chain of trust.
  • Around 17:00 UTC, the new DNSKEY was also removed, leaving .AL with no DNSKEY records.
  • At approximately 19:15 UTC, the DS record was removed from the root zone, restoring normal resolution but leaving the entire TLD unsigned.
  • The outage affected government services, banks, media, and any other .AL domain accessed through DNSSEC-validating resolvers.

Why Negative Trust Anchors Were Used

  • An NTA, defined by RFC 7646, tells recursive resolvers to treat a zone as unsigned and skip DNSSEC validation.
  • Cloudflare applied an NTA to .AL at 17:15 UTC, restoring resolution for 1.1.1.1 users.
  • The measure was considered acceptable because the failure was public, confirmed, and affected validating resolvers broadly.
  • Communication with the registry was difficult because its contact addresses were themselves hosted under .AL.
  • The NTA was removed the following day after the DS record had been removed from the root zone.

The Security Tradeoff

  • NTAs prevent widespread SERVFAIL responses but remove cryptographic protection against DNS spoofing.
  • Previously, clients could not distinguish an NTA-served response from a fully DNSSEC-validated response.
  • Public status pages provide disclosure, but applications, monitoring systems, and users cannot reliably discover the bypass from DNS responses alone.

EDE-Based Transparency

  • Extended DNS Errors, defined in RFC 8914, let resolvers attach explanatory information to DNS responses.
  • Cloudflare implemented a proposed EDE code, “Negative Trust Anchor,” developed with Quad9 contributors.
  • During the incident, .AL responses included:
    • EDE 9: DNSKEY Missing, explaining the broken DNSSEC chain.
    • EDE 33: Negative Trust Anchor, explicitly stating that validation had been bypassed.
  • A query such as google.al could therefore return a successful answer while clearly indicating that it was not DNSSEC-validated.

Resolvers and applications should use EDE information where possible, while DNS operators should disclose NTAs and remove them promptly once the underlying DNSSEC problem is resolved.