A number of DNS software and service providers have announced that we will all cease implementing DNS resolver workarounds to accommodate DNS authoritative systems that don’t follow the EDNS protocol. Each vendor has pledged to roll out this change in some version of their software by the ‘Flag Day.’
In the case of BIND 9, the EDNS workarounds will be removed from BIND 9.14.0 resolvers, which will be released by Flag Day. This change is already available in the 9.13 development branch but will NOT be back-ported to the 9.11 or earlier branches of BIND. This is because of our policy of not making unnecessary changes to our stable extended support release. BIND’s authoritative server is already EDNS compliant.
Non-compliant domains may become unavailable
Domains served by DNS servers that are not compliant with the standard will not function reliably after February 1, 2019, and may become unavailable.
If your company’s DNS zones are served by non-compliant servers, your online presence will slowly degrade or disappear as ISPs and other organizations update their resolvers. When you update your own internal DNS resolvers to versions that don’t implement workarounds, some sites and email servers may become unreachable.
Test your domains
Operators of DNS authoritative systems should check their own systems at https://dnsflagday.net/ to ensure they are EDNS-compliant. If you are running any supported version of BIND 9, you are compliant already. If that site isn’t working (there has been quite a spike in traffic recently!), you can also run the tests manually, using DIG. See instructions on that at the bottom of this blog.
If you run the tests from ednscomp.isc.org, and you get failures due to timeouts, consider that possibly your RRL settings on your authoritative servers may be rate-limiting the queries coming from the test tool. ednscomp.isc.org is sending dig queries from 149.20.1.48 and 2001:4f8:1:f::48. Temporarily whitelisting those addresses in your RRL settings could fix the problem
Most common failures are due to firewalls blocking EDNS traffic, or older DNS servers that need to be upgraded.
Operators of resolvers should consult the table below, or their vendor, to determine whether the software they are running has EDNS workarounds or not. If/when workarounds are removed, resolver operators need to be on the lookout for reports that some websites have become unreachable, and should consider EDNS incompatibility to be one of the potential causes when troubleshooting. If that turns out to be the case, the solution is to notify the problem site so that they can remediate the problem.
At ISC, we have been reaching out to operators of non-compliant sites for years, urging them to update. Although there are still a few non-compliant sites that will cause problems for the rest of us, the only long-term solution is for the operators of those sites to resolve their compliance problems.
Open Source DNS Software plans
The organizations who have agreed to update their software or systems include:
| Organization | Resolver Version for Flag Day | Notes |
|---|---|---|
| ISC | BIND 9.14 (stable) to be released at the end of January 2019 – removes resolver workarounds for servers that misbehave when queried with EDNS | BIND 9’s authoritative systems are compliant with EDNS already. The change in 9.14 removes the workarounds from the BIND resolver. |
| CZ NIC | Knot Resolver – newer than the others – already written without most of the workarounds for misbehaving servers. 3.3.0 (soon to be released) has some minor changes | In general, anyone already running the latest version (3.2.0) should not notice any significant differences when upgrading to 3.3.0. |
| NLNET Labs | Unbound versions released after 1st February 2019 (1.8.4, 1.9.0 and newer) will no longer retry without EDNS when no response is received | Unbound will still accept answers without EDNS, and will still send a query without EDNS when it receives a FORMERR or NOTIMPL answer. (This change will only affect queries that result in a time-out because of EDNS in the query.) |
| PowerDNS | PowerDNS recursor 4.2 (to be released soon) will be the first one to no longer accommodate non-compliance | On the authoritative side, PowerDNS 4.1 is fully compliant; 4.0 has some corner cases that ednscomp notices but that are not a problem in practice – disabling caching removes those edge cases. |
Why make this change now?
Extension Mechanisms for DNS were specified in 1999, with a minor update in 2013, establishing the ‘rules of the road’ for responding to queries with EDNS options or flags. Despite this, some implementations continue to violate the rules. DNS software developers have tried to solve the problems with the interoperability of the DNS protocol and especially its EDNS extension (RFC 6891 standard) by various workarounds for non-standard behaviors. This is not unlike the way a driver with the right-of-way might hesitate at an intersection before proceeding if there were another driver in the intersection behaving erratically. These workarounds excessively complicate DNS software and are now also negatively impacting the DNS as a whole.
The most obvious problems caused by these workarounds are slower responses to DNS queries and the difficulty of deploying new DNS protocol features. Some of these new features (e.g. DNS Cookies) would help reduce DDoS attacks based on DNS protocol abuse.
At ISC we have been probing the DNS for years, looking at EDNS compatibility. We have seen significant improvement over the past four years, and we think this is about as good as we will get as long as we have workarounds in place. Below is one of a number of charts available at https://ednscomp.isc.org/compliance/summary.html.
Our goal is a reliable and properly functioning DNS that cannot be easily attacked.
Resources and References
DNS Flag Day website: https://dnsflagday.net
Cathy Almond of ISC Technical Support gave a presentation on DNS Flag Day on January 15th, 2019 at UKNOF: https://indico.uknof.org.uk/event/44/contributions/580/attachments/886/1082/UKNOF42-DNS-Flag_Day-2019-01-15-CA3.pdf
Checking EDNS Compatibility with DIG
The ednscomp.isc.org site uses a modified version of DIG to run multiple tests in series. This test version is available for download at https://gitlab.isc.org/isc-projects/DNS-Compliance-Testing.
You can also use the dig included in the BIND distribution and run multiple queries yourself to check a suspicious domain.
Testing Methodology
The level of EDNS compliance of a authoritative server is relatively easy to determine with a few simple DNS queries.
- dig +norec +noedns soa zone @server
- dig +norec +edns=0 soa zone @server
- dig +norec +edns=100 +noednsneg soa zone @server
- dig +norec +ednsopt=100 soa zone @server
- dig +norec +ednsflags=0x80 soa zone @server
- dig +norec +dnssec soa zone @server
- dig +norec +dnssec +bufsize=512 +ignore dnskey zone @server
- dig +norec +edns=100 +noednsneg +ednsopt=100 soa zone @server
What we expect to see:
-
Plain DNS
dig +norec +noedns soa zone @server
expect: SOA
expect: NOERROR -
Plain EDNS
dig +norec +edns=0 soa zone @server
expect: SOA
expect: NOERROR
expect: OPT record with version set to 0
See RFC6891 -
EDNS – Unknown Version
dig +norec +edns=100 +noednsneg soa zone @server
expect: BADVERS
expect: OPT record with version set to 0
expect: not to see SOA
See RFC6891, 6.1.3. OPT Record TTL Field Use -
EDNS – Unknown Option
dig +norec +ednsopt=100 soa zone @server [1]expect: SOA
expect: NOERROR
expect: OPT record with version set to 0
expect: that the option will not be present in response
See RFC6891, 6.1.2 Wire Format -
EDNS – Unknown Flag
dig +norec +ednsflags=0x80 soa zone @server [1]expect: SOA
expect: NOERROR
expect: OPT record with version set to 0
expect: Z bits to be clear in response
See RFC6891, 6.1.4 Flags -
EDNS – DO=1 (DNSSEC)
dig +norec +dnssec soa zone @server
expect: SOA
expect: NOERROR
expect: OPT record with version set to 0
expect: DO flag in response if RRSIG is present in response
See RFC3225 -
EDNS – Truncated Response
dig +norec +dnssec +bufsize=512 +ignore dnskey zone @server
expect: NOERROR
expect: OPT record with version set to 0
See RFC6891, 7. Transport Considerations -
EDNS – Unknown Version with Unknown Option
dig +norec +edns=100 +noednsneg +ednsopt=100 soa zone @server
expect: BADVERS
expect: OPT record with version set to 0
expect: not to see SOA
expect: that the option will not be present in response
See RFC6891
The above expectations are based on the following preconditions.
- Only EDNS version 0 is defined.
- EDNS option 100 is not defined.
- the only EDNS flag defined in DNSSEC OK (DO).
When EDNS version 1 is defined we expect to see: OPT record with version set to 0 or 1
When sending an EDNS version other than zero, you expect to see BADVERS or a EDNS version greater than or equal to the version you send in the response. If the version is less than the version you send and the status is NOERROR, NXDOMAIN, or YXDOMAIN, the server is non-compliant.
[1] +ednsopt and +ednsflags require BIND 9.11.0 or later. BIND 9.11.0 can be found on the ISC.ORG web site.
