DNSSEC disruption affecting .de domains – Resolved

682 points - yesterday at 8:16 PM

Source

Comments

krystofbe yesterday at 8:27 PM
Looks like a DNSSEC issue, not a nameserver outage. Validating resolvers SERVFAIL on every .de name with EDE:

RRSIG with malformed signature found for a0d5d1p51kijsevll74k523htmq406bk.de/nsec3 (keytag=33834) dig +cd amazon.de @8.8.8.8 works, dig amazon.de @a.nic.de works. Zone data is intact, DENIC just published an RRSIG over an NSEC3 record that doesn't validate against ZSK 33834. Every validating resolver therefore refuses to answer.

Intermittency fits anycast: some [a-n].nic.de instances still serve the previous (good) signatures, so retries occasionally land on a healthy auth. Per DENIC's FAQ the .de ZSK rotates every 5 weeks via pre-publish, so this smells like a botched rollover.

Aldipower yesterday at 9:59 PM
Apparently the DENIC team was on a party this evening! Party hard, but not too hard. https://bsky.app/profile/denic.de/post/3ml4r2lvcjg2h
tom1337 yesterday at 10:48 PM
Cloudflare has now disabled DNSSEC validation on their 1.1.1.1 resolver: https://www.cloudflarestatus.com/incidents/vjrk8c8w37lz
pocksuppet yesterday at 9:11 PM
I must be early. There's not a single tptacek DNSSEC rant in this thread yet.
sundiver yesterday at 8:34 PM
Yes, all .de domains down because of DNSSEC failure at Denic https://dnsviz.net/d/de/dnssec/
tom1337 yesterday at 9:47 PM
I have never used DNSSEC and never really bothered implementing it, but do I understand it correctly that we took the decentralized platform DNS was and added a single-point-of-failure certificate layer on top of it which now breaks because the central organisation managing this certificate has an outage taking basically all domains with them?
chromehearts yesterday at 8:53 PM
I was STRESSING tf out because I wasn't able to connect to my services & apps through my domains like at all .. they only work when using my phone data ? .. thank god it's not my fault this time
siva7 yesterday at 9:22 PM
Crazy. I can't remember an incident like this ever happened before and it's still not fixed? .de is probably the most important unrestricted domain after .com from an economical perspective. Millions of businesses are "down".
sunaookami yesterday at 8:52 PM
https://status.denic.de/ says "Partial Service Disruption" for DNS Nameservice now.

EDIT: it says "Service Disruption" now

kuerbel yesterday at 8:52 PM
I just spent the better half of an hour to debug unbound and the pihole because I thought it's a me problem...

Good news though, if you add domain-insecure: "de" to your unbound config everything works fine

__michaelg yesterday at 8:52 PM
Finally establishing the concept of Feiertag on the internet. Come back tomorrow.
1vuio0pswjnm7 yesterday at 8:48 PM
.de TLD is online. DNS working fine

DNSSEC not working

If using an open resolver, i.e., a shared DNS cache, e.g., third party DNS service such as Google, Cloudflare, etc., then it might fail, or it might not. It depends on the third party DNS provider

https://datatracker.ietf.org/meeting/118/materials/slides-11...

SEJeff yesterday at 11:14 PM
Just gonna leave this absolute gem from Thomas Ptacek on DNSSEC here:

https://sockpuppet.org/blog/2015/01/15/against-dnssec/

iknowstuff yesterday at 8:44 PM
Kurzgesagt predicted this, Germany is OVER
kaltsturm yesterday at 10:01 PM
Denic will be added to the "Major DNSSEC Outages and Validation Failures" list: https://ianix.com/pub/dnssec-outages.html
yassiniz yesterday at 10:05 PM
Shops open normally from 8am to 8pm in Germany. Today we decided to pilot opening hours for .de domains as well
aboardRat4 yesterday at 11:58 PM
yowmamasita yesterday at 10:05 PM
The same day Kurzgesagt posted their video “Germany is over”. Huh. https://youtu.be/n-gYFcVx-8Y
basilikum yesterday at 11:46 PM
This is the kind of system failure that we need really good and well tested disaster recovery plans for. While not necessary this time, DENIC and any critical infrastructure provider should be able to rebuild their entire infrastructure from scratch in a tolerable amount of time (Rather days than hours in the case of a full rebuild). Importantly the disaster recovery plan has to work without reliance on either the system that is failing, but also on adjacent systems that might have hidden dependencies on the failing system.

I'm really not too close to Denic and know nothing about their internals, but just close enough to have experienced the stress of someone working for DENIC second hand during the outage. From the very limited information I happened to gather DENIC had some trouble in addressing the issue because, surprise, infrastructure that they need to do so runs on de domains. [1]

I'm convinced there are all kinds of extended cyclic decencies between different centralization points in the net.

If some important backbone of the internet is down for an extended time, this will absolutely cause cascading failures. And thesw central points of failure are only getting worse. I love Let's Encrypt, but if something causes them to hard fail things will go really bad once certificates start to expire.

We need concrete plans to cold start extended parts of the internet. If things go really bad once and communication lines start to fail, we're in for a bad time.

Maybe governments have redundant, ultra resistant, low tech communication lines, war rooms and a list of important people in the industry who they can find and put in these war rooms so they can coordinate the rebuild of infrastructure. But I doubt it.

[^1] I don't know if there is some kind of disaster plan in the drawer at DENIC that would address this. I don't mean to allege anything against DENIC specifically, but broadly speaking about companies and infrastructure providers, I would not be surprised if there was absolutely no plan on what to do if things really go down and how to cold start cyclic dependencies or where they even are.

edb_123 yesterday at 10:29 PM
Things seem to be on their way up now, and https://status.denic.de/ is working again, at least from here.

DENIC's status page currently says "Frankfurt am Main, 5 May 2026 – DENIC eG is currently experiencing a disruption in its DNS service for .de domains. As a result, all DNSSEC-signed .de domains are currently affected in their reachability. The root cause of the disruption has not yet been fully identified. DENIC’s technical teams are working intensively on analysis and on restoring stable operations as quickly as possible.

alper today at 9:24 AM
I'd expect political escalation for something like this but given that this is Germany, who knows.
nfreising yesterday at 9:55 PM
They can join the (rather long) list of TLD DNSSEC outages https://ianix.com/pub/dnssec-outages.html
elevation yesterday at 9:12 PM
I've considered hard-coding some addresses into firmware as a fallback for a DNS outtage (which is more likely than not just misconfigured local DNS.) Events like this help justify this approach to the unconcerned.
kangalioo yesterday at 8:23 PM
So glad I found someone mention this. Amazon.de, SPIEGEL.de is down. Highly prominent sites unreachable. I wonder how long this will last and how big of a thing this ends up being once people talk about it :o Feels big to me
Zopieux yesterday at 11:02 PM
That postmortem should be a fun read, can't wait.
kaltsturm yesterday at 9:03 PM
merb yesterday at 8:44 PM
Well at least it’s night time which means it’s hopefully resolved in the morning.

Looks like it failed after a maintenance: https://www.namecheap.com/status-updates/planned-denic-de-re...

https://status.denic.de/

taf2 yesterday at 10:45 PM
ok i picked a bad day to move from one register to another... i just spent the last hour frantically trying to figure out why the new register screwed us or the old register was screwing us...
dwedge yesterday at 9:16 PM
On a slightly unrelated note, I was setting nameservers for two .de domains a few weeks ago and thought my provider was being crazily strict because they kept getting rejected. Turns out you can't point to a nameserver until that nameserver has a zone for the domain, and you can't use nameservers from two providers unless those two providers are both in the NS records at both ends
deleted yesterday at 8:24 PM
hmilch99 yesterday at 8:25 PM
https://pastebin.com/2mQUB8xX seems like someone's going to have a lot of fun tonight
nuil yesterday at 8:28 PM
adamas today at 8:03 AM
I wasn't even aware that was possible..?
kaltsturm yesterday at 9:56 PM
Denic should work out a desaster recovery test - like: https://blog.apnic.net/2022/02/14/disaster-recovery-with-dns...
0x80h yesterday at 9:51 PM
Am I reading this correctly? All .de domains are down? Looking forward to reading the postmortem.
0xbadcafebee today at 4:46 AM
I can't wait for the .com TLD outage. Ya'll thought Cloudflare down was bad? Lol
baby today at 4:54 AM
Should I do my usual rent about how the web PKI refuses to move to a consensus protocol
g4cg54g54 yesterday at 9:44 PM
funfact: enabling DNS sec NOW will fix your domain instantly if dnssec was disabled before

-> no idea if that also "heals" anyone who had dnssec on before.

-> no idea if maybe they need to roll back something and then rebreak the new dnssec i made a minute later lol...

warpspin yesterday at 8:16 PM
Whole .de TLD seems to go offline right now due to dnssec or missing nic.de nameservers?
yosamino yesterday at 9:03 PM
The last time .de I remember .de had a major outage like this was 2010. I would cite some sources but... you know. That was a fun afternoon, though.

I am very happy that it doesn't happen more often.

Oarch yesterday at 10:22 PM
Germany has fallen.
jamietanna yesterday at 8:37 PM
Was wondering why a few of my sites aren't CSSing, as they use https://classless.de
victorbjorklund yesterday at 8:53 PM
I was just wondering what was up with our .de site.
kaltsturm yesterday at 9:07 PM
even their own status page is not reachable: https://status.denic.de/

As fallback they should use their X account: https://x.com/denic_de

deleted yesterday at 9:46 PM
lxgr yesterday at 8:51 PM
Wow, I thought I was somehow unaffected but my resolver must just have cached the sites I'd tried.
deleted yesterday at 11:25 PM
kaltsturm yesterday at 10:19 PM
from my analysis DENIC resigned the .de zone today (May 5, 2026, ~17:49 UTC). The DNSSEC signature (RRSIG) for the NSEC3 record covering the hash range of nearly all .de TLD is cryptographically broken (malformed).
binghatch yesterday at 8:30 PM
Wow
 it’s definitely not all .de TLDs, but a lot of prominent ones definitely.
tarruda yesterday at 9:09 PM
Mailbox.org (also from Germany) seems to be experiencing issues too.
jdthedisciple today at 6:17 AM
Seems up again. How briefly did the outage last?
jiveturkey yesterday at 10:32 PM
It’s not DNS

There’s no way it’s DNS

It was DNSSEC

bflesch yesterday at 10:09 PM
On Monday there was a huge outage affecting several cities quite close to Frankfurt because someone cut major fiber line; today DENIC is having a party and right when everyone is drunk this happens because some post-rotation task cannot be completed.

There are too many coincidences happening.

kaltsturm yesterday at 10:22 PM
With chrome it works again
whalesalad yesterday at 9:35 PM
You can visually see this anomaly in many of CF Radar's charts: https://radar.cloudflare.com/dns/de?dateRange=1d
NooneAtAll3 yesterday at 11:54 PM
quad9 seems to be having problems with DNSSEC as well
Animux yesterday at 10:21 PM
Seems to be fixed now.
dark-star yesterday at 9:05 PM
How come I have zero problems with any .de domain I tried accessing in the last half hour?
jiggawatts yesterday at 8:55 PM
I work with a few people specialised in IT security, and some of them take their jobs too seriously and will "lock down" everything to the point that it becomes a very real risk that they lock out everyone including themselves.

Fundamentally, security is a solution to an availability problem: The desire of the users is for a system to remain available despite external attack.

Systems that become unavailable to everyone fail this requirement.

A door with its keyhole welded shut is not "secure", it's broken.

sanbaideng yesterday at 9:57 PM
aiimageupscaler
siginator yesterday at 9:12 PM
how is that possible?
pogii123 yesterday at 8:37 PM
For me bmw.de works but www.bmw.de not
neverrroot yesterday at 10:18 PM
[flagged]
deleted yesterday at 8:27 PM
evan0721 today at 3:54 AM
[dead]
blmaniac yesterday at 8:49 PM
[dead]
siginator yesterday at 9:14 PM
[dead]
lpcvoid yesterday at 8:39 PM
[dead]
amelius yesterday at 10:46 PM
Maybe related to this? Crazy idea, but nothing surprises me anymore.

https://edition.cnn.com/2026/05/01/politics/us-troop-withdra...