Criminals this morning massively attacked Dyn, a company that provides core Internet services for Twitter, SoundCloud, Spotify, Reddit and a host of other sites, causing outages and slowness for many of Dyn’s customers.
Twitter is experiencing problems, as seen through the social media platform Hootsuite.
In a statement, Dyn said that this morning, October 21, Dyn received a global distributed denial of service (DDoS) attack on its DNS infrastructure on the east coast starting at around 7:10 a.m. ET (11:10 UTC).
“DNS traffic resolved from east coast name server locations are experiencing a service interruption during this time. Updates will be posted as information becomes available,” the company wrote.
DYN encouraged customers with concerns to check the company’s status page for updates and to reach out to its technical support team.
A DDoS is when crooks use a large number of hacked or ill-configured systems to flood a target site with so much junk traffic that it can no longer serve legitimate visitors.
DNS refers to Domain Name System services. DNS is an essential component of all Web sites, responsible for translating human-friendly Web site names like “example.com” into numeric, machine-readable Internet addresses. Anytime you send an e-mail or browse a Web site, your machine is sending a DNS look-up request to your Internet service provider to help route the traffic.
ANALYSIS
The attack on DYN comes just hours after DYN researcher Doug Madory presented a talk on DDoS attacks in Dallas, Texas at a meeting of the North American Network Operators Group (NANOG). Madory’s talk — available here on Youtube.com — delved deeper into research that he and I teamed up on to produce the data behind the story DDoS Mitigation Firm Has History of Hijacks.
That story (as well as one published earlier this week, Spreading the DDoS Disease and Selling the Cure) examined the sometimes blurry lines between certain DDoS mitigation firms and the cybercriminals apparently involved in launching some of the largest DDoS attacks the Internet has ever seen. Indeed, the record 620 Gbps DDoS against KrebsOnSecurity.com came just hours after I published the story on which Madory and I collaborated.
The record-sized attack that hit my site last month was quickly superseded by a DDoS against OVH, a French hosting firm that reported being targeted by a DDoS that was roughly twice the size of the assault on KrebsOnSecurity. As I noted in The Democratization of Censorship — the first story published after bringing my site back up under the protection of Google’s Project Shield — DDoS mitigation firms simply did not count on the size of these attacks increasing so quickly overnight, and are now scrambling to secure far greater capacity to handle much larger attacks concurrently.
The size of these DDoS attacks has increased so much lately thanks largely to the broad availability of tools for compromising and leveraging the collective firepower of so-called Internet of Things devices — poorly secured Internet-based security cameras, digital video recorders (DVRs) and Internet routers. Last month, a hacker by the name of Anna_Senpai released the source code for Mirai, a crime machine that enslaves IoT devices for use in large DDoS attacks. The 620 Gbps attack that hit my site last month was launched by a botnet built on Mirai, for example.
Interestingly, someone is now targeting infrastructure providers with extortion attacks and invoking the name Anna_senpai. According to a discussion thread started Wednesday on Web Hosting Talk, criminals are now invoking the Mirai author’s nickname in a bid to extort Bitcoins from targeted hosting providers.
“If you will not pay in time, DDoS attack will start, your web-services will
go down permanently. After that, price to stop will be increased to 5 BTC
with further increment of 5 BTC for every day of attack.NOTE, i?m not joking.
My attack are extremely powerful now – now average 700-800Gbps, sometimes over 1 Tbps per second. It will pass any remote protections, no current protection systems can help.”
Let me be clear: I have no data to indicate that the attack on Dyn is related to extortion, to Mirai or to any of the companies or individuals Madory referenced in his talk this week in Dallas. But Dyn is known for publishing detailed writeups on outages at other major Internet service providers. Here’s hoping the company does not deviate from that practice and soon publishes a postmortem on its own attack.
Update, 3:50 p.m. ET: Security firm Flashpoint is now reporting that they have seen indications that a Mirai-based botnet is indeed involved in the attack on Dyn today. Separately, I have heard from a trusted source who’s been tracking this activity and saw chatter in the cybercrime underground yesterday discussing a plan to attack Dyn.
Update, 10:22 a.m. ET: Dyn’s status page reports that all services are back to normal as of 13:20 UTC (9:20 a.m. ET). Fixed the link to Doug Madory’s talk on Youtube, to remove the URL shortener (which isn’t working because of this attack).
Update, 1:01 p.m. ET: Looks like the attacks on Dyn have resumed and this event is ongoing. This, from the Dyn status page:
This DDoS attack may also be impacting Dyn Managed DNS advanced services with possible delays in monitoring. Our Engineers are continuing to work on mitigating this issue.
Oct 21, 16:48 UTCAs of 15:52 UTC, we have begun monitoring and mitigating a DDoS attack against our Dyn Managed DNS infrastructure. Our Engineers are continuing to work on mitigating this issue.
Oct 21, 16:06 UTC
Tags: Akamai, Anna_Senpai, DDoS, DNS, Doug Madory, Dyn, internet of things, IoT, mirai, OVH, Reddit, twitter
Thanks for the article. The attack is still in effect. Github, Heroku, Twitter are still down and that’s just some, there are many others.
The original attack seemed more northeast coast but I can see its global now. I’ve been trying different sites since 12:30 pm and still nothing.
I hope this is resolved soon.
PlayStation network is down too.
There is a good chance your route is being remembered which is why the sites are still down. Each domain has a value called a time to live value that keeps track of how long your ISPs recursive dns servers remember that path. If it didn’t resolve to an IP within that path, it’ll remember the result for a certain period of time.
You’re wrong.
3:30 EST update: Twitter, Github, Soundcloud, PayPal all still down. Netflix is up but loading very slowly. reddit and Spotify are up.
What if Google/Microsoft/network providers etc.. joined together to scan the internet for ISPs that allow IP spoofing, and other misconfigured routers that lend themselves to amplification attacks, and inject warnings into all IPs from those networks anytime they do a google search, letting them know their ISP is vulnerable. If the ISP does not fix it within a month of discovery, they block access to google, bing, Wikipedia… to that entire network, until they properly configure their routers. Could even to it on a smaller level to IoTs that use default U/P that are open on the web, or are already bots. Fix your stuff, or get blacklisted.
The Web gods are trying to mute truth and free speech.
Actually, it looks like the Internet criminals are trying to mute truth and free speech.
Looks like they are testing their new IoT Bot farm. I believe that we are the front lines and it’s up to us to prevent a dystopian future from happening. The future is what we make it. I choose to make it Star Trek. Quote from Carl Sagan Contact:
David Drumlin: I know you must think this is all very unfair. Maybe that’s an understatement. What you don’t know is I agree. I wish the world was a place where fair was the bottom line, where the kind of idealism you showed at the hearing was rewarded, not taken advantage of. Unfortunately, we don’t live in that world.
Ellie Arroway: Funny, I’ve always believed that the world is what we make of it.
The link to the YouTube video is a t.co shortened URL, and since Twitter is down…
I believe this is the youtube link (if it gets shortened, it’s v=LFJzu0AFDpU ) in question for the talk:
https://www.youtube.com/watch?v=LFJzu0AFDpU
Yes, that’s it. Updated the story to reflect that and to note that link shorteners via Twitter aren’t working at the moment.
Why is Akamai a tag? Are they related to Dyn?
Krebs was on akamai, prior to the last attack on this site. He had to migrate away from them, as it was effecting other, paying customers.
Paying Anna-senpai to make it go away isn’t a solution. Releasing Anna-senpai’s real name and address could give pause-to-rethink-bad-behavior to this jackal.
Think you’re up to it Brian?
Yeah Bryan.. Get these guys! What is taking you so long!?
😉
Your Twitter link for Doug Madory’s talk is affected as well.
It is titled “BackConnects Suspicious BGP Hijacks” on YouTube (assuming you don’t allow links in comments).
Do we know where the majority of the attacks are coming from? I am looking at Shodan searches trying to figure out what type of vulnerability is involved on the attacking hosts.
In an side note: I’ve almost started called all the self-starting videos on sites DDOS attacks themselves. They pull down an increasing large, maybe even huge (dunno yet), amount of data and prevent me from seeing or otherwise getting to the article I may have come to the site for. Also, increasingly there is no way to 1) prevent the download regardless of play and/or 2) no way to really shut them off and even if you can hit the pause button (no stop buttons are on these) there is no way to prevent any further loading of video in the background.
And I do video, but not these. The irony is the idea that the more media and in particular moving or animated video, the more you get your message out. That is simply not so. The more you shove at anyone the more our brains shut down on input. And the move movement you have in the corners the less you can focus on 1) the article and even 2) the ad contents if you want. I will also add in the constantly shifting menus in fast food places which make it irritating to try to order from.
Anyway, I hope I am not too far off the subject of this particular article (my apologies if so) but I was just on a couple of news sites which were pushing all this junk down my pipe (junk I don’t want to pay for but get stuck for), when I came over here hoping for the most recent info on Dyn’s DDOS attack. I couldn’t help make the comparison and think about the slippery slope in terms of DDOS (in effect) coming from the main site.
I had to add adblock plus a week ago for the very same reason on a desktop. This same 3 to 6 gigabyte an hour download and a nice big upload made even scrolling through some sites impossible. Started happening on a few more but not all sites. netstat is as far as I know, so I could only guess whether it is adware gone mad, or a hacking exploit that used large downloads perhaps just fishing in memory. Would be nice if someone dropped a clue. Thanks.
Why don’t people change their DNS settings to Cisco OpenDNS? 208.67.222.222
Problem solved for those in life altering situations. I hope those in vulnerable positions have good IT folks on hand to understand you can get around a major of issues.
Also, this site even went down for me after I refreshed and before I switched my DNS server. Many, many more people need to learn basic concepts about how networking works in this world to mitigate these things. Far too many people are obsessed with their touch screen pretty GUI’s to do things online, without putting in the effort to learn WHY it works.
Most people do not care any more about how the Internet works than they do about how their automobiles work.
Dude, if the domain DNS is down, switching your DNS will do nothing. Only if the DNS still have cached the query you will able to enter the web. If not, when your DNS try to ask the domain’s DNS server, will have no answer.
You need to learn the basics of how networks works, and how DNS works.
OpenDNS:
http://imgur.com/a/y3kvQ
Default DNS (Time Warner/Charter ISP here):
http://imgur.com/a/iksdK
Anything else? Every single DNS server is not down.
Dude Daniel, Shane’s suggested Cisco OpenDNS change works.
If the authoritative DNS server for a Doman is on Dan, changing the DNS server you are using does nothing. All DNS servers have to be able to reach the authoritative DNS server for a domain to be able to resolve a request. The only exception is when the request is cached on the server already or you create your own authoritative zone for the domain. If using cached queries, the TTL of a record usually determines how long the cache sticks around.
Again, all authoritative servers are not down, that’s why it works. Outage maps show the regions that are hit, and the Midwest US hasn’t been hit with anything. Neither have other parts of the world.
I mean, YES, obviously if every single redundancy of an authoritative name server for a site is down, then you’re working off the cache of a recursive one. But these are large companies, and both Dyn and the websites affected surely have redundancies in place to be operation in at least SOME areas.
So open DNS is still operational due to their own smart cashe feature.
So, you are saying nothing. You don’t know who/what is caching, and for how long. Hint: were I doing a DNS server (such as Cisco Open DNS), there is nothing to prevent me from using a cached record if there was a problem reaching an authoritative server. You seen to understand all this, by reading your response. It’s supposed to be this way, but if it isn’t … Obviously, you don’t understand the difference between regular operations and emergency operations, and you don’t know the difference between RFCs and “implementation variations based on experience.”
Well I apologize, I was just trying to provide information, and then I get a lecture about what my suggestion was (that worked) to give people a workaround and how some should understand possible workarounds.
You’re right, I don’t know exactly if it’s a cache from Cisco, or if it’s redundancies. I did a little more research in nslookup and the primary dyn server for Twitter is down. OpenDNS expiry is 7 days when I ran the -type=soa switch.
Just, nevermind. This is why I barely post on the internet. I just don’t want anyone to die, because I’ve read doctor’s offices can’t get to their records and other things in some places, and it would be horrible if someone was given the wrong medicine or something when there’s an option out there to change DNS settings to a workable setting temporarily to mitigate that.
I’m done, you won over whatever it is why I’m being argued with over trying to help.
Shane, the fix you had suggested also worked for me. I learned my lesson after the big Comcast DNS outage which I think was in 2015. I changed my config to use OpenDNS servers and then had no trouble accessing twitter and other sites for which I had received DNS errors when using Comcast’s default DNS server. Not sure why people think it’s appropriate to flame you for a great and useful suggestion. Competitive douchery, some people don’t like it when you know something they don’t.
So, you are saying nothing. You don’t know who/what is caching, and for how long. Hint: were I doing a DNS server (such as Cisco Open DNS), there is nothing to prevent me from using a cached record if there was a problem reaching an authoritative server. You seen to understand all this, by reading your response. It’s supposed to be this way, but if it isn’t … Obviously, you don’t understand the difference between regular operations and emergency operations, and you don’t know the difference between RFCs and “implementation variations based on experience.” Think before typing.
Can confirm that switching to OpenDNS works. Couldn’t get into Intacct.com, switched secondary DNS to OpenDNS and website now loads.
Dude… you need to learn how DNS works. Not all resolvers are down.
Dude, if the domain DNS is down, switching your DNS to OpenDNS will actually work because of their SmartCache feature.
“SmartCache uses the intelligence of the OpenDNS network at large, providing DNS service to tens of millions of people around the world, to locate the last known correct address for a Web site when its authoritative nameserver is offline or otherwise failing.”
You need to learn the basics of how networks works, and how OpenDNS works.
Political Governance can be criminal. Maybe the “criminals” are trying to mute Wikileaks? Just a suggestion.
Hours after dyn gave a talk about back connects poor presence in the net space and this happens? This is more than enough smell for anyone to ponder at this point. It’s time to investigate the company and the individuals that are anyway familiar or connected to this marshal webb and Tucker guy. I consider to everyone to investigate these individuals pasts this is all too frequent surrounding their company. I also believe these actors even attacked your site krebs and used vdos as a riot shield to get away with it. Krebs after all this are you still not questioning the attacks may have something to do with them afterall?
Did you even read the story you’re commenting on? What do you think I’ve been doing ever since these attacks started? Who else do you see who’s exposing the people who are facilitating these attacks?
Yeah, exactly.
You’re doing a great job! You are modest to imply Doug Madory’s talk may have had been more of a factor than the publication of, Spreading the DDoS Disease and Selling the Cure. Today’s attack is surely no coincidence!
I bet you get some help in your efforts now. This attack is getting a lot of attention. And lot of attention means a lot of complaints. And lot of complaints, especially from large companies, get a law enforcement response.
You don’t have the power of subpoena. And you can’t search and seize computers for evidence. Many of the people you wrote about reside in the US. They will be investigated.
This attack may have a silver lining, it may just be big enough and annoying enough to finally get people’s attention concerning the systemic structure vulnerabilities.
It’s the canary in the coal mine, the question – is anyone watching or will they just pass it off, again, as merely a sick bird rather than the early indications of the criminal poison gas that it is?
FYI – Box and Okta still down due to DDoS attack on Dyn.
We formerly used PowerDNS (not the software, the service) in Europe, until it became a proxy battleground for Ukraine-Russia disputes and hence unfortunately useless for those of us without a dog in that fight. PowerDNS is now out of business– once customers have to move, there’s no going back. In our case we moved to AWS but w/TB-level hosing on tap there’s not really anywhere safe.
I do have to wonder if this is an escalation of saber–rattling w/regard to the US election and Russian meddling, the freezing of RT bank accounts, etc. A message, perhaps?
What kind of DDoS methods are they employing this time? Reflective DNS + Amplification? Or direct HTTP requests to the target?
I find this a strange target, a general DNS provider. What political agenda could this serve, other than to simply annoy the general public?
Did my comment post?
From Brian’s article the implication is that its folks butthurt over the exposure of the DDoS for hire types. So right now I imagine that DHS and the FBI are looking very carefully at the feeds coming out of Clemson U given that the biggest drama queen exposed so far seems to be attending college over there.
My twitter is still not logging in from any mobile or desktop through any network means WIFI in Pakistan. When shall b this problem resilved
Ambreen, you can watch the status page for Dyn here. As they get their issues resolved, I imagine you’ll be able to hit twitter again.
https://www.dynstatus.com/
Huh? Just logged into twitter 20 minutes after you didn’t. From floriduh, the state with all the brains in the country.
Isn’t it possible to configure your phone to use another DNS server, like OpenDNS?
Question: ( possibly answered by Daniel )
Suppose your primary authoritative DNS server is at DYN , and your secondary authoritative server is at Rackspace, or Google, or Neustar, etc ..
If you primary DNS server at DYN fails to respond, then shouldn’t the dns client then query your seconday DNS server hosted at Rackspace, or Google, or Neustar etc. ?? ( i.e wherever you have your secondary DNS hosted.)
For example ( w/ fictitious info)
http://www.yourdomain.com
Registrar: DYNAMIC NETWORK SERVICES, INC
DNS Name servers:
ns.dyn.net 209.38.40.1
ns.ultradns.biz 126.194.26.66
So if dyn.net is under attack and not responding, should a dns client the query the ultradns server instead ?
Isn’t that the whole reason domain registration requires 2 dns servers ? so that if the first one is inaccessible , dns clients can query the second one.
Perhaps I am not clear on how DNS works.
It appears that Dyn have been making some statements to The Register that you may wish to append to your article.
http://www.theregister.co.uk/2016/10/21/dyn_dns_ddos_explained/?mt=1477082947065
It appears that Dyn have been making some statements to The Register that you may wish to append to your article.
http://www.theregister.co.uk/2016/10/21/dyn_dns_ddos_explained/?mt=1477082947065
Feels like a dry-run for election day.