Anycast vs.

DDoS
The Nov. 2015 DNS Root Event

Presented by

Ricardo de Oliveira Schmidt

October 25, 2016

Madrid, Spain

Presentation copyright © 2016 by Ricardo de Oliveira Schmidt

Reference:

Anycast vs. DDoS: Evaluating the November 2015 DNS Root Event

Giovane C. M. Moura, Ricardo de O. Schmidt, John Heidemann, Wouter B. de Vries,

Moritz Müller, Lan Wei and Cristian Hesselman

In: ACM Internet Measurement Conference (IMC), 2016, Santa Monica, USA.
Technical Report ISI-TR-2016-708, USC/Information Sciences Institute, May 2016
• http://www.isi.edu/~johnh/PAPERS/Moura16a.pdf
Distributed Denial of Service
Distributed Denial of Service

? ?
Distributed Denial of Service

? ?
Distributed Denial of Service
Big and getting bigger
2012: 100 Gb/s
2016: 100 Gb/s is common, >1 Tb/s is possible

Easy and getting easier

2012: many botnets with 1000+ nodes
2016: DDoS-as-a-service (Booters) offer few Gb/s @ US$ 5

Frequent and getting frequent-er

2002: the October 30 DNS Root event
2016: 3 recent big attacks (2015-11-30, 2015-12-01, 2016-06-25)
Distributed Denial of Service
New record!
665 Gb/s!!!
Distributed Denial of Service
New record!
665 Gb/s!!!

Even Akamai "gave up"

Distributed Denial of Service
New record!
665 Gb/s!!!

Even Akamai "gave up"

"Someone has a botnet with capabilities we haven't seen before"

Martin McKeay, Akamai
Distributed Denial of Service
Big and getting bigger
2012: 100 Gb/s
2016: 100 Gb/s is common, >1 Tb/s is possible

Easy and getting easier

2012: many botnets with 1000+ nodes
2016: DDoS-as-a-service (Booters) offer few Gb/s @ US$ 5

Frequent and getting frequent-er

2002: the October 30 DNS Root event
2016: 3 recent big attacks (2015-11-30, 2015-12-01, 2016-06-25)
Distributed Denial of Service
vDos homepage

More than
150,000 DDoS
in two years
with profit of
US$ 600,000
Distributed Denial of Service
Big and getting bigger
2012: 100 Gb/s
2016: 100 Gb/s is common, >1 Tb/s is possible

Easy and getting easier

2012: many botnets with 1000+ nodes
2016: DDoS-as-a-service (Booters) offer few Gb/s @ US$ 5

Frequent and getting frequent-er

2002: the October 30 DNS Root event
2016: 3 recent big attacks (2015-11-30, 2015-12-01, 2016-06-25)
Distributed Denial of Service

Image copyrights © thehackernews.com

Distributed Denial of Service
"Someone Just Tried to Take Down Internet's Backbone with 5
Million Queries/Sec"
Swati Khandelwal, thehackernews.com

Image copyrights © thehackernews.com

Distributed Denial of Service
"Someone Just Tried to Take Down Internet's Backbone with 5
Million Queries/Sec"
Swati Khandelwal, thehackernews.com

"Root DNS servers DDoS'ed: was it a show off?"

Yuri Ilyin, Kaspersky

Image copyrights © thehackernews.com

Distributed Denial of Service
"Someone Just Tried to Take Down Internet's Backbone with 5
Million Queries/Sec"
Swati Khandelwal, thehackernews.com

"Root DNS servers DDoS'ed: was it a show off?"

Yuri Ilyin, Kaspersky

"Someone Is Learning How to Take Down the Internet"

Bruce
Image copyrights Schneier, Schneier on Security
© thehackernews.com
The Nov. 30 Event

DDoS attack on the Root DNS

Peak of 35+ Gb/s

5 million queries/sec
Impact was moderate
Thanks to the redundancy of the whole system
 The Root DNS

Servers
ess are withdraw or absorb; r1 ... rn (internal
s vs. capacity per catchment load balancing)

s (50 Mq/s, an upper bound), ...

Sites
s1 s33 (unique location
ly traffic
and BGP route)
vere loss (1% to 95%)
ss each letter’s anycast sites; Root letters
a b c ... k l m
ict user-observed loss at sites (unique IP
sites; anycast addr.)
overloaded sites (recursive resolver
s su↵ered disproportionately user and its root.hints)
ccurred to co-located services
Figure 1: Root DNS structure, terminology, and mech-
anisms in use at each level.
ons in this paper.
 The Root DNS

Servers
ess are withdraw or absorb; r1 ... rn (internal
s vs. capacity per catchment load balancing)

s (50 Mq/s, an upper bound), ...

Sites
s1 s33 (unique location
ly traffic
and BGP route)
vere loss (1% to 95%)
Horizontal
ss each distribution
letter’s anycast sites; Root letters
a b c ... k l m
Multiple letters
ict user-observed loss at sites (unique IP
sites; Multiple operators anycast addr.)
overloaded sites (recursive resolver
s su↵ered disproportionately user and its root.hints)
ccurred to co-located services
Figure 1: Root DNS structure, terminology, and mech-
anisms in use at each level.
ons in this paper.
 The Root DNS

Servers
ess are withdraw or absorb; r1 ... rn (internal
Verticalper
s vs. capacity distribution
catchment load balancing)

s (50 Mq/s,Multiple sitesbound),

an upper Sites
s1 ... s33
ly traffic Multiple servers (unique location
and BGP route)
vere loss (1% to 95%)
ss each letter’s anycast sites; Root letters
a b c ... k l m
ict user-observed loss at sites (unique IP
sites; anycast addr.)
overloaded sites (recursive resolver
s su↵ered disproportionately user and its root.hints)
ccurred to co-located services
Figure 1: Root DNS structure, terminology, and mech-
anisms in use at each level.
ons in this paper.
Measurement Data

Measurement data:
Built-in periodical CHAOS queries @Atlas
RSSAC-002 data
BGPmon
 The Impact of the Attack

Servers
ess are withdraw or absorb; r1 ... rn (internal
s vs. capacity per catchment load balancing)

s (50 Mq/s, an upper bound), ...

Sites
s1 s33 (unique location
ly traffic
and BGP route)
vere loss (1% to 95%)
ss each letter’s
What was anycast sites;
the impact Root letters
a b c ... k l m
ict user-observed loss at sites (unique IP
sites;
at individual letters? anycast addr.)
overloaded sites (recursive resolver
s su↵ered disproportionately user and its root.hints)
ccurred to co-located services
Figure 1: Root DNS structure, terminology, and mech-
anisms in use at each level.
ons in this paper.
The Impact of the Attack
9000

2000
B C
0
What was the impact?
5000

number of VPs with successful queries

Problems on reachability!
E F
0
9000
Most letters suffered
a bit (E, F, I, J, K)
1000 G H
a lot (B, C, G, H)
7000
Did not see attack traffic 4500

D, L, M 0
I J
9000
6000

K A D L M
0
0 5 10 15 20 25 30 35 40 45 0 5 10 15 20 25 30 35 40 45
hours after 2015-11-30t00:00 UTC
The Impact of the Attack
350
B-Root
G-Root C-Root
300 G-Root
H-Root
K-Root
250
What was the impact?

median RTT (ms)

200
B-Root
For those that still see service...
150
...performance problems
100 H-Root
... 6x higher delay for G C-Root
50 K-Root

0
0 5 10 15 20 25 30 35 40 45
hours after 2015-11-30t00:00 UTC
 The Impact of the Attack

Servers
ess are withdraw or absorb; r1 ... rn (internal
s vs. capacity per catchment load balancing)

s (50 What
Mq/s, an
wasupper
thebound),
impact ...
Sites
s1 s33 (unique location
ly traffic
at individual sites? and BGP route)
vere loss (1% to 95%)
ss each letter’s anycast sites; Root letters
a b c ... k l m
ict user-observed loss at sites (unique IP
sites; anycast addr.)
overloaded sites (recursive resolver
s su↵ered disproportionately user and its root.hints)
ccurred to co-located services
Figure 1: Root DNS structure, terminology, and mech-
anisms in use at each level.
ons in this paper.
The Impact of the Attack
~48 hours (one response per pixel)
300 VPs (one per pixel)

Nov. 30th Dec. 1st

06:50 - 09:30 (UTC) 06:50 - 09:30 (UTC)
The Impact of the Attack

FRA

LHR

AMS

Blackout during attacks

The Impact of the Attack

FRA

LHR

AMS

Site flipping
 The Impact of the Attack
Zoomed in: 40 VPs initially reaching LHR site

LHR

AMS

Nov. 30th
06:50 - 09:30 (UTC)
 The Impact of the Attack

What was the impact Servers

ess are withdraw or absorb; r1 ... rn (internal
at individual
s vs. capacity servers?
per catchment load balancing)

s (50 Mq/s, an upper bound), ...

Sites
s1 s33 (unique location
ly traffic
and BGP route)
vere loss (1% to 95%)
ss each letter’s anycast sites; Root letters
a b c ... k l m
ict user-observed loss at sites (unique IP
sites; anycast addr.)
overloaded sites (recursive resolver
s su↵ered disproportionately user and its root.hints)
ccurred to co-located services
Figure 1: Root DNS structure, terminology, and mech-
anisms in use at each level.
ons in this paper.
 The Impact of the Attack
800
K-FRA-S1

K-FRA-S2

K-FRA-S3
700
What was the impact? 600
K-FRA-S2
K-FRA-S3
500
400
Impact at sites may depend... 300
200
... on load balancing 100

number of VPs
... on link resource 0

... on queuing 350

300
250
200
Individual server performance 150
K-NRT-S1
and reachability may not reflect 100
K-NRT-S2
50 K-NRT-S3
site-wide situation. 0
0 5 10 15 20 25 30 35 40 45
hours after 2015-11-30t00:00 UTC
The Additional Impact
660
620
Collateral damage! 580
D-FRA

540

number of VPs
120 D-SYD
100
D-Root was not targeted...
80
... but felt the attack 60 D-AKL

40 D-DUB
20 D-BUR
0
0 5 10 15 20 25 30 35 40 45
hours after 2015-11-30t00:00 UTC

Even SIDN (.nl) felt the attack:

NO traffic in FRA and AMS
The Additional Impact
660
620
Collateral damage! 580
D-FRA

540

number of VPs
120 D-SYD
100
D-Root was not targeted...
80
... but felt the attack 60 D-AKL

40 D-DUB
20 D-BUR
0
0 5 10 15 20 25 30 35 40 45
hours after 2015-11-30t00:00 UTC

NL-FRA

Even SIDN (TLD) felt the attack:

.nl instances

NL-AMS
NO traffic in FRA and AMS
0 7 29 45
hours after 2015-11-30t00:00 UTC
The Lessons Learned

The Root DNS handled the situation quite well...

... at no time the service was completely unreachable

Resilience of the Root DNS is not an accident...

... consequence of fault tolerant design and good engineering!

True diversity is key to avoid collateral damage

And, What Now?
Learn from the Root DNS experiences

Have in mind the possible very large DDoS attacks when...

... designing distributed systems
... improving countermeasures and mitigation strategies

It does not matter if...

... someone was showing off
... someone was testing/scanning the infrastructure
... someone is learning how to take down the Internet

It was a big wake up call, this is critical infrastructure!

Things are escalating pretty fast and apparently we are not fully aware of
what we are dealing with.
 r.schmidt@utwente.nl
http://www.ricardoschmidt.com

Acknowledgements:

Arjen Zonneveld, Jelte Jansen, Duane Wessels, Ray Bellis, Romeo Zwart, Colin Petrie,
Matt Weinberg and Piet Barber

SIDN Labs, NLnet Labs and SURFnet

Self-managing Anycast Networks for the DNS (SAND) project | http://www.sand-project.nl/

NWO DNS Anycast Security (DAS) project | http://www.das-project.nl/