Some thoughts on modern denial of service

http://www.e-gerbil.net/ras/dos.txt

Table of Contents

1. Modern Denial of Service
2. What we face
3. Distributed Denial of Service
4. High packet per second floods
5. Attacking the infrastructure
6. LAN attacks
7. Dropping packets on the floor
8. How to filter
9. What to filter
10. Protecting router interfaces
11. Protecting everyone else
12. Assume = Ass + U + Me




Section 1 - Modern Denial of Service

However Denial of Service truly began, most people acknowledge that it has
evolved into its current form by Internet Relay Chat. Denial of Service
didn't just happen, it required effort and motivation to reach today's
levels.

Unfortunantly, what was once a game of "high tech" petty vandalism for
teenagers has begun leaking out into the rest of the internet. Once it
could be argued that if you simply weren�t on IRC, you would not ever have
to deal with attacks. Many providers have even gone so far as to blame the
victim for inviting the attacks, and if called for assistance the answer
would be to simply unplug or null-route the victim. This solution may have
worked for small-time customers, but how do you say that to a Yahoo, eBay,
or CNN?

But on the other side of the coin, what provider wants to advertise that
they are attack-proof and invite someone to prove otherwise? Who wants to
invite customers that will attract attacks? After all, what most providers
see as a "big attack" is pretty small and infrequent in comparison to what
could be done if there was sufficient motivation for the attackers.

Back in Feburary 2000, a canadian kid with the nick "mafiaboy" wanted to
impress some people on IRC. He turned some scripts and tools which took
almost no advanced technical skills to use from their usual target of IRC
and against some big names in "e-Commerce", and proceeded to change the
way people think about DoS. The only reason he was caught was he bragged
about it on IRC. The most supprising thing about this is that no major
attacks have followed.



Section 2 - What we face

So you think the tools which exist now are bad? Think about this:

If attacks were rate-limited to not use "full force", they would be
extremely difficult to detect and trace. What if an attacker used 10,000
boxes which generated a relatively "unnoticable" amount of traffic?

What happens when someone writes a DDoS network which uses more then badly
ripped crypto from eggdrop source code? What happens when someone designs
a self-maintaining network which can't be traced back to a single human.

What if there was a totally modular DDoS network, where when a new attack
comes out, a new module is simply pushed out through the network.

What if thousands of network nodes were able to automatically select the
best method of attack? Can't spoof outbound, well use an unspoofed attack
which blends in with spoofed attacks generated by other nodes.

What if nodes analysed the paths they took to the victim and tuned for the
most damage. This could also be used to bypass any automatic path
auto-detection, auto-tracing, and/or auto-filtering mechanisms.

What if an attacking node automatically backed off the attack if it
detected reasons to do so. What if the attacking nodes reported
information about which are the most useful nodes so the human operator
could make better decisions about what to use, and where to hack next.
After all, one of the biggest causes of smurf failures is the attacker
using broadcasts which generate a lot of nasty looking dupes at low rates
and small packet sizes, but which actually put out an extremely small
amount of traffic in an actual attack. Does the same situation apply to
DDoS nodes?


Section 3 - Distributed Denial of Service

You suddenly find you're being attacked by an unspoofed UDP flood. Easy to
filter, and have the source shut down, right?

What if you're suddenly attacked by 70 machines simultaneously? What do
you do in that situation? What do you go after? Many victims are so
overwelmed by this simple fact that they do nothing.

What if attacks are a mix of spoofed and unspoofed traffic? What if the
attacks are intentionally designed to make it impossible to guess which is
which? What if the attacks are designed to look unspoofed and implicate
yet another victim?

What if the attack is disguised as something which slips through ordinary
filters? How do you plan to filter a spoofed "dns attack" from the
root-servers?



Section 4 - High packet per second floods

When SYN floods were first popularized, the goal was simply to overwhelm a
small buffer of outstanding new connections and prevent new ones. In most
respects it was an attack against the logic of the TCP implementation, it
could be used from a dialup to hold down a well connected server, and
coders quickly set to work fixing the problem. Today it is virtually
impossible to do the same kind of attack against any modern operating
system. SYN floods still continue, but the goal has changed.

Modern SYN floods pit CPU vs. CPU, in a battle for efficiency. If an
attacker can assemble a decent collection of relatively inexpensive PCs on
relatively mediocre connections, he can take down top of the line servers
or million dollar routers with almost remarkable ease.

As if this wasn�t bad enough, high packet per second floods highlight the
problems with our existing network structures and routing designs. The
burden is not in the amount of data, but the work that must go into
handling each frame, inspecting each header, processing each packet, and
the scheduler and queuing algorithms involved in handling the interrupts
and processing the load.

The next problem is in tracing them. In circumstances where we are barely
able to route the attack, there is no time to stop and look at it, and
attempt to trace it back to its origins.

Remember when doing packet/sec calculations that overhead starts to play a
significant factor. For example ethernet overhead works out something like:
  Preamble (8 bytes) + Ethernet header (14 bytes) + Payload (20+20 bytes?) +
  Padding (6 bytes) + FCS (4 bytes) + Inter-Frame Gap (12 bytes)
  Or 44 bytes of overhead per frame for every 40 bytes of data.

This has the benefit of slowing down the attacker, AND the victim. Note
in the example above we used an IP+TCP header, nothing fancy. The actual
frames/sec rate will be a bit less then half of what you would expect
based on bandwidth calculations alone. The numbers for 64-byte frames
(which is the minimium after padding) taking in to account the other 20
bytes of overhead are 14,880 for 10Mbps, increasing by factors of 10 for
FastEthernet and GigabitEthernet respectively.



Section 5 - Attacking the infrastructure

The design of most routers involves a central route processor which
handles routing protocols, administrative functions, and packets which
require special processing. When these functions are shared on a single
processing unit, especially one with poor scheduling controls, it becomes
very easy to target that processor. Individual line cards with distributed
processors and caching tables may continue to forward packets, but without
the central route processor routing protocols will die. Given a
sufficiently strong attack, the router may not even be responsive at the
local console, as the CPU spends its time processing interrupts and
packets.

Some networking products, particularly "layer 3 switches", use a
cache-based architecture in which the first packet in a flow must always
hit a CPU, and future packets are handed by ASIC. Random source or
destination floods can wreck havoc on these systems, and many of them
cannot add new flows at anything close to line rate.

Attacks which disrupt performance of the router-processors can be
particularly bad when BGP is involved. If a BGP-speaking router is held
down long enough that its peers fail to receive a keep-alive response,
they will tear down the BGP connection and remove the associated routes.
When the attack can no longer be routed to its destination (potentially
affecting upstream routers trying to discard these now destination
unreachable packets), the original BGP speaker returns to service, and the
cycle repeats. This leads to routing flapping. Flapping leads to
dampening, dampening leads to suffering.

Juniper routers can fare much better against this kind of attack because
of their clean separation between packet forwarding and routing processes.
Even exceptional packets never touch the processor which handles routing
functionality.

One of the simplest ways for the central processors to be targeted is
packets directed at an IP of the router itself. These are considered
control communications, and are forwarded to the central processor for
further processing. Even a top of the line� Cisco GRP can be crippled by
as few as 20,000 packets per second. Ironically while some thought has
been given to SYN floods directed to open ports, such as those used for
remote administrative access (telnet), floods to random closed ports can
be worse.

Another way to generate exceptional packets is the use of IP options. Most
times this is more of a local network attack since the routers which
deliver the attack must struggle the attack as well.

Route caches can be heavily stressed by both random source and random
DESTINATION floods. One of the things which can add heavily to a victim
network's pain is the replies generated by a victim machine to the
random-sourced attack packets. In addition to the cache-thrashing, many
products must always handle the first packet of a flow at the process
level. Obviously any time a pre-generated forwarding table can be used
(such as Cisco CEF), you will see better performance.

High rates of packets per second can be generated with a twist in the
usual use of a �smurf� attack, using small packets to generate high rates
and router-harming effects instead of large packets to generate excessive
bandwidth use.

SYN floods are often the attack of choice, since this combines many of the
router-crippling properties in a single well known attack. Even when the
attackers are not thinking about the intended effects of their programs,
they are highly motivated by what works and what doesn�t.



Section 6 - LAN attacks

Many potential denial of services can be launched from within your
network. In fact, these may be some of the most effective because they can
slip past external defenses and potentially go unnoticed. Fortunately,
when you do figure out what�s happening you don�t have far to look for the
source.

One variant of the broadcast flood not commonly discussed is a link layer
broadcast flood. In a broadcast attack like �smurf�, packets are directed
at an IP broadcast address, which the router supported ip
directed-broadcasts converts into link layer broadcasts. Disabling it
prevents this attack from being used remotely, but it is still possible to
use it locally by sending frames with a link layer broadcast. If a raw
frame is generated, it is possible to spoof the mac address of the attack
generator, further confusing tracing of the attacker. Making things worse,
many switches which have the ability to limit broadcast storms cannot
distinguish between broadcast and multicast traffic, preventing its use if
you are trying to support a multicast service.

Another aspect of attacks involving layer 2 switches concerns their
behavior when an attack completely disables its target so that it can no
longer reply to ARP. When the address falls out of the switch�s arp cache,
the attack can end up being broadcast by the switch looking for the
correct destination port.

One well-meaning and sometimes useful feature which can be subverted for
evil purposes is ICMP Redirects. By spoofing these, an attacker can not
only cause a denial of service by providing false routes to anything they
choose, they can potentially redirect traffic to their own gateway, then
log or alter packets and forward them unnoticed. The same holds true for
ARP, and this is even more true for routing protocols which can have even
more wide ranging effects. Another area commonly overlooked is HSRP/VRRP
redundancy protocols, which have poor authentication.



Section 7 - Dropping packets on the floor

One disadvantage to having a good network is that you will successfully
receive more of an attack. Many attacks, no matter how strong, may be
self-regulated by an overloaded backbone or peer somewhere along the way.
This is one of the key reasons that Distributed DoS can be so devastating,
in addition to sheer numbers and bandwidth.

Cisco routers implement something known as TCP Intercept as part of the
firewall feature-set, which provides assistance to hosts in stopping SYN
floods. Unfortunately, it is only about as effective as a modern
well-designed OS. If you are aiming to protect highly vulnerable legacy
systems from relatively simple SYN floods, this can be an effective tool.
However, it is not recommended against high rate SYN floods which are the
subject of this presentation.

When faced with attacks designed to overwhelm a route processor or central
CPU, there may be scheduler options available. For example with Cisco IOS
there is a command �scheduler interval� or �scheduler allocate� which can
be used to set guaranteed amounts of processing time during which
interrupts will be masked and routing protocol processes are guaranteed
execution time.

In Cisco'�s, while flow caching can be highly useful for traffic analysis
and tracing spoofed IP streams without placing excessive load on older
more fragile routers, random IP packets can wreck havoc on it.

One of the other important areas to focus on is the role of layer 2
devices. While simple link layer switches may be useful for many tasks,
they also have a great potential to be abused. Anywhere that IP layer
routing decisions can be directly mapped to switch ports, through the use
of VLANs and 802.1q or layer 3 switches for example, can reduce the
potential for problems later. If this is not possible, you should
carefully weigh the values of quick network reconfiguration vs. possible
switch broadcast flooding when selecting an ARP cache timeout, and/or
statically assign the MAC addresses of high profile targets.



Section 8 - How to filters

The easiest solution to a Denial of Service attack, aside from not being
vulnerable and thus having to worry about it at all, is to filter the
attack. Quite simply this means you must determine what is good and what
is bad, and then proceed to separate them and discard the waste. This is
not as easy of a task as it seems, especially in the face of more devious
attacks which are designed to slip through or overwhelm filters. At first
glance some attacks might seem like they can�t be filtered at all, for
example an ACK flood. It would seem there is simply no good way to
separate an attack of ACKs without totally disrupting legitimate service.
But if you look closely, you�ll notice patterns which can be separated
from legitimate traffic.

Rate limits can be effective tools for stopping a flood while still
allowing for functionality at reasonable levels. They can also be used to
filter things which can not otherwise be filtered with simple �yes or no�
criteria. But applied incorrectly, rate limits can be not only ineffective
but can actually make matters worse. For example, if you rate limit SYNs
on a 100Mbps ethernet connection to 1Mbps, you'�ve just made yourself able
to be DoS�d with 1Mbps of traffic.



Section 9 - What to filter

Lets look at a quick example, SYN floods. Many SYN flooders have been
written, many have been published and used, but if you look at every
packet kiddy SYN flooder out there it�s missing one thing that essentially
every legitimate TCP/IP stack includes. An MSS option. No one has yet
written a SYN flooder which includes the MSS option and calculates the TCP
checksum correctly. If you can rate limit SYNs with this criteria, you�ve
just made major progress towards stopping SYN floods without affecting
legitimate traffic.

Another example, Stream, which was released as an ACK flooder. If you look
at the actual code you�ll notice something, it was obviously not
originally written as an ACK flooder. The TCP Sequence number is
randomized with every packet, while the Acknowledgement number remains
fixed at 0. The odds of an ack number of 0 occurring are 1 in 2^32, so if
you can place it in an extremely small rate limit you�ve just stopped all
out of the box Stream attacks without affecting legitimate traffic.

Obviously this is a very perilous road, since code can be improved and
patterns can be carefully studied and removed, but it gives some insight
into the nature of creative and granular filtering. It also requires more
detailed packet matching and pattern matching tools then are available on
many platforms. This kind of filtering usually ends up being used to
protect individual systems.



Section 10 - Protecting router interfaces

One of the most obvious targets for Denial of Service against the network
infrastructure is the IP which shows up in a traceroute. In order to
provide additional security against Denial of Service as well as
unauthorized access, many providers have chosen to use RFC1918 IP space to
number their router interconnects.

Unfortunately this presents problems of its own. In addition to the
property of not being globally routed, RFC1918 IPs are not globally
unique. This means there can be a potential conflict between the addresses
of two separate networks, when ICMP messages are generated by the routers
with RFC1918 source addresses. While this is not a serious problem, many
networks choose to filter RFC1918 sourced packets out of the mistaken
belief that it is, or that they gain some security by doing so. In
addition, using RFC1918 addresses reduces the ability to diagnose problems
and requires extra work to get informational DNS names.

A potentially better way get the same benefits is simply to use a common
block of real IPs which are not announced, or which are filtered at
network borders. This gives you global uniqueness and functional DNS,
while still preventing attacks from ever reaching their target. The
downside is much more extensive planning is required when assigning your
IP addresses, and using an unannounced block may not be possible due to IP
allocation constraints.

Another approach to hiding potentially vulnerable IPs is to make the
�real� IP a secondary address, and use a false address as the primary and
thus the source of ICMP messages generated. This can be used to provide
obscurity without being forced to change all references from an existing
numbering scheme. You should be aware of how this could potentially affect
your IGP before blindly trying this.

The common theme in these approaches is to plan your IP assignments well.


Section 11 - Protecting everyone else

In a perfect world, there would never be any spoofed packets allowed on
the internet. The world isn�t perfect, but every little bit helps. When
you stop spoofs from leaving your network, you not only help out others
who could potentially be attacked from your network, but you protect
yourself from being abused to attack others. A surprising number of
networks simply don�t give any consideration to this, and this is
particularly important for university networks and colo providers more so
then dialup providers.

One key thing to be aware of is the potential for one-off spoofing, or in
other words spoofing an IP which is still within the block of source
addresses allowed, but not the true IP of the attacker. With many
common-hack high bandwidth targets like universities deploying RFC2267
filters, this has become a theme of DDoS. Those networks which are
filtered simply pick a one-off IP for their source, while other unfiltered
attacking nodes choose a random source. When both types of nodes attack,
it is extremely difficult to tell which is a spoof and which is not even
if a large amount of contributing attack is non-spoofed. Even if the
attack is detected and the source is contacted, they will often pursue the
spoofed IP as the source of the attack, allowing the true attacker to
continue.

Stopping spoofing is not a guarantee of stopping DoS, it is still entirely
possible for an attacker to conduct a campaign using entirely hacked
throw-away machines and unspoofed attacks, even ones without root access.
But spoof-filters give one very important thing if used correctly,
accountability.



Section 12 - Assume = Ass + U + Me

One of the most overlooked areas of Denial of Service is that your
assumptions are not always right. Many times while trying to stop attacks,
well meaning people can jump to conclusions and create quite a bit of
trouble for innocent people.

Random source spoofed attacks can lead to �correct behavior�
random-destination replies. These can be mistaken for attacks or scanning
by overzealous admins.

Intentionally framing an innocent party for abuse is simple, and proof of
innocence is next to impossible. Only a slightly substantial basis for
accusation is needed.


- Richard A Steenbergen 
  rev 1.0   01/27/01


Back to the Index