Google

Responding to a Security Incident

Responding to a Security Incident

Contact:[email protected]

Jose Nazario Autumn, 2000 
This article focuses on gathering information about systems or networks
that are trying to compromise your security. We discuss using the
network databases to find contact information, what information to
include and what style of memo to write. 


------------[ Introduction
By now, nearly everyone who has been using Linux for some time and had
their system connected to the Internet has seen attempts to compromise
their security. The next questions that often comes up is what to do
about it. Unless it's a financial or safety issue, it's probably going
to get laughed at by the legal authorities, but it's worth reporting. 

I spend a good chunk of my time on mailing lists and organizations
concerned with monitoring hacker activity. Such lists are the INCIDENTS
list from SecurityFocus.com and the SANS GIAC effort, providing a daily
update of hacker activities from various parties around the world.
Often, the question of the value of reporting an incident is debated. I
routinely counsel people to report most incidents they see. What this
does for the ISP to whom you are reporting this is a set of
independently correlated data about a nefarious customer, or a
compromised machine on their network. Just don't expect much to be done
about it, to be honest with you. Most ISPs don't react, and aren't very
neighborly. Some of us in the business routinely block entire networks
from connecting to our networks based on patterns of allowing unseemly
activity to continue. 

We'll not go into detecting incidents, but we will define them as port
probes, port scans, denial of service (DoS) attempts, and unauthorized
access attempts. Each of these warrants investigation, some more than
others. Combining intrusion detection software with log analysis (which
you should be doing anyhow), these events should stand out. 


-----------[ An Example Detect
We'll start with a simple detect and extract information from it,
deciding how best to proceed. This first one is a log message of a
refused connect request from a service wrapped using TCP wrappers
(tcpd). As of this writing, FTP sweeps are all the rage: 

$ grep refuse /var/log/secure
Nov 8 15:26:31 linux ftpd[3689]: refused connect from
7dyn94.ztm.casema.net 

TCP_wrappers makes an excellent basic intrusion detection system,
logging both successes and failures. You should be using at least
TCP_wrappers for access control, and minimizing what services people can
connect to at all. 

A brief note about connection attempts: One important thing that gets
overlooked by the casual paranoid Internet user is the threshold of what
defines a probe or an attack, and what is worth reporting. Usually it's
not worth reporting SMTP (port 25/TCP) or HTTP (port 80/TCP) probes,
unless you can demonstrate that it was more than a simple connection
attempt. Why? Say I see you on a chatline, and I want to send you some
mail directly, or see if you have an interesting website. If you are
blocking those requests, they would show up as port probes. But they're
usually more innocent than they may appear. The same goes for ping
requests (ICMP_ECHO), which are used, for example, in gaming an Napster
to determine latency. Don't get in a tizzy about simple things, focus on
actual intrusion attempts, like an FTP exploit or a network sweep for a
service. 

--------------[ Finding Information 
One of the first things we do is to to map an IP address to that
hostname. I prefer to work with IP addresses rather than hostname as
sometimes ISPs will sell their netblocks in smaller quantities, yet
retain administrative control of them: 

$ nslookup 7dyn94.ztm.casema.net 

Name: 7dyn94.ztm.casema.net
Address: 212.64.110.94 

Now we begin to dig around for domain information using this network
address. You can use command line whois information to find out about
this network address. 

Two different major types of databases exist, depending on if you are
searching using hostname or network addresses. Network names are handled
by registries like Network Solutions, and network numbers are handled by
the ARIN database, or the American Registry of Internet Numbers. Because
we're using a network number, we'll be using the ARIN database, but
we'll do both to illustrate how it is done. 

A quick note on using whois. You can specify query@whois-engine using
fwhois, which is very common on Linux boxes. However, this doesn't work
on normal whois clients. You have to use the -h whois-engine option,
instead. Hence, the queries fwhois [email protected]
and whois -h whois.networkslutions.com domain.com are equivalent. In
these examples, we'll be using the fwhois command line tool or web based
whois searches, and whois is just a symbolic link to fwhois. 

First, the ARIN search: 

$ whois [email protected] 
[whois.arin.net]
European Regional Internet Registry/RIPE NCC (NET-RIPE-NCC-)
   These addresses have been further assigned to European users.
   Contact information can be found in the RIPE database, via the
   WHOIS and TELNET servers at whois.ripe.net, and at
   http://www.ripe.net/db/whois.html

   Netname: RIPE-NCC-212
   Netblock: 212.0.0.0 - 212.255.255.255
   Maintainer: RIPE

   Coordinator:
      RIPE Network Coordination Centre  (RIPE-NCC-ARIN) [email protected]
      +31 20 535 4444
Fax- - +31 20 535 4445

   Domain System inverse mapping provided by:

   NS.RIPE.NET                  193.0.0.193
   NS.EU.NET                    192.16.202.11
   AUTH03.NS.UU.NET             198.6.1.83
   NS2.NIC.FR                   192.93.0.4
   SUNIC.SUNET.SE               192.36.125.2
   MUNNARI.OZ.AU                128.250.1.21
   NS.APNIC.NET                 203.37.255.97

   To search on arbitrary strings, see the Database page on
   the RIPE NCC web-site at http://www.ripe.net/db/

   Record last updated on 16-Oct-1998.
   Database last updated on 9-Nov-2000 07:02:34 EDT.

The ARIN Registration Services Host contains ONLY Internet
Network Information: Networks, ASN's, and related POC's.
Please use the whois server at rs.internic.net for DOMAIN related
Information and whois.nic.mil for NIPRNET Information.

 
 


Aside from all of the extraneous information, we can see that with this
record that ARIN doesn't handle exact information for this network
block. We'll then use network name in a query: 

$ whois [email protected] 
Registrant:
N.V. Casema (CASEMA-DOM)
   P.O. Box 345
   Delft, 2600 AH
   THE NETHERLANDS

   Domain Name: CASEMA.NET

   Administrative Contact:
      Network Operations Centre  (NOC137-ORG)  [email protected]
      EuroNet Internet BV
      Muiderstraat 1
      Amsterdam
      NL
      +31 20 5355555
      Fax- +31 20 5355400
   Technical Contact, Zone Contact:
      Davids, Marco  (MD2446)  [email protected]
      N.V. Casema - IKC
      Brassersplein 2
      Delft
      ZH
      2612 CT
      NL
      +31(0)15 8881000 (FAX) +31(0)15 8881099
   Billing Contact:
      Finance Departement  (FD5-ORG)  [email protected]
      EuroNet Internet BV
      Postbus 11095
      Amsterdam
      NL
      +31 20 5355555
      Fax- +31 20 5355400

   Record last updated on 13-Jun-2000.
   Record expires on 30-Jan-2001.
   Record created on 28-Jan-1997.
   Database last updated on 7-Nov-2000 19:15:09 EST.

   Domain servers in listed order:

   NS.CASEMA.NL                 195.96.96.97
   NS1.CASEMA.NET               195.96.96.33


 
 


We now have the information we need to proceed. We'll be sending our
memo to the addresses listed as the billing contact, technical contact
and the administrative contact. For good measure, we'll toss in the
often used addresses security@ and abuse@, hopefully targetting the
right people. 

Additional information you may want to gather include a traceroute to
find their upstream network providers and the AUP (acceptable use
policy) of the ISP. Usually digging around on www.domain.com can turn it
up. Very rarely do ISPs have hacker friendly AUP terms, but it's been
known to happen. 

---------[ The Memo
Now we'll put this information into a sample memo to send to the domain
contacts to register our complaint. Remember the following things: 

Addresses that you record can be forged quite easily 
It's usually not the staff of the ISP or network that are attempting to
violate security measures 
Further general rules of thumb that have proven pretty successful over
the years for many people are: 
If it's a dedicated line, like a cable modem or a DSL line, it's
probably a compromised machine 
If it's a dial-up line, it is often a stolen account 
With this in mind we can craft our memo. 
Things that must go into the memo are: 

Your identification, including name, organization (if any), and role in
that organization 
The brief purpose of why you are contacting them 
Log output to prove your point, time stamped 
The address of the host that was hit 
Your timezone information, preferably with an offset of GMT (Greenwich
Mean Time). 
If a compromise occured 
I always add for good measure: 

A very friendly tone 
A consideration to make is to PGP sign your message. This will provide a
stronger signature for you, and provide proof that you sent the message
to the ISP. This will also help verify that the message was received
without it being modified in transit. If you do this, include in your
memo the location of where you PGP key can be downloaded. In your
signature is a good place. 

Don't be stupid, don't make threatening gestures, don't threaten legal
action unless you have spoken to a lawyer about things, don't demand
action be taken, and don't threaten retaliation. Don't even retaliate,
you'll just become guilty of a violation of your AUP as well. I have
seen this numerous times before, please do not fall victim to it as
well. 

Below is a sample memo, one which mirrors one that I would send
regarding the the above detected incident: 

To: [email protected], [email protected],
[email protected],
[email protected], [email protected], [email protected] 
From: Jose Nazario ([email protected]) 
Subject: [SECURITY] FTP probe from casema.net domain 

Good day,

My name is Jose Nazario and I am a customer of BigCorp ISP. I am writing
to you today to note that a machine I own was probed by a host in the
casema.net network. This was a probe to the FTP daemon, port 21/TCP. You
are listed as a responsible party in the domain records. As there are a
number of problems with FTP servers currently making the rounds with
hackers, this may represent someone attempting to find find vulnerable
hosts to compromise. This may represent a legitimate user in violation
of
their AUP or a compromised machine on your network.

The host on my network that received the traffic from your domain is
myhost.bigisp.com (10.10.32.4). The log entry for this incident looks
like 
this:

Nov 8 15:26:31 linux ftpd[3689]: refused connect from 
        7dyn94.ztm.casema.net

All times are in US Eastern (GMT-5).

While no reply is expected, the favor of an acknowledgment would be
appreciated.

Thank you for your attention to this matter,

Jose Nazario                       [email protected]
			http://www.bigisp.com/~jnazario/

 
 



--------[ Contacting Legal Authorities
This is the first thing that many people think about when documenting a
security incident. Most people have this image of a scruffy 15 year old
hacker being led to a prison in handcuffs. The truth is this rarely
happens, even with the best of documentation on your end. The reasons
for this are many and varied, but can be summarized in large measure by
the difficulty in proving who was using what system at what time, and
the forensic value of the evidence. 
The FBI will not investigate a security incident unless the monetary
damage is above US$5000, someone's life was in danger, or interstate
commerce was affected. Furthermore, even then the evidence may have lost
forensic value for a ciminal prosecution. 

If you think the legal authorities are best contacted, you should speak
to your site supervisors and any legal advisors and ensure that you have
a plan before any security incidents take place. Two books you may want
to begin with are listed below in the references section. 

--------------------[ Contacting CERT Organizations 
One questions people have is 'Should I contact CERT or a similar
organization with this information?' In my experience, it is usually not
necessary except under pretty uncomon circumstances. 
Most of the incidents you will see are probes of one type or another.
Port scans, probes for services like RPC services or DNS servers, maybe
even a few web probes for cgi-bin scripts, but just probes. While it's
pretty pbvious they're sizing you up for an attempt to break in, they
didn't get in and they didn't cause any damage. 

Times when I have contacted CERT here in the United States are when
legitimate intrusions have taken place, novel exploits have been used
which are not documented in the security community, and when the system
has been used to gain entry into other computers or for DoS attacks.
This helps provide a central place for the information on the attack to
be stored and evaluated, and, potentially, a third party to show that
actions were being taken to remedy the situation. Also, contacting CERT
is a good idea if the incident is above a probe, such as a real
documented attempt or a successful break-in, and occured from a host
outside of the United States. 

CERT type organizations exist all over the world, and are worth
reporting to if you file an incident with CERT. File a similar report
with the organization in the originating country. A comprehensive list
of CERT and similar organizations from around the world can be found at
FIRST's contact information page. FIRST is an organization that provides
a forum for incident handling. 

Note that CERT has no legal authority, but does work with the
authorities to investigate security incidents when they are warranted. 


--------------[ Now What? 
So, what will happen now? Should you sit back and wait for the ISP you
just contacted to report back, "Wow! We got rid of that user!" Think
you're going to break up a large international ring of hackers just
trying to take over the Pentagon? 
Think again. 

You'll be lucky to get any notification that your memo was received,
unless it's an automated reply. You'll probably not get any followup
done, either, on their end. So why bother? 

Rarely do potential attackers work just on you. Unless you hang out on
the Internet and antagonize people for a hobby, or a living, chances are
they're not targetting you, but they're doing wide sweeps of the
Internet to find vulnerable hosts. What you've done is you have provided
the ISP with data that says 'They were here, too'. This is why you
bother. If enough people complain they'll see it's a real problem, and
probably fix it. 

Remember, also, that the ISP staff are busy people. Routers explode,
billing situations come up with their customers, new hardware has to be
installed, in short, they have daily business to take care of. Their
response will be tailored to the severity of the incident. One port
probe on one machine is nothing to worry about for them. If you can
demonstrate that it was a network scale issue or a recurring problem,
then they may start to care. 

So, carry on with your network activities, keep a watchful eye out for
problems, and improve your defenses as needed. 

------------------[ Summary 
The most important thing to keep in mind when reporting an incident is
perspective. Keep in mind the scale of the probe or attack so you can
respond with at the appropriate level; the perspective of the person you
are contacting so that they can act on the information; and the larger
perspective of life on the Internet. I hope this article has been
informational for you about how to contact a network person when you
think you've seen a security incident. Below are some resources you may
find useful. 
---------[ Resources 
The SecurityFocus INCIDENTS list is an excellent mailing list about
security incidents. Sit back and learn. http://www.securityfocus.com/. 
The SANS GIAC effort is also a good place to learn about detects, what
generates them and the like. http://www.sans.org/giac.htm. 

The GeekTools Whois Proxy is my favorite method of searching for domain
information about a hostname, a network name or even a numerical
address. It's smart enough to hit the right servers for information. You
can also download the source and run your own local instance.
http://www.geektools.com/cgi-bin/proxy.cgi. 

Documentation on using TCP_wrappers and general Linux hardening can be
found at http://www.enteract.com/~lspitz/linux.html. 

Using grep on your logs will make your life a lot easier. A good piece
of documentation on it is at
http://www.sunworld.com/sunworldonline/swol-11-1999/swol-11-unix101.html
. 

Two books in investigating computer crimes for a legal standpoint are 1)
Saferstein, Richard. Criminalistics: An Introduction to Forensic
Science, Sixth Edition, Prentice Hall, Upper Saddle River, New Jersey,
1998, and 2) Clark, Franklin and Diliberto, Ken. Investigating Computer
Crime CRC Press, New York, 1996


Back to the Index