ISQS 6342 Presentation

Post Mortem of compromised systems

Presented by : Pradeep

Important steps to be taken after a break-in Disable important credentials immediately

(PGP, SSH and SSL keys that may have been compromised)

Change any passwords that may have been compromised by sniffing / social engineering

Block flow of goods and money if financial systems have been compromised (may include closing bank accounts, stopping outgoing shipments etc. till detailed analysis is complete)

Find the cracker’s running processes Any program could have been

compromised, therefore it is safe to operate using an unprivileged user account that doesn’t have access to important things.

Keeping a stealth version of ps under an unassuming name would be helpful in such circumstances.

Handling deleted executables Crackers often remove executables from the file

system. This removes the reference to the name of the file in

the directory to be removed, however the file will still exist until all programs that have the file open close it.

If you can detect these executing programs, you could make copies of them

The symbolic link in /proc to the executable is good even though the original file has been removed from the file system

cp /proc/479/exe /homesamspade/del_cracker

Detecting Popular Trojan HorsesTripwire can be used to detect TrojansThe periodic use of tar –d or rpm also

works wellScanning the system for open ports

with a careful comparison to past results from netstat or ports would reveal suspicious ports not open in the past.

Suggested Immediate response Sever connections between the

computer and the outside world. (Disconnect from the network and modems)

Logs Check the /var/log logs Scanning tools are available. Scan daily

for critical servers (eg.logcheck) syslogd and and klogd daemons provide

standard logging mechanisms for daemons, other programs and the kernel.

Remote logging- /etc/syslog.conf file can accept an action to send messages to a remote Linux or Unix system for logging.

Interpreting Log file entries Sophisticated attackers alter log files

if they succeed in gaining root access. Quota limitations – Some crackers will

run your disk out of space so that actions cannot be logged.(No need of root access for this)

Some attackers truncate or remove log files before they exit.

Examining Log Files Command to find possible log

entriesdd bs=10k if=/dev/sda1 | grep

‘^May 1 ..:…: abcd’ | more

lastloglastlog – Shows the last time each

user has logged in. If what it shows is different from what you expect, it could be a useful indicator

messages file Catch-all for all the logs of many

processes Most systems have their

/etc/syslog.conf file configured to write to the messages file.

Intrusion attempts / break-ins all logged in along with “all is well” entries.

Interesting entries to watch for in the messages file PAM_pwdb entries

- Available with PAM. Logs the start and end of interactive sessions started via login .rsh or su

PAM_rhost_auth entries - Reveal things such as a remote system doing a rsh

(remote shell) and rcp request to your system Kernel entries - Show mounting of file systems, loading and unloading

removable media and device drivers

Interesting entries in messages fileftpd entries – Show when each FTP client

starts a session, the client system and user name and when the session ends

login entries – Shows unsuccessful login entries listing the user, the tty device and remote system. Login only logs the name of the account that someone unsuccessfully tried to log in on if it is an existing account. If an invalid account is used it shows an entry only as UNKNOWN.

Interesting entries in messages file sendmail entries - Show remote

systems connecting to your sendmail

syslogd entries – Show syslogd entries exiting, which actually might be a cracker stopping syslogd to avoid logging his actions

Interesting entries in messages file init entries – are made by init, the

initial non kernel process that forks all other processes in the system

named entries – are made by named, the DNS daemon. Typical entries would be for named starting, updating its zone information and rejected requests

Interesting entries in messages file lpd entries – Show errors encountered

by the Line Printer Daemon. These show incorrect configuration or possible exploits

dhcp entries – are from Dynamic Host Configuration Program Daemon that allow a central server to specify the IP address that your system should use.

Interesting entries in messages file Last message repeated entries –

are used when a message occurs a number of times in succession. Indicates how many times it has been repeated to avoid many lines of log file entries for a repeated event, such as being out of memory or encountering bad disk sectors.

syslog Unlike messages file syslog logs

only problems. Typical problems logged are failed attempts to su, sendmail problems, syslogd conditions and in.telnetd refusing access.

kernlog /etc/syslog.conf file should be

configured to log kernel messages. The file should have a line similar to

Kern.* /var/log/kernlogLogs messages like ‘device drivers

being loaded’, system reboots, attempts to write to a floppy set Read/Only

cron This file logs each command that

the cron daemon , crond forks.

xferlog xferlog file This is a log of FTP transfers that

may show what files the cracker copied on / off your system

daemon This file logs activities by other

daemons which were not discussed earlier

mail This file sometimes called maillog

contains an entry for each piece of e-mail sent into and out of the system

Other logs that may reveal information Shell history files for root and other

accounts User’s mailboxes /tmp, /usr/tmp and /var/tmp Hidden directories such as /home/*/.??* Other cracker created files frequently

beginning with “.” Back up tapes Freespace in the file system Logs of other systems such as firewalls,

intermediate, and the ISP’s systems

Check TCP Wrapper responses TCP wrappers log attempted

connections which are denied due to rules specified in /etc/hosts.allow and /etc/hosts.deny

Copies of vital programs Crackers normally alter ps, ls, who

and other trusted programs. It is advisable to keep copies of these basic programs buried in an obscure directory so that they can be used in emergencies.

Finding the attacker’s system Tracing a numeric IP address nslookupnslookup –type=any 4.25.9.192 .in –addr.arpaServer: mindspring.comAddress : 207.69.200.2014.25.9.192.in –addr.arpa name=pluto.sun.COM

dig dig –x 192.9.25.4;; ANSWERS:4.25.9.192.in-addr.arpa. 86400 PTR

pluto.sun.COM

Finding .com owners http://www.networksolutions.com/cgi-bin/whois/whoisFinding entities directly from the IP

addressFor American entitieswww.arin.net/whois/arinwhois.htmlFor European entitieswww.ripe.net/db/whois.htmlFor Far East and Pacific entitieswww.apnic.net/apnic-bin/whois.pl

Government and military sites Looking up Government sites http://www.nic.gov/whois.htmlTo trace a US military address usehttp://www.nic.mil/

Node detection and tracking ping – to check whether a node is

up or not and on the internet

traceroute – useful for tracing the route to a node , such as your crackers

Legal Procedures Some state and federal agencies are now

very well set up to investigate and follow through to getting a conviction.

Amount of help varies tremendously between jurisdictions and even between different offices of the same agency.

Complaint should be backed by proper evidence and substantial proof of damage

FBI Investigates all crimes involving interstate

commerce. Major investigation of ordinary crimes is done only if there is substantial dollar loss.

Minimum loss to get FBI’s attention probably $ 3,000 - $ 25,000

Damage threshold much less for more serious issues like espionage, bank thefts, cases involving viruses and high visibility cases.

FBI Higher priority for banks, airlines,

US government agencies and classified data theft cases.

FBI collects information and presents it to US Department of Justice

US Secret Service Handles intrusions involving credit card

fraud and illegal publication of credit card information

Also handles cases of access device fraud (Includes fraudulent use of passwords)

Atlanta office handles credit card fraud ranging from $50000 - $ 100,000. Smaller jurisdiction’s threshold may be $ 10,000

US Secret Service A e-commerce site that suffers

large theft of credit card data should contact the Secret Service.

FBI and US Secret Service work jointly on many cases involving financial crimes.

Other Federal Agencies If military computers are involved,

contact the controlling military branch. The Army, Navy, Air Force, Marines and

Coast Guard all have separate Military Police agencies.

CIA has jurisdiction if someone located outside the US attempts to get at confidential US data.

In some cases , even agencies such as the Bureau of Alcohol, Tobacco and Firearms will have jurisdiction.

State Agencies State of Georgia State of North Carolina State of New York

Generally, local police are not competent enough to handle computer crime.

Care of evidence Defense attorneys tend to question

the chain of evidence. Any tampered evidence is considered tainted.

All evidence must be securely locked or must be under guard at all times.

Liability of ISP’s allowing illegal activities ISPs hopscotch between the laws that apply to

common carriers (telephone companies) and publishers.

Telephone companies are not liable for what is spoken over the phone.

Newspapers that publish information without reasonably checking the truth are liable for libel.

If an ISP does not edit the contents of its Websites or the e-mail that it handles, it is considered a common carrier and is exempt from libel.

Counter Offenses Not advisable Many times the attack might be

originating from a compromised system.

Spamming Ping of death

Hostile Java applets Malicious Java applets on Web

pages that can cause serious harm to a cracker’s system such as reformatting the hard disk.

US military has used this technique against US citizens.

This can be effectively used against enemy nations in the event of war

Black Bag Jobs It is rumored that some entities will

send someone to the cracker’s home and remove his equipment and destroy it.

Risky from legal stand point therefore not practical

ConclusionDisconnecting the system from the network is the only safe way to protect your machine in the networked world

ReferencesReal World Linux Securityhttp://www.tripwire.comhttp://www.readnotify.comhttp://www.linuxsecurity.com/docs/http://lsap.org/http://www.linux-sec.net/