Post on 14-Dec-2015
Network Monitoring & Troubleshooting plus Log
AnalysisFaculty:
Scott Greeneof
Evidence Solutions, Inc.
Scott@EvidenceSolutions.comwww.EvidenceSolutions.com
►10 signs that you aren't cut out for IT 1: You lack patience
10 Signs you aren’t cut out for IT
U of Nebraska Incident
►An Undergrad suspected in Univ. of Nebraska breach where more than 650K personal records were compromised in attack.
►The intrusion was into a university database containing personal information on more than 650,000 students, parents and employees.
U of Nebraska Incident
►The intrusion, which was described by university officials as a "skilled attack," exposed the Social Security Numbers (SSNs), names, addresses, course grades financial aid and other information on students who attended the university since 1985.
U of Nebraska Incident
►The breach also exposed personal data and financial information for parents of students who applied for financial aid at UNL, according to the university.
►A staff member in UNL's Computing Services Network discovered the breach in the Nebraska Student Information System (NeSIS) on May 23.
U of Nebraska Incident
►An Undergrad suspected in Univ. of Nebraska breach where more than 650K personal records were compromised in attack.
►The intrusion was into a university database containing personal information on more than 650,000 students, parents and employees.
U of Nebraska Incident
►The system manages student admissions, campus housing and course registration.
►It was built over a three-year period at a cost of $29.9 million, has been operational for the past two years and is based on Oracle's PeopleSoft Enterprise Campus Solution platform.
U of Nebraska Incident
►An FAQ on the incident posted by the university makes it clear that personal data in the breached server was not encrypted. "However, we are confident that the type of attack we experienced would have bypassed any encryption that was in place," the university said.
U of Nebraska Incident
►The vulnerability that enabled the intrusion has since been closed and the university is currently working with a third-party firm to review and address remaining vulnerabilities.
20 Critical Security Controls
►1) Inventory of Authorized & Unauthorized Devices
►2) Inventory of Authorized & Unauthorized Software
►3) Secure Configurations for Hardware & Software on Laptops, Workstations, & Servers
20 Critical Security Controls
►4) Continuous Vulnerability Assessment & Remediation
►5) Malware Defenses
20 Critical Security Controls
►6) Application Software Security Code Reviews Proper Logging Abnormal operation reporting
20 Critical Security Controls
►7) Wireless Device Control►8) Data Recovery Capability►9) Security Skills Assessment and
Appropriate Training to Fill Gaps►10) Secure Configurations for Network
Devices such as Firewalls, Routers, and Switches
20 Critical Security Controls
►11) Limitation and Control of Network Ports, Protocols, and Services Including custom applications Development departments need to
communicate with network departments
20 Critical Security Controls
►12) Controlled Use of Administrative Privileges
►13) Boundary Defense See 10 Include penetration testing Include review of firewall rules Remote Users Mobile Devices
20 Critical Security Controls
►14) Maintenance, Monitoring, & Analysis of Security Audit Logs
►15) Controlled Access Based on the Need to Know
20 Critical Security Controls
►16) Account Monitoring and Control What do users have rights to
►Why? What do processes have rights to
►Why?
20 Critical Security Controls
►17) Data Loss Prevention►18) Incident Response Capability
Who responds Test those responses Who gets notified
►Hr►Legal
20 Critical Security Controls
►19) Secure Network Engineering aka “Develop a Secure Infrastructure”
►20) Penetration testing
#14
►Validate audit log settings for each hardware device and the software installed on it, ensuring that logs include: Date Timestamp source addresses destination addresses Any other useful information
#14
►Normalize Logs Syslog Common Event Expression initiative Use normalization tools to convert logs
#14
►Reports Security personnel and/or system
administrators should run weekly reports that identify anomalies in logs.
They should then actively review the anomalies, documenting their findings.►A log for the log events
#14
►Time Synch Use at least two synchronized time
sources All servers and network equipment should
be in synch.►Test►Validate their synchness
Federal Security Standards
►NIST Special Publication (SP) 800-37 Categorize
►The information system and the information processed, stored, and transmitted by that system based on an impact analysis
Federal Security Standards
►NIST Special Publication (SP) 800-37 Baseline
►Select an initial set of baseline security controls for the information system based on the security categorization; tailoring and supplementing the security control baseline as needed based on an organizational assessment of risk and local conditions.
Federal Security Standards
►NIST Special Publication (SP) 800-37 Implement
►The security controls and describe how the controls are employed within the information system and its environment of operation.
Federal Security Standards
►NIST Special Publication (SP) 800-37 Assess
►Assess the security controls using appropriate assessment procedures to determine the extent to which the controls are implemented correctly, operating as intended, and producing the desired outcome with respect to meeting the security requirements for the system.
Federal Security Standards
►NIST Special Publication (SP) 800-37 Authorize
►Information system operation based on a determination of the risk to organizational operations and assets, individuals, other organizations, and the Nation resulting from the operation of the information system and the decision that this risk is acceptable.
Federal Security Standards
►NIST Special Publication (SP) 800-37 Monitor
►The security controls in the information system on an ongoing basis including assessing control effectiveness, documenting changes to the system or its environment of operation, conducting security impact analyses of the associated changes, and reporting the security state of the system to designated organizational officials.
Federal Security Standards
►NIST Special Publication (SP) 800-37►The final step in the cycle, Monitor, is of
particular importance because it evaluates the effectiveness of a security control. But what if you only performed this evaluation periodically—for example, to satisfy a quarterly or annual audit for a regulation or other compliance related demand? Unfortunately, it could be months or even a year before you’d realize that the security control was not functioning as intended.
Federal Security Standards
►NIST Special Publication (SP) 800-37 Assess
►Assess the security controls using appropriate assessment procedures to determine the extent to which the controls are implemented correctly, operating as intended, and producing the desired outcome with respect to meeting the security requirements for the system.
Some Things to Monitor
►Patch management►Network management tools►Security tools such as:
Change management Configuration management Log monitoring Vulnerability scanning solutions
►10 signs that you aren't cut out for IT 1: You lack patience 2: You have no desire to continue
your education
10 Signs you aren’t cut out for IT
Logs?
►Syslog is the predominant standard for computer system logging
►Microsoft, in its infinite wisdom chose their own called “Windows Event Log”. There are several converters to convert
the Windows Event Log to the Syslog standard.
Log Log Log
►Many incidents can be readily revealed with a bit of logging and analysis those logs.
Logs
►Solutions Almost everything that has a log should
have the log turned on. Logs should include:
►Date/time►Source IP►Destination IP►Port►Etc
Logs
►Solutions Use standard SYSLOG entries or use
software that converts logs to a common log format.
Store logs for a while – space & DVDs are cheap
Create systems & procedures for analyzing logs.►These systems should have ‘normal’ items and
‘abnormal’ items
Logs
►Solutions All remote access logging:
►should be in detail►Should be rigorously analyzed.
All security alerts should be logged.►Workstation►Servers►Devices
Logs
►Solutions Use unified time
►This allows logs to be matched up across many devices and / or networks.
Border devices►Should log verbosely►Should log all traffic
Blocked Allowed
Logs
►Solutions Logs should be secured Logs should be exported & saved on Write
Once devices.or Logs should be written to dedicated
logging servers. The dedicated logging servers with
separate security credentials
Logs
►Solutions Test the logs and review after:
►Normal / acceptable traffic►Push the system►Attempt to penetrate the network.
Inside Outside
►Compare and correlate the data on all of the logs for validity.
Logs
►Solutions Review
►Logs everyday►Use automated tools to analyze large amounts
of data. Test
►Attack a system►Test the response time.
Discovery Action taken to attack
Log Review Tools
►Windows -> Syslog conversion Snare agent (
intersectalliance.com/projects/index.html) and ProjectLasso remote collector (sourceforge.net/projects/lassolog) are used to convert Windows Event Logs into syslog, a key component of any log management infrastructure today (at least until Visa/W7 log aggregation tools become mainstream
►10 signs that you aren't cut out for IT 1: You lack patience 2: You have no desire to continue
your education 3: You refuse to work outside 9-to-5
10 Signs you aren’t cut out for IT
Database Activity Management
►Database Activity Monitoring (DAM) is a database security technology
for monitoring and analyzing database activity that operates independently of the database management system (DBMS)
It does not rely on any form of native (DBMS-resident) auditing or native logs such as trace or transaction logs.
DAM is typically performed continuously and in real-time.
Database Activity Management
►Add prevention and you get (DAMP) This extension to DAM goes beyond
monitoring and alerting to also block unauthorized activities.
DAM helps organizations address compliance:►HIPAA►PCIDSS►Sarbanes-Oxley (SOX)►NIST 800-53
Database Activity Management
►Features include: Event aggregation Correlation Reporting Auditing
►Does not require access to native database audit functions
Database Activity Management
►Privileged User Monitoring: Monitoring privileged users:
►DBAs►Sysadmins►Developers
who typically have unfettered access to corporate databases
Protects against external and internal threats
Database Activity Management
Monitors all activities and transactions Identifies anomalous activities
►Viewing sensitive data►Creating new accounts
(with superuser privileges?)
►Adding or Deleting tables
Database Activity Management
►Most organizations have perimeter protection
►The next need is to monitor and protect privileged user accounts
Database Activity Management
►There is a high correlation between and protection from the insider.
►Privileged users are capable of: Stored procedures Triggers Views
Database Activity Management
►Targeted attacks frequently result in attackers gaining privileged user credentials: Monitoring of privileged activities is also
an effective way to identify compromised systems.
Database Activity Management
►Privileged user monitoring helps ensure: Data Privacy Integrity
Implementation of Logs
►Procedures and Tools to Implement and Automate this Control Most Everything allows logging Evaluate what is and what is not being
logged.►compare them with the asset inventory
Implemenatation of Logs
►Manual inspection analyze logs on individual devices correlation (SIEM) tools can make audit
logs far more useful
Implementation of Logs
►SIEM & Consolidation tools can be quite helpful in identifying subtle attacks. These tools are not a replacement for
skilled information security personnel and system administrators.
Even with automated log analysis tools, human expertise and intuition are often required to identify and understand attacks.
Log Measurement - Manual
►Item: Network time protocol (NTP) Measurement: Confirm that NTP is
being used to synchronize time for all devices and that all clocks are in synch.►Pass or fail.
Log Measurement - Manual
►Item: Vulnerability scanner Measurement: Run a non intrusive
vulnerability scanner against random servers.►Review logs to determine whether the
information appeared in the logs.Pass or fail.
Log Measurement - Manual
►Item: Security Event Information Management system (SEIM / SIEM / etc) Measurement: Correlate logs to a
central source and determine that all servers are properly logging.►Compare to inventory list►Start @ 100% and back off 5% for each
device not logging.
SIEM
►security information and event management
►security incident and event management
SIEM
►SIEM is a computerized tool used on data
networks to centralize the storage and interpretation of logs, and events.
The logs and events are generated by other hardware and software products on the network.
SIEM
►SIEM should include: Gathering the logs
SIEM
►SIEM should collect logs from: Syslogs Firewall logs IDS logs Windows server logs Database logs Web server logs Application logs.
SIEM
►SIEM may need to: Handle multiple data centers Collect data centrally
SIEM
►SIEM Reports: Correlating
►Their big value is in the correlation of data from multiple sources in multiple formats.
Regulatory Trouble-shooting Investigating Alerting
SIEM
►SIEM can also detect: Distributed attacks Complicated attack paths Insider abuse
►As well as: Normal network performance failures
►(Requires a capable analyst)
Commercial Software Products
►CorreLog Enterprise Server – software $5k
►ManageEngine EventLog Analyzer $350 / 10 sources
►NitroSecurity NitroView $29k►Prism Microsystems EventTracker $30k►Tripwire Log Center $19k
►10 signs that you aren't cut out for IT 1: You lack patience 2: You have no desire to continue
your education 3: You refuse to work outside 9-to-5 4: You don’t like people
10 Signs you aren’t cut out for IT
NMap
►Nmap A security scanner originally written by
Gordon Lyon (aka Fyodor Vaskovich)►1) used to discover hosts►2) user to discover services on hosts►3) can determine;
which ports are open and closed the operating system names and versions of the listening services estimated uptime type of device presence of a firewall.
Nmap
NMap
►DEMO
►10 signs that you aren't cut out for IT 1: You lack patience 2: You have no desire to continue
your education 3: You refuse to work outside 9-to-5 4: You don’t like people 5: You give up quickly
10 Signs you aren’t cut out for IT
Windows Log Analysis
►Free Products LogZilla (code.google.com/p/php-syslog-
ng) Analyzes Syslogs Is PHP-based visual front-end
►For syslog servers Searches Reports etc
Windows Log Analysis
►Free Products Splunk ( Free for first 500mb )
Windows Log Analysis
►Free Products►OSSEC (ossec.net):
an open source tool Analyzes Real-Time:
►Unix systems►Windows servers►Network devices
Includes:►default alerting rules
Windows Log Analysis
►Free Products Snare agent & ProjectLasso (
intersectalliance.com/projects/index.html) and remote collector (sourceforge.net/projects/lassolog)►Open sourc tool►Analyzes real-time:
Windows Event Logs Syslog Network devices
Windows Log Analysis
►Free Products Log2timeline (log2timeline.net/) Analyzes Logs
►Used as an investigation tool it can create a timeline view out of raw log data
►Runs on Linux and Mac using Perl
Windows Log Analysis
►Free Products syslog-ng (
balabit.com/network-security/syslog-ng/) Open Source Analyzes
► Windows Logs by installing an agent on the server
►Syslog
SpiceWorks
►Spiceworks headquartered in Austin, Texas.
►It was formed in early 2006 to provide a Facebook-like community integrated with a free ad-supported IT: Systems Management Inventory Help desk software
SpiceWorks
►The product is designed for network administrators working in small- to mid-sized businesses and managing up to a few thousand network devices.
SpiceWorks
►SpiceWorks discovers:► Windows► Unix► Linux► Mac OS X► Routers► VOIP phones► Printers► etc.
SpiceWorks
►DEMO
►10 signs that you aren't cut out for IT 1: You lack patience 2: You have no desire to continue
your education 3: You refuse to work outside 9-to-5 4: You don’t like people 5: You give up quickly 6: You’re easily frustrated
10 Signs you aren’t cut out for IT
What exactly is WireShark?
►Wireshark is a sniffer Sniffer, Packet Analyzer, also known as a
network analyzer, protocol analyzer. A software product and / or hardware product
that has the ability to intercept all network traffic and allow for the analysis of the packets of data contained in that traffic.
What exactly is WireShark?
►Originally named Ethereal, Wireshark is a free and open-source packet analyzer. It is used for network troubleshooting, analysis, software and communications protocol development, and education
►It runs on all popular computing platforms, including Unix, Linux, and Windows.
What is a sniffer?
►Protocol Analyzer Who is talking to who What they are saying Header and Overhead Payload Problems
Use of a sniffer for security:
►Basic Information: 7 layer OSI model
►Application►Presentation►Session►Transport►Network►Datalink►Physical layers
Use of a sniffer for security:
►Many believe that you look at everything from the bottom up just like:
Building a building Math Employment, etc
►But, that is not always true…. With a house you start with the overall design….
►In the case of sniffing It is best to start somewhere in the middle, usually
at TCP or ICMP, then move down or up based upon what you discover and what you are looking for.
Requirements of a Sniffer
►Hubs►Switches►Switches
Port Mirroring Switched Port Analyzer (SPAN) Roving Analysis Port (RAP)
►Local Connection must operate in promiscuous mode
Example Sequence 1
►DHCP request and response►ARP for gateway►DNS “A” request and response►Web session setup►Payload delivery
DHCP
DHCP: src & dst
DHCP: IP Component
DHCP: UDP Component
DHCP: BootStrap Parameters
DHCP: First Packet Results
DHCP: Second Packet Results
DHCP: Third Packet Results
DHCP: Fourth Packet Results
►10 signs that you aren't cut out for IT 1: You lack patience 2: You have no desire to continue your
education 3: You refuse to work outside 9-to-5 4: You don’t like people 5: You give up quickly 6: You’re easily frustrated 7: You can’t multitask
10 Signs you aren’t cut out for IT
DNS
►DNS is the internet’s system to translate the human readable name such as EvidenceSolutions.com to the actual IP address of the desired site.
DNS: Request
DNS: Response
We get to the website: syn
We get to the website: syn & ack
Website final: ack
Website Sends Graphics
How to use the product….
►Basics out of the way Start at the bottom & work your way up
►Works great for troubleshooting
Or start at the top an work your way down►Not so good for security- top down is best here
►Real world virus example Multiple protocols Disguised traffic Peel the onion to analyze
Real World
STATISTICS- PROTOCOL HIERARCHY TCP CONVERSATIONS
► Compare to your list of valid IP’s► Compare to your list of valid protocols► Compare with what you have seen in
the past►Drill down into any anomalies or
unusual instances
Real World
FTP Traffic High…
FTP Traffic High… to .10
FTP Traffic High… between .10 & .25
.25 is the culprit but what is it doing
Conclusion for this scan:
►Normally, a port scan shows up very loudly and easily
► In this case, .2 was controlling .10 via a remote trojan on port 25 (mail) . .10 was passing the instructions on to .25 via port 21 (ftp)
►All of this trojan traffic passes firewall rules and doen’t get a second glance.
►You would have busted .25 for the port scan, but left .10, .25 infected and the master of it all, .2, is still at large.
Conclusion for this scan:
References
► Cool sample traffic, including security issues: http://wiki.wireshark.org/SampleCaptures
#head-6c6fb4051dfbe9b992057ea1533eb8dc85c9a13a
► Filters http://wiki.wireshark.org/DisplayFilters
► 10 signs that you aren't cut out for IT 1: You lack patience 2: You have no desire to continue your education 3: You refuse to work outside 9-to-5 4: You don’t like people 5: You give up quickly 6: You’re easily frustrated 7: You can’t multitask 8: You have dreams of climbing the corporate ladder 9: You hate technology 10: You turn off your phone at night
► By Jack Wallen; February 24, 2012
10 Signs you aren’t cut out for IT
Evalution
►I value your comments. Please fill in your evaluation form found at the end of your packet.
Scott Greene: Other topics available
► Computer Forensics► Computer Forensics for Defense Attorneys► Personal Privacy in the Information Age► High Technology: Just where is technology going?► Bypassing Security: How They Steal Company Data ► Fundamentals of Digital Forensics► Technology Forensics: Theory & Potential... is it Science or Art?► Technology Forensics: Case Examples► Technology Forensics: Intellectual property and identity theft► Technology Forensics: Hardware and Software tools / Show and Tell► Portable Devices Issues and Answers: A discussion about cell phones and the
stories they can tell.► Anti-Digital Forensics. Or is it Digital Anti-Forensics?► Data Security and Confidentiality Issues► E-mail: The digital Smoking Gun
Contact InformationScott Greene, SCFE
Evidence Solutions, Inc
866-795-7166Scott@EvidenceSolutions.com