I MPLEMENTING AN I NCIDENT R ESPONSE P ROCESS W ITH T EETH Del
A. Russ, GCFA, CISSP August 18, 2011
Slide 2
Welcome Speaker: Del A. Russ, GCFA, CISSP Title: Implementing
an Electronic Incident Response Process with Teeth Abstract: Most
mature Infosec organizations have an Electronic Incident Response
Process of some sort, and many are well-founded upon best practices
that have existed for years. Even with decent processes in place,
however, real- time IR engagements can still be painful experiences
for those involved, plagued with miscommunications, disorder,
preventable mistakes, and/or poor decision making that ultimately
drives the duration and cost of responding through the roof. How
your IR process is actually implemented makes a big difference!
This presentation will drill into the root causes for why most IR
processes lack the teeth necessary to promote fast, accurate,
balanced, and authoritative responses, and demonstrate tactics that
can be directly applied to help guarantee improved handling of
incidents from the time of initial response to full follow-through!
The material is suitable for Infosec professionals at all levels,
technical or managerial. Bio: Del A. Russ is currently employed as
a Senior IT Security Analyst at Xerox Corporation, where he has
been involved in numerous Information Security programs since 2001.
Mr. Russ founded Xerox Information Managements Computer Forensics
Program in 2005, and the Xerox Electronic Incident Response Program
(EIRP) which he managed from 2007-2010. He has participated
directly in the handling of hundreds of electronic security
incidents, at all levels of complexity and severity. Dels other
expertise is in Threat Management programs and solutions, including
Network Based Vulnerability Scanning (NBVS), Data Leak Protection
(DLP), Intrusion Detection Systems (IDS), Log Monitoring Systems
(LMS), and other related areas. Prior to entering the Information
Security field, he spent ten years in Software Engineering and IT
Consulting, primarily with Computer Science Corporation (CSC). Mr.
Russ holds a Bachelor of Science Degree in Computer Science from
the State University of New York at Buffalo, and a Minor study in
Psychology. He holds GCFA and CISSP professional
certifications.
Slide 3
Topics Incident Response Problems 10 Big New Teeth for Your IR
Process Taking Control - What to Implement New Operational
Definition of an Incident Observed Benefits & Caveats Open
Discussion / Q&A
Slide 4
Incident Response Problems
Slide 5
Are we having fun yet? You may have been here before Even
following the basics of a good IR process: High stress,
antagonistic participation Slow - difficult to make confident
decisions Lack of trust, biased decision making Exceeding/ignoring
authority Costly technical mistakes Disorganized data collection
Frequent over-engagement vs. risk involved Frequent
under-engagement vs. risk involved
Slide 6
Post-Mortem Root Causes to Target: Ambiguity / Inconsistency
Confusion Disorderly Procedures Lack of Control Mistrust Incomplete
Fact Collection Poor Decisions Disjointed sub-processes: Physical
Security vs. Infosec Real-Time IR vs. Forensics as a back-office
function Unclear Roles Authority & Bias Issues High-level
processes lack teeth sufficient for good execution.
Slide 7
IR Teeth What to Implement
Slide 8
(1) Measure Evidence Strength Q: What are you really looking
at?! Rating Scheme follow the courts Computer Records and the
Federal Rules of Evidence, Orin S. Kerr; USA Bulletin (March 2001);
U.S. Department of Justice:
http://www.usdoj.gov.criminal/cybercrime/usamarch2001_4.htmhttp://www.usdoj.gov.criminal/cybercrime/usamarch2001_4.htm
Example: Physical (strong) Electronically Generated (strong)
Electronically Stored (contains human content) Verbal / Hearsay /
Opinion (weak) Stronger Less Human Bias in Content
Slide 9
(1) Measure Evidence Strength - Examples
Slide 10
(2) Define Incident Types / Subtypes No industry standard
convention Opinions vary Vendors use of basic terminology varies
Most incidents have more than one type Distinguish the means for
exploitation from the impacts Consider how threats are practically
managed One type leads to another, interrelationships
Slide 11
(2) Incident Types / Subtypes - Example DoSIntentionally
stopping or hindering business progress via electronic means. Brute
Force Overloading; Sabotage; Indirect Impact via Other
Deliberate/Malicious Action; Other: Unauthorized Use Any applied
use of IT Assets (systems or data) for purposes that avert the
interests of the org and its affiliates, which constitute
violations of one or more Xerox policies, civil or criminal laws,
or regulatory statutes. Inappr. Use of IT Assets; Employee
Misconduct; Harassment; Defacement; Slander; Espionage; Fraud;
Piracy; Copyr/Patent Infringement; Pornography, Child Porn; Other:
Unauthorized Access The act of deliberately averting authentication
or authorization related info security controls to gain access to
assets not entrusted to you. Hacking; Cracking; Exploit SW Vuln for
Priv Escalation; SQL Injection; Cross-Site Scripting; Other:
Unauthorized Disclosure Trusted persons making trusted data
available to others (untrusted parties) via electronic delivery or
providing direct authentication and/or authorization to data stores
or other media. Public Posting of Private Data; Inappropriate
Sharing of Confidential Info; Accidental Disclosure via IT
means/email; Lost PC; Other: Social Engineering Gathering IT-assets
information using primarily non-electronic means, that involve one
or more forms of deceit/trickery; human/process interfaces
Phishing; Dumpster Diving; Shoulder Browsing; False Impersonation;
Other: Infosec Controls Bypassing engagement of required info
security Processes subsequently resulting in absent or insufficient
info security controls undue risk your business is not authorized
to accept on behalf of the org and affiliates. Unapproved Internet
Gateway; Bypassing Change Control; Data Center Priv. Acct Abuse;
Other: Malicious Code Software (any sort), with covert behavior
capability, violates authentication or authorization controls to
gain access to protected/senstive data Rootkits; Spyware; Bots;
Backdoors; Trojans; Keylogger; Droppers; Downloaders; Other:
Network Mapping Gathering information that reveals understanding of
operation or weaknesses, without an approved business need to
know/possess. Port Scans; Ping Plots; Tracerouting; Other:
HoaxIntentional fabrication of the truth, content delivered
electronically.Chainmail; False Blog Article; Other: Worms /
Viruses Auto-Replicating or Self-Propagating software potentially
used as a transport for more serious Malicious Code Incidents,
targeting systems without regard for specific business value
(random, linear, automated, etc.) Mass-Mailers; File Infectors;
Network-Scanning BO Worm; Other: Undesired / Nussance Content
Unsolicited content with no org/business value delivered via email
or other means Spam; Adware; Pop-Ups; Prank, Other:
Slide 12
(2) Incident Types Interrelationship
Slide 13
(3) Architecture-Based Data Collection Architecture-Timeline
View Visual depiction of incident scene using logical architecture
diagram Dataflow diagram plus Evidence articles Numerous
value-added uses Capture IT Assets, Org/People, and Geographic data
all at one time Every Incident can be defined as a set of ordered
Events
Slide 14
(3) Architecture-Timeline View - Example
Slide 15
(3) Architecture-Timeline View (cont.) Electronic Incident
Sequence of one or more Events involving IT Assets and/or Data and
potential violations of policy/law Event attributes Source (Person,
IT Asset, Location) Destination (Person, IT Asset, Location)
Environment traversed (Persons, IT Assets, Locations) Action Taken
Relative order of Event within overall incident Incident Type /
Subtype Implied Verified vs. Potential Attempted vs. Successful
Source, Environment, and Destination attributes: Inherent Data Risk
(PII, SPII, etc.) Means (i.e. tactic, exploit, etc.) Vulnerability
Targeted (if any) Evidence Articles Electronically Generated,
Stored, or Verbal/Hearsay
Slide 16
(4) Risk Assessment Checklist Have an objective table /
checklist pre-defined: First consider impacts to people, life,
health, well-being, social standing, etc. Criminal Implications
Laws & Regulations that apply PR Implications, long-term
fallout Business / Monetary Estimate potential cost impacts, if
possible
Slide 17
(5) Quantify Evidence Collection - Example Natural Cost/Skill
Divide-Lines (i.e. Tiers) Example: Tier-A: Simple High-Level,
research, lookups, etc. Discussion notes with witnesses Tier-B:
Sample collection from systems involved Logs, file samples,
registry, etc. Interview statements from tech support staff and
operations Tier-C: Network Forensics Logical drive imaging on
running systems (no outage) Tier-D: Full Evidence Collection &
Preservation Seize laptops, shutdown systems, physical hard drive
images, etc. Formally interview perpetrators for signed admission,
etc.
Slide 18
(6) Quantify People Involvement - Example
Slide 19
(7) Quantify Response Time Participants in IR usually have
other things to do. Clarify aggressiveness of IR actions required.
Risk Assessment factors should justify speed required Example:
24/7: work non-stop until some steps are completed i.e. at least
until Contained and volatile evidence preserved ASAP Business
Hours: Highest priority work during normal hours Normal Operations:
Balance IR tasks with other day-to-day priorities
Slide 20
(8) Structure Your Action Plan SANS Institutes IR Phases are a
good foundation: Computer Security Incident Handling, V.2.3.1,
Stephen Northcutt
http://security.gmu.edu/ComputerIncidentHandling.pdf Include
Forensic IR Steps (FIR)! Most of you may be familiar with this
model already: Identification Containment Include Forensic Incident
Response (FIR) tasks Eradication Recovery Reporting Include
back-office lab forensics analysis & reports Follow-Up
Slide 21
(9) Define Reporting Types & Points Management will push
for status Dont mislead, over-state, or speculate Managerial
instinct is to take control, initiate actions Set realistic
expectations early on Scale level of detail as necessary for target
audience Examples: Brief Interim Status Summary Current State
Structured email Update after every IR team engagement / cycle
Executive Summary Report Final, for management Deliver after most
IR phases are complete, but before Follow-Ups Executive Summary w/
Full Forensic Report Final, for technical staff, law enforcement,
attorneys, etc. Issue as the last step in Reporting Phase
Slide 22
(10) Formal / Rigid IR Facilitation Dedicate a person to the
Facilitator role Runs the IR machine Explains the process quickly
to field personnel and managers up front Totally new experience for
most victims Do not understand risks/impacts of poor IR actions
Runs conference calls using checklists, data collection forms, and
tight procedures Maintains control over participant actions Leads
& earns the trust of those involved
Slide 23
Summary 10 New IR Teeth 1.Measurement of Evidence Strength
2.Clear Definition of Incident Types / Subtypes 3.Collection of
Data via Architecture-Timeline View 4.Using a Risk Assessment
Checklist 5.Quantifying Extent of Evidence Collection 6.Quantifying
Level of Personnel Engagement 7.Quantifying Response Time for
Actions Taken 8.Having a structured Action Plan w/ FIR Built-In
9.Defining Report Types (and Set Expectations) 10.Formally
Facilitating IR Engagements
Slide 24
New Operational Definition of Incident
Slide 25
Stateful Nature of Incidents Observations Incidents are
stateful, and have a life-cycle Complex, multi-faceted,
progressional A good process meets up to this reality Defensible
point-in-time decision making Actions are based upon: (a) what you
already know, and also (b) what the current Incident Type leads to
(i.e.) Infosec Control violation Unauth Access
Slide 26
Incident Review Cycle (or Session) Facilitator should identify
current state attributes: Evidence articles, rated for strength
Chronological Listing of Events What current evidence points to
Also, what may have happened before, and after Incident Types
involved Verified vs. Potential Attemped vs. Successful Risk
Factors that Apply Current Evidence collection Tier And which one
is appropriate to ascend to next
Slide 27
Incident Review Cycle (cont.) (attributes) Current Personnel
Engagement Level And which level should be ascended to next Tasks
in the Action Plan With names, target completion dates, etc.
Response Times to be Taken by IR Team
Slide 28
Example Interim Incident Report Date of Last Review Cycle:
8/18/2011 Facilitator: Del Russ IR Team Formed: John Doe, Jane
Plain, Ricky Henderson, Art Vandelay Incident: Suspicious
Unexpected Outage on EX-Bogus Application Summary of Evidence and
Incident Types Identified: Strong electronic evidence exists which
Verifies that the outage on EX-Bogus was a Denial of Service (Dos)
Incident invoked via Sabotage by an employee. Weak evidence exists
suggesting that an Unauthorized Access event may have proceeded the
outage, via Use of Stolen Logon Credentials. Further Verification
is being pursued at this time. Strong evidence exists to suggest
that the employee Attempted an Unauthorized Disclosure of his own
login credentials to an untrusted outside party.
Slide 29
Example Interim Incident Report (cont) Risk Factors Identified:
EX-Bogus processes customer PII, national and state breach /
disclosure laws apply Federal Computer Fraud & Abuse law may
apply, felony criminal offense Business outage has resulted in 400
staff hours of impact, and $75,000 in lost revenue to-date Current
Operating State: Tier-A and B evidence collection and forensics
have applied thus far, no forced system outages yet Level-2
Personnel Engagement has been in effect, with Infosec and field
operations triaging the scene We have been working 24/7 to secure
evidence, contain, and recover from the initial outage Next Steps
(Action Plan): John Doe will continue RCA, and to see if evidence
exists to Verify the account Disclosure violation Level-3 team will
be engaged Senior management and legal counsel needed to determine
if disclosure should/will occur, or law enforcement to be engaged
Evidence Collection will plan for Tier-C network-based means,
awaiting business decision by Level- 3 team on whether to incur a
full outage for hard drive imaging on EX-Bogus NOTE: An Executive
Summary report will not be possible until the analysis is complete,
perhaps in another 2-3 weeks. Further interim email reports of this
sort will occur 2x per week until then. Next Review Cycle: Mon
8/22/11
Slide 30
Ascending Capability Further Can develop escalation criteria
ahead of time Formulate decision matrix (i.e.) Under what
conditions should Tier-D (Full) forensic acquisition occur? What
necessitates 24/7 responsiveness? When should Level-3 DA be
involved or not? Eliminates bias! Predictive incident modeling
possible
Slide 31
Benefits And Caveats
Slide 32
First, the Caveats It takes time to implement all of this
Learning curve for Facilitator and IR team Changing habits and
culture of IR individual contributor decision making ends The
process dictates direction, not individuals Forces everyone to
remain honest!
Slide 33
Primary Benefits Observed Handle ANY electronic incident
optimally Earns trust of tough IR participants Reduces rate of
costly mistakes Reduced friction, more order & control Promotes
better understanding by managers who are accountable Balances
response to the inherent risk Less over-engagement Less
under-engagement
Slide 34
Some Additional Benefits Incident Types useful as metrics
framework for many areas of Threat Management Architecture-Timeline
View useful in final reports to summarize for all audiences
Predictive potential, incident modeling Point-in-Time Defensible
Decision Making Works well in uncharted incident terrain, new
situations handled consistently Scalable Drives us closer to
industry standardization?