Developing Analytic Technique and Defeating Cognitive Bias in Security
-
Upload
chrissanders88 -
Category
Technology
-
view
3.914 -
download
1
description
Transcript of Developing Analytic Technique and Defeating Cognitive Bias in Security
![Page 1: Developing Analytic Technique and Defeating Cognitive Bias in Security](https://reader036.fdocuments.in/reader036/viewer/2022070303/5498f6d5b47959774d8b4697/html5/thumbnails/1.jpg)
Defeating Cognitive Bias and
Developing Analytic Technique
Chris SandersBSides Augusta 2014
![Page 2: Developing Analytic Technique and Defeating Cognitive Bias in Security](https://reader036.fdocuments.in/reader036/viewer/2022070303/5498f6d5b47959774d8b4697/html5/thumbnails/2.jpg)
Chris Sanders
• Christian & Husband• Kentuckian and South
Carolinian• MS, GSE, CISSP, et al.• Non-Profit Director• BBQ Pit Master
![Page 3: Developing Analytic Technique and Defeating Cognitive Bias in Security](https://reader036.fdocuments.in/reader036/viewer/2022070303/5498f6d5b47959774d8b4697/html5/thumbnails/3.jpg)
Chris Sanders
![Page 4: Developing Analytic Technique and Defeating Cognitive Bias in Security](https://reader036.fdocuments.in/reader036/viewer/2022070303/5498f6d5b47959774d8b4697/html5/thumbnails/4.jpg)
Chris Sanders
“[Practical Packet Analysis] gives you everything you need, step by step, to become proficient in packet analysis. I could not find a better book.”
– Amazon Reviewer
![Page 5: Developing Analytic Technique and Defeating Cognitive Bias in Security](https://reader036.fdocuments.in/reader036/viewer/2022070303/5498f6d5b47959774d8b4697/html5/thumbnails/5.jpg)
Outline
Objectives: What is Analysis? What is Bias? Recognizing Bias Defeating Bias Analysis Methods
“How to make better technical decisions in any kind of security analysis.“
![Page 6: Developing Analytic Technique and Defeating Cognitive Bias in Security](https://reader036.fdocuments.in/reader036/viewer/2022070303/5498f6d5b47959774d8b4697/html5/thumbnails/6.jpg)
**Disclaimer**
I’m going to talk about matters of the brain, not sure the normal tech stuff.
My research for this presentation involved consultation with psychologists.
I, however, am not one.
![Page 7: Developing Analytic Technique and Defeating Cognitive Bias in Security](https://reader036.fdocuments.in/reader036/viewer/2022070303/5498f6d5b47959774d8b4697/html5/thumbnails/7.jpg)
Bias – A very personal story
![Page 8: Developing Analytic Technique and Defeating Cognitive Bias in Security](https://reader036.fdocuments.in/reader036/viewer/2022070303/5498f6d5b47959774d8b4697/html5/thumbnails/8.jpg)
2 AM
![Page 9: Developing Analytic Technique and Defeating Cognitive Bias in Security](https://reader036.fdocuments.in/reader036/viewer/2022070303/5498f6d5b47959774d8b4697/html5/thumbnails/9.jpg)
The Pain Begins
*Dramatization
![Page 10: Developing Analytic Technique and Defeating Cognitive Bias in Security](https://reader036.fdocuments.in/reader036/viewer/2022070303/5498f6d5b47959774d8b4697/html5/thumbnails/10.jpg)
![Page 11: Developing Analytic Technique and Defeating Cognitive Bias in Security](https://reader036.fdocuments.in/reader036/viewer/2022070303/5498f6d5b47959774d8b4697/html5/thumbnails/11.jpg)
Ultrasounds == Magic?
![Page 12: Developing Analytic Technique and Defeating Cognitive Bias in Security](https://reader036.fdocuments.in/reader036/viewer/2022070303/5498f6d5b47959774d8b4697/html5/thumbnails/12.jpg)
At this point…
So, I went to see a surgeon…
![Page 13: Developing Analytic Technique and Defeating Cognitive Bias in Security](https://reader036.fdocuments.in/reader036/viewer/2022070303/5498f6d5b47959774d8b4697/html5/thumbnails/13.jpg)
“Let’s Cut it Out!” - Surgeon
![Page 14: Developing Analytic Technique and Defeating Cognitive Bias in Security](https://reader036.fdocuments.in/reader036/viewer/2022070303/5498f6d5b47959774d8b4697/html5/thumbnails/14.jpg)
Missing Parts
![Page 15: Developing Analytic Technique and Defeating Cognitive Bias in Security](https://reader036.fdocuments.in/reader036/viewer/2022070303/5498f6d5b47959774d8b4697/html5/thumbnails/15.jpg)
Thus…
“Would it be accurate to say that I’m a medical miracle?” - Me
“Absolutely.” – Surgeon
![Page 16: Developing Analytic Technique and Defeating Cognitive Bias in Security](https://reader036.fdocuments.in/reader036/viewer/2022070303/5498f6d5b47959774d8b4697/html5/thumbnails/16.jpg)
Cause and Effect
• Cause: Bias…lots of it!– Confirmation Bias– Outcome Bias– Congruence Bias
• Effect: Unnecessary Surgery– 1 Week Recovery– Financial Loss– Pessimism Bias
![Page 17: Developing Analytic Technique and Defeating Cognitive Bias in Security](https://reader036.fdocuments.in/reader036/viewer/2022070303/5498f6d5b47959774d8b4697/html5/thumbnails/17.jpg)
Analysis
![Page 18: Developing Analytic Technique and Defeating Cognitive Bias in Security](https://reader036.fdocuments.in/reader036/viewer/2022070303/5498f6d5b47959774d8b4697/html5/thumbnails/18.jpg)
Analysis is Everywhere
• Making judgments based upon data• Security Analysis Happens for:– Malware Analysts– Intelligence Analysts– Incident Response Analysts– Forensic Analysts– Programming Logic Analysts
• My main focus is network intrusion analysis, so this talk will be framed through that.
![Page 19: Developing Analytic Technique and Defeating Cognitive Bias in Security](https://reader036.fdocuments.in/reader036/viewer/2022070303/5498f6d5b47959774d8b4697/html5/thumbnails/19.jpg)
Network Security Monitoring
• The collection, detection, and analysis of network security data.
• The goal of NSM is escalation, or to declare that an incident has occurred to that incident response can occur.
![Page 20: Developing Analytic Technique and Defeating Cognitive Bias in Security](https://reader036.fdocuments.in/reader036/viewer/2022070303/5498f6d5b47959774d8b4697/html5/thumbnails/20.jpg)
Evolution of NSM Emphasis
![Page 21: Developing Analytic Technique and Defeating Cognitive Bias in Security](https://reader036.fdocuments.in/reader036/viewer/2022070303/5498f6d5b47959774d8b4697/html5/thumbnails/21.jpg)
The Need for Analytic Technique
• Kansas State University Anthropological Study on SOCs - Key Finding:– “SOC analysts often perform sophisticated
investigations where the process required to connect the dots is unclear even to analysts.”
• Analysis == “Tacit Knowledge”
![Page 22: Developing Analytic Technique and Defeating Cognitive Bias in Security](https://reader036.fdocuments.in/reader036/viewer/2022070303/5498f6d5b47959774d8b4697/html5/thumbnails/22.jpg)
Analysis: Thinking About Thinking
• We need to critically examine how we think about information security analysis.
• We aren’t alone!– Scientific– Medical– Legal
![Page 23: Developing Analytic Technique and Defeating Cognitive Bias in Security](https://reader036.fdocuments.in/reader036/viewer/2022070303/5498f6d5b47959774d8b4697/html5/thumbnails/23.jpg)
Perception vs. Reality
• Perception: – “A way of regarding, understanding, or interpreting
something.”
• Reality: – “The state of things as they actually exist.”
Let’s take a test…
![Page 24: Developing Analytic Technique and Defeating Cognitive Bias in Security](https://reader036.fdocuments.in/reader036/viewer/2022070303/5498f6d5b47959774d8b4697/html5/thumbnails/24.jpg)
RED
![Page 25: Developing Analytic Technique and Defeating Cognitive Bias in Security](https://reader036.fdocuments.in/reader036/viewer/2022070303/5498f6d5b47959774d8b4697/html5/thumbnails/25.jpg)
GREEN
![Page 26: Developing Analytic Technique and Defeating Cognitive Bias in Security](https://reader036.fdocuments.in/reader036/viewer/2022070303/5498f6d5b47959774d8b4697/html5/thumbnails/26.jpg)
BLUE
![Page 27: Developing Analytic Technique and Defeating Cognitive Bias in Security](https://reader036.fdocuments.in/reader036/viewer/2022070303/5498f6d5b47959774d8b4697/html5/thumbnails/27.jpg)
BLACK
![Page 28: Developing Analytic Technique and Defeating Cognitive Bias in Security](https://reader036.fdocuments.in/reader036/viewer/2022070303/5498f6d5b47959774d8b4697/html5/thumbnails/28.jpg)
YELLOW
![Page 29: Developing Analytic Technique and Defeating Cognitive Bias in Security](https://reader036.fdocuments.in/reader036/viewer/2022070303/5498f6d5b47959774d8b4697/html5/thumbnails/29.jpg)
Test Results
• Variation of Stroop Test (John Stroop, 1935)• Measures Cognition – The Process of Perception
• Identifies Gap Between Perception & Reality• Used to Measure– Selective Attention– Cognitive Flexibility– Processing Speed
![Page 30: Developing Analytic Technique and Defeating Cognitive Bias in Security](https://reader036.fdocuments.in/reader036/viewer/2022070303/5498f6d5b47959774d8b4697/html5/thumbnails/30.jpg)
What is Bias?
“Prejudice in favor of or against one thing, person, or group compared with another, usually in a way considered to be unfair.”
•Perception != Reality•Perception is Everything, but Fallible•We tend to perceive what we expect/are conditioned to perceive
![Page 31: Developing Analytic Technique and Defeating Cognitive Bias in Security](https://reader036.fdocuments.in/reader036/viewer/2022070303/5498f6d5b47959774d8b4697/html5/thumbnails/31.jpg)
I’m Going to Show You an Image
![Page 32: Developing Analytic Technique and Defeating Cognitive Bias in Security](https://reader036.fdocuments.in/reader036/viewer/2022070303/5498f6d5b47959774d8b4697/html5/thumbnails/32.jpg)
![Page 33: Developing Analytic Technique and Defeating Cognitive Bias in Security](https://reader036.fdocuments.in/reader036/viewer/2022070303/5498f6d5b47959774d8b4697/html5/thumbnails/33.jpg)
I’m Going to Show You a Picture of a White Vase.
![Page 34: Developing Analytic Technique and Defeating Cognitive Bias in Security](https://reader036.fdocuments.in/reader036/viewer/2022070303/5498f6d5b47959774d8b4697/html5/thumbnails/34.jpg)
![Page 35: Developing Analytic Technique and Defeating Cognitive Bias in Security](https://reader036.fdocuments.in/reader036/viewer/2022070303/5498f6d5b47959774d8b4697/html5/thumbnails/35.jpg)
First Image Results
• Prompted for Face– 88% See Face– 12% See Sax Player
• No Prompt– 57% See Face– 43% See Sax Player
![Page 36: Developing Analytic Technique and Defeating Cognitive Bias in Security](https://reader036.fdocuments.in/reader036/viewer/2022070303/5498f6d5b47959774d8b4697/html5/thumbnails/36.jpg)
Second Image Results
• Prompted for Vase– 94% See White Vase– 6% See Two People
• No Prompt– 62% See White Vase– 38% See Two People
![Page 37: Developing Analytic Technique and Defeating Cognitive Bias in Security](https://reader036.fdocuments.in/reader036/viewer/2022070303/5498f6d5b47959774d8b4697/html5/thumbnails/37.jpg)
Bias Examples
![Page 38: Developing Analytic Technique and Defeating Cognitive Bias in Security](https://reader036.fdocuments.in/reader036/viewer/2022070303/5498f6d5b47959774d8b4697/html5/thumbnails/38.jpg)
Let’s Hit Closer to Home…
![Page 39: Developing Analytic Technique and Defeating Cognitive Bias in Security](https://reader036.fdocuments.in/reader036/viewer/2022070303/5498f6d5b47959774d8b4697/html5/thumbnails/39.jpg)
A Recent Example
![Page 40: Developing Analytic Technique and Defeating Cognitive Bias in Security](https://reader036.fdocuments.in/reader036/viewer/2022070303/5498f6d5b47959774d8b4697/html5/thumbnails/40.jpg)
Anchoring
• Defined: Heavily relying on a single piece of information.
• Examples:– Src/Dst Country -> OMG China!– IDS Alert Name -> It say this is X, so it must be X.– Timing -> It’s every 5 minutes!
![Page 41: Developing Analytic Technique and Defeating Cognitive Bias in Security](https://reader036.fdocuments.in/reader036/viewer/2022070303/5498f6d5b47959774d8b4697/html5/thumbnails/41.jpg)
Clustering Illusion
• Defined: Overestimating the value of perceived patterns in random data.
• Examples:– The great “beaconing”
fallacy– Unguided Visualizations
![Page 42: Developing Analytic Technique and Defeating Cognitive Bias in Security](https://reader036.fdocuments.in/reader036/viewer/2022070303/5498f6d5b47959774d8b4697/html5/thumbnails/42.jpg)
Availability Cascade
• Defined: Strong belief in something due to its repetition in public discourse
• Example:– “Chinese Traffic is Bad.”– “That rule generates a lot of false positives.”
![Page 43: Developing Analytic Technique and Defeating Cognitive Bias in Security](https://reader036.fdocuments.in/reader036/viewer/2022070303/5498f6d5b47959774d8b4697/html5/thumbnails/43.jpg)
Belief Bias
• Defined: Occurs when a decision is based on the believability of the conclusion.
• Examples:– “We wouldn’t be a target for a nation-state actor.”– “This is probably a false positive because it’s
unlikely someone would attack our VoIP system.”
![Page 44: Developing Analytic Technique and Defeating Cognitive Bias in Security](https://reader036.fdocuments.in/reader036/viewer/2022070303/5498f6d5b47959774d8b4697/html5/thumbnails/44.jpg)
Confirmation Bias
• Defined: Interpreting data during analysis with a focus on confirming one’s preconception.
• Ego is a big factor here
• Examples:– “I think this is nothing.”– “I think there is something going on here.”
![Page 45: Developing Analytic Technique and Defeating Cognitive Bias in Security](https://reader036.fdocuments.in/reader036/viewer/2022070303/5498f6d5b47959774d8b4697/html5/thumbnails/45.jpg)
Impact Bias
• Defined: Tendency to overestimate the significance of something based on the potential impact.
• Signature/Alert Naming + Lack of Experience Contribute to this.
• Example:– “The alert says this is a known APT1 back door, so I
need to spend all day looking at this.”
![Page 46: Developing Analytic Technique and Defeating Cognitive Bias in Security](https://reader036.fdocuments.in/reader036/viewer/2022070303/5498f6d5b47959774d8b4697/html5/thumbnails/46.jpg)
Irrational Escalation
• Defined: Justifying increased time investment based on existing time investment when it may not make sense.
• Sunk Cost Fallacy• Example:– “What do you mean this is nothing? I’ve spent all
day looking at this. I’ll spend all day tomorrow digging into it; I’m sure I’ll find something else there.”
![Page 47: Developing Analytic Technique and Defeating Cognitive Bias in Security](https://reader036.fdocuments.in/reader036/viewer/2022070303/5498f6d5b47959774d8b4697/html5/thumbnails/47.jpg)
Framing Effect
• Defined: Interpreting information differently based on how or from whom it was presented.
• Important in interaction with other analysts• Example:– Old Vet: “Steve doesn’t know what he is doing, so if
he is telling me this it probably doesn’t mean much.”– New Guy: “None of the more experienced guys said
anything about this, so it must not matter.”
![Page 48: Developing Analytic Technique and Defeating Cognitive Bias in Security](https://reader036.fdocuments.in/reader036/viewer/2022070303/5498f6d5b47959774d8b4697/html5/thumbnails/48.jpg)
Overconfidence Effect
• Defined: Excessive confidence in ones own decisions, especially in light of contrasting data.
• Example: • 99% Paradox – “I’m 99% sure this is right.”• One psych study suggest this statement is
wrong ~40% of the time.
![Page 49: Developing Analytic Technique and Defeating Cognitive Bias in Security](https://reader036.fdocuments.in/reader036/viewer/2022070303/5498f6d5b47959774d8b4697/html5/thumbnails/49.jpg)
Pro-Innovation Bias
• Defined: Excessive optimism and biased decisions based on an invention of one’s own making being involved in the analysis.
• Invention == System / Code / Concept• Example:– “My tool can do that.”– “I wrote that signature so I know it’s accurate.”– “This fits perfectly in my model!”
![Page 50: Developing Analytic Technique and Defeating Cognitive Bias in Security](https://reader036.fdocuments.in/reader036/viewer/2022070303/5498f6d5b47959774d8b4697/html5/thumbnails/50.jpg)
There are over 100 types of bias. How can we overcome them?
![Page 51: Developing Analytic Technique and Defeating Cognitive Bias in Security](https://reader036.fdocuments.in/reader036/viewer/2022070303/5498f6d5b47959774d8b4697/html5/thumbnails/51.jpg)
Overcoming Bias
![Page 52: Developing Analytic Technique and Defeating Cognitive Bias in Security](https://reader036.fdocuments.in/reader036/viewer/2022070303/5498f6d5b47959774d8b4697/html5/thumbnails/52.jpg)
What Can We Do?
• Preconception and Bias Cannot Be Fully Avoided
• Therefore: – Develop Repeatable Analytic Technique– Recognize Key Assumptions– Allow them to be Challenged
![Page 53: Developing Analytic Technique and Defeating Cognitive Bias in Security](https://reader036.fdocuments.in/reader036/viewer/2022070303/5498f6d5b47959774d8b4697/html5/thumbnails/53.jpg)
Analytic Techniques
Common Techniques:– Relational
Investigation– Differential
Diagnosis
![Page 54: Developing Analytic Technique and Defeating Cognitive Bias in Security](https://reader036.fdocuments.in/reader036/viewer/2022070303/5498f6d5b47959774d8b4697/html5/thumbnails/54.jpg)
Relational Investigation
• “Link Analysis”• Commonly Used in Criminal Investigations• Focuses on Entities, Relationships,
Interactions, and Degrees of Separation
![Page 55: Developing Analytic Technique and Defeating Cognitive Bias in Security](https://reader036.fdocuments.in/reader036/viewer/2022070303/5498f6d5b47959774d8b4697/html5/thumbnails/55.jpg)
Relational Investigation
![Page 56: Developing Analytic Technique and Defeating Cognitive Bias in Security](https://reader036.fdocuments.in/reader036/viewer/2022070303/5498f6d5b47959774d8b4697/html5/thumbnails/56.jpg)
Setting the Stage – Primary Relationships
![Page 57: Developing Analytic Technique and Defeating Cognitive Bias in Security](https://reader036.fdocuments.in/reader036/viewer/2022070303/5498f6d5b47959774d8b4697/html5/thumbnails/57.jpg)
Partial Story – Secondary Relationships
![Page 58: Developing Analytic Technique and Defeating Cognitive Bias in Security](https://reader036.fdocuments.in/reader036/viewer/2022070303/5498f6d5b47959774d8b4697/html5/thumbnails/58.jpg)
Full Attack Diagram – Tertiary Relationships
![Page 59: Developing Analytic Technique and Defeating Cognitive Bias in Security](https://reader036.fdocuments.in/reader036/viewer/2022070303/5498f6d5b47959774d8b4697/html5/thumbnails/59.jpg)
Differential Diagnosis
• Commonly Used in Medical Diagnosis
• Relies on Lists of Possibilities, and Systematically Eliminating Possibilities
![Page 60: Developing Analytic Technique and Defeating Cognitive Bias in Security](https://reader036.fdocuments.in/reader036/viewer/2022070303/5498f6d5b47959774d8b4697/html5/thumbnails/60.jpg)
Differential Diagnosis
![Page 61: Developing Analytic Technique and Defeating Cognitive Bias in Security](https://reader036.fdocuments.in/reader036/viewer/2022070303/5498f6d5b47959774d8b4697/html5/thumbnails/61.jpg)
Incident M&M
• Dr. Ernest Codman at Mass. General Hospital• Post-Patient Meetings to Discuss What
Occurred and How to Better It• Incident M&M
1. Handler/Analyst Presents Case2. Followed by Alternative Analysis
![Page 62: Developing Analytic Technique and Defeating Cognitive Bias in Security](https://reader036.fdocuments.in/reader036/viewer/2022070303/5498f6d5b47959774d8b4697/html5/thumbnails/62.jpg)
Alternative Analysis
• Developed by Richards Heuer Jr. (FBI)• Series of Peer Analysis Methods• Designed to Help Overcome Bias and Improve
Quality of Analysis
![Page 63: Developing Analytic Technique and Defeating Cognitive Bias in Security](https://reader036.fdocuments.in/reader036/viewer/2022070303/5498f6d5b47959774d8b4697/html5/thumbnails/63.jpg)
Group A / Group B
• Group A – Presenting Analyst/Team• Group B – Secondary Analyst/Team
• Two Independent Analysis Efforts• Note are Compared During the Presentation• Identify Differing Conclusions from Same Data
![Page 64: Developing Analytic Technique and Defeating Cognitive Bias in Security](https://reader036.fdocuments.in/reader036/viewer/2022070303/5498f6d5b47959774d8b4697/html5/thumbnails/64.jpg)
Red Cell Analysis
• Peer Focus on Attacker’s Viewpoint• Questioning in Relation to Attackers Perceived
Goals• Requires Some Offensive Experience• Best Executed by Red Team if Available
![Page 65: Developing Analytic Technique and Defeating Cognitive Bias in Security](https://reader036.fdocuments.in/reader036/viewer/2022070303/5498f6d5b47959774d8b4697/html5/thumbnails/65.jpg)
What If Analysis
• Focus on Cause/Effect of Actions That May Not Have Actually Occurred– What is the attacker had done X? How would you
have changed your approach?– What if you didn’t stumble across X in Y data?
• Enhances Later Investigations
![Page 66: Developing Analytic Technique and Defeating Cognitive Bias in Security](https://reader036.fdocuments.in/reader036/viewer/2022070303/5498f6d5b47959774d8b4697/html5/thumbnails/66.jpg)
Key Assumptions Check
• Presenter Identifies Assumptions During Analysis
• Peers Challenge Assumptions• Pairs Well with “What If” Analysis– “What if it were possible for that malware to
escape that virtual machine?”– “Would you come to the same conclusion if you
knew this was APT3 instead of APT1?”
![Page 67: Developing Analytic Technique and Defeating Cognitive Bias in Security](https://reader036.fdocuments.in/reader036/viewer/2022070303/5498f6d5b47959774d8b4697/html5/thumbnails/67.jpg)
Incident M&M Best Practices
• Limit Frequency• Set Expectations• Require a Strong Mediator• Keep it at the Team Level – No Sr. Managers• Encourage Servant Leadership• Discourage Personal Attacks• Write it Down!
![Page 68: Developing Analytic Technique and Defeating Cognitive Bias in Security](https://reader036.fdocuments.in/reader036/viewer/2022070303/5498f6d5b47959774d8b4697/html5/thumbnails/68.jpg)
Conclusion
• The Era of Analysis is Upon Us• Bias is Inevitable – Learn to Recognize It• Overcome Analysis Hurdles With:– Analytic Technique– Alternative Analysis
![Page 69: Developing Analytic Technique and Defeating Cognitive Bias in Security](https://reader036.fdocuments.in/reader036/viewer/2022070303/5498f6d5b47959774d8b4697/html5/thumbnails/69.jpg)
Thank You!
E-Mail: [email protected]: @chrissanders88
Blog: http://www.chrissanders.orgBook Blog: http://www.appliednsm.com
Testimony: http://www.chrissanders.org/mytestimony