Retaining Collective Intelligence in Incident Response and ... · Effectiveness through...
Transcript of Retaining Collective Intelligence in Incident Response and ... · Effectiveness through...
Retaining Collective Intelligence
in Incident Response and Controls
Effectiveness through Gamification
Mark Jaster, Founder & CEO [email protected]
(610) 742-9366
www.418intelligence.com
RADICAL TRANSPARENCY
BELIEVABILITY WEIGHTED DECISION MAKING
BACKTESTED DECISIONS WITH OUTCOMES
• Incident Responders’ and inputs were
“Cyber Believability Weighted?”
• Detection & Mitigation Methods were
Back Tested against outcomes?
• Responders and stakeholders were
rewarded for the value of their inputs?
What if we tried this in IR?
Continuous Collective Intelligence Calibrated Up-to-Date Answers on Call
Can we turn IR into a game?
• IARPA developed and tested
• Gamifies, scores & retains collaborative
intelligence in a Bayes Net Model
• Merges human and machine intelligences
through the language of probabilities
FOURSight Technology
As covered by …
Get Points Analyze Incidents Bet Outcome Probabilities Gamified Rewards
How FOURSight works
Trends Predictive Analytics
Prototype User Experience & Design
8
FOURSight Game Board
FOURSight Game Board
• Timed Rounds
• Currency Updated Continuously
• Running EV Score (not shown)
FOURSight Game Board
• All primary navigation occurs here
• Assess risk factors
• Collaboratively analyze attack TTPs
• Estimate effectiveness of IR options
FOURSight Game Board
• Background
Information Window
FOURSight Game Board
• Probabilities
Assessments Window
FOURSight Game Board
• Top Ten
• Player’s 1 over, 1 under
FOURSight Game Board
• Achievements
(Badges)
FOURSight Game Board
• Player Submissions
• IOCs
• Playbooks
Let’s Begin!
17
Each Round has a new briefing FOURSight
Artifacts served up for context FOURSight
Sim artifacts revealed by Rounds FOURSight
Users capture IOCs FOURSight
• This is a test!
(Sorry no
partial credit!)
Find the Easter Eggs to unlock
bonus content!
Assess Macro Situation
Now that you’ve seen the scenario
and some initial artifacts…
• What is at risk?
• How severe?
• What do we know from priors?
• What forecasts can we make on
the outcomes?
FOURSight
Careful, things may change next round!
Show us what you know! FOURSight
• What is at risk?
• How severe?
• Who else knows
something?
(Hint – Check the
Info window)
• What will happen
next?
Take your long positions early
when the “price” is cheap.
Assess the Target FOURSight
• What is the Target’s Security
Baseline?
• What is their maturity?
Assess the Threat FOURSight
• STIX Threat Factors
• These can modulate Detection and
Mitigation Efficacies
(eg. NIDS should perform differently
in APTs than in SQL Injection attacks)
• And they can be inferred from
TTPs
Analyze Incident TTPs FOURSight
• ATT&CK Model
• All 11 Tactics
• Prototype has 20% of Techniques
Show you can spot a red herring,
and profit from it!
Which Techniques are present? FOURSight
Going long, and going short can be
just as profitable!
How confident are you, and when?
What is the community consensus?
FOURSight
The community says Automated
Collection was used to Collect Data.
What Detection methods are best? FOURSight
• Each Technique is mapped to
between 5 and 20 Methods
• Post Prototype, the Techniques
and Methods will be chosen
dynamically from Pick Lists
What Detection tools would you bet on? FOURSight
How to Detect Automated
Collection? That is the question!
Do threat factors change your bets? FOURSight
Well, maybe it depends… (This is a side-bet that
pays extra if APT is True, else costs nothing).
What do your peers think, and why? FOURSight
Deeper insights for better actions FOURSight
• Because this is likely an
APT type of attack…
• Centralized Logging
moves to the top
Detection method
• The insight is retained
for new cases –
Community Memory!
Which Mitigation methods are best? FOURSight
• Same mappings as Detection in
the prototype
How would you mitigate this TTP? FOURSight
The community knows something new FOURSight
• Share emerging best
practices
Use Cases FOURSight
• Skills assessment and development –
Individuals and Teamwork
• Community brain bank and leverage –
All that knowledge at your back!
• Tools investment decisions – Base
your reco’s on proven tool experts.
FOURSight Collective IR Analysis Platform
Gamified Information Market
Incident Analysis Playbooks & Countermeasures
The End Game