AI in Cybersecurity: Applications, Open Problems, and ... · Intelligent Automation for cyber...
Transcript of AI in Cybersecurity: Applications, Open Problems, and ... · Intelligent Automation for cyber...
![Page 1: AI in Cybersecurity: Applications, Open Problems, and ... · Intelligent Automation for cyber security •…ut AI becomes a target of attack –Traditional ML and Deep Neural Networks](https://reader035.fdocuments.in/reader035/viewer/2022070912/5fb41e1a9f97053cae6ff45e/html5/thumbnails/1.jpg)
AI in Cybersecurity: Applications, Open Problems, and Future
Directions
Alina Oprea
Associate Professor
Northeastern University
December 6 2018
![Page 2: AI in Cybersecurity: Applications, Open Problems, and ... · Intelligent Automation for cyber security •…ut AI becomes a target of attack –Traditional ML and Deep Neural Networks](https://reader035.fdocuments.in/reader035/viewer/2022070912/5fb41e1a9f97053cae6ff45e/html5/thumbnails/2.jpg)
AI is Everywhere
2
![Page 3: AI in Cybersecurity: Applications, Open Problems, and ... · Intelligent Automation for cyber security •…ut AI becomes a target of attack –Traditional ML and Deep Neural Networks](https://reader035.fdocuments.in/reader035/viewer/2022070912/5fb41e1a9f97053cae6ff45e/html5/thumbnails/3.jpg)
Connected Cars
• Sensors for data collection• Assist drivers in making decisions to increase safety
3
![Page 4: AI in Cybersecurity: Applications, Open Problems, and ... · Intelligent Automation for cyber security •…ut AI becomes a target of attack –Traditional ML and Deep Neural Networks](https://reader035.fdocuments.in/reader035/viewer/2022070912/5fb41e1a9f97053cae6ff45e/html5/thumbnails/4.jpg)
Personalized Medicine
• Treatment adjusted to individual patients• Predictive models using a variety of features • Better outcome and reduced cost
4
![Page 5: AI in Cybersecurity: Applications, Open Problems, and ... · Intelligent Automation for cyber security •…ut AI becomes a target of attack –Traditional ML and Deep Neural Networks](https://reader035.fdocuments.in/reader035/viewer/2022070912/5fb41e1a9f97053cae6ff45e/html5/thumbnails/5.jpg)
A Bit of History
1865 1969
> 100 years
5
![Page 6: AI in Cybersecurity: Applications, Open Problems, and ... · Intelligent Automation for cyber security •…ut AI becomes a target of attack –Traditional ML and Deep Neural Networks](https://reader035.fdocuments.in/reader035/viewer/2022070912/5fb41e1a9f97053cae6ff45e/html5/thumbnails/6.jpg)
A Bit of History
1940
Unimate Robot1961
Sony Dream 2001
> 50 years
6
![Page 7: AI in Cybersecurity: Applications, Open Problems, and ... · Intelligent Automation for cyber security •…ut AI becomes a target of attack –Traditional ML and Deep Neural Networks](https://reader035.fdocuments.in/reader035/viewer/2022070912/5fb41e1a9f97053cae6ff45e/html5/thumbnails/7.jpg)
Fast Forward in the Near Future
AI Transportation in Cities of the Future (10-20 years)7
![Page 8: AI in Cybersecurity: Applications, Open Problems, and ... · Intelligent Automation for cyber security •…ut AI becomes a target of attack –Traditional ML and Deep Neural Networks](https://reader035.fdocuments.in/reader035/viewer/2022070912/5fb41e1a9f97053cae6ff45e/html5/thumbnails/8.jpg)
Fast Forward in the Near Future
AI Robots in Medicine of the Future (10-20 years)8
![Page 9: AI in Cybersecurity: Applications, Open Problems, and ... · Intelligent Automation for cyber security •…ut AI becomes a target of attack –Traditional ML and Deep Neural Networks](https://reader035.fdocuments.in/reader035/viewer/2022070912/5fb41e1a9f97053cae6ff45e/html5/thumbnails/9.jpg)
Fast Forward in the Far Future
What will happen in 100 years?
9
![Page 10: AI in Cybersecurity: Applications, Open Problems, and ... · Intelligent Automation for cyber security •…ut AI becomes a target of attack –Traditional ML and Deep Neural Networks](https://reader035.fdocuments.in/reader035/viewer/2022070912/5fb41e1a9f97053cae6ff45e/html5/thumbnails/10.jpg)
Implications for Cyber Security
• AI has potential in security applications– Complement traditional defenses (crypto, multi-factor
authentication, trusted hardware)
– Design intelligent and adaptive defense algorithms
• …But AI becomes a target of attack– Deep Neural Networks are not resilient to adversarial
manipulations• [Szegedy et al. 13]: “Intriguing properties of neural networks”
– Many critical real-world applications are vulnerable
– New adversarially-resilient algorithms are needed!
AI
10
![Page 11: AI in Cybersecurity: Applications, Open Problems, and ... · Intelligent Automation for cyber security •…ut AI becomes a target of attack –Traditional ML and Deep Neural Networks](https://reader035.fdocuments.in/reader035/viewer/2022070912/5fb41e1a9f97053cae6ff45e/html5/thumbnails/11.jpg)
AI in Cybersecurity
Can AI Improve Security?
11
![Page 12: AI in Cybersecurity: Applications, Open Problems, and ... · Intelligent Automation for cyber security •…ut AI becomes a target of attack –Traditional ML and Deep Neural Networks](https://reader035.fdocuments.in/reader035/viewer/2022070912/5fb41e1a9f97053cae6ff45e/html5/thumbnails/12.jpg)
Industry
12
![Page 13: AI in Cybersecurity: Applications, Open Problems, and ... · Intelligent Automation for cyber security •…ut AI becomes a target of attack –Traditional ML and Deep Neural Networks](https://reader035.fdocuments.in/reader035/viewer/2022070912/5fb41e1a9f97053cae6ff45e/html5/thumbnails/13.jpg)
AI-Enabled Defenses• Spam and phishing detection
– [Castillo et al. 07], [Ma et al. 09]
• Detect compromised accounts in social networks– [Egele et al. 13], [Thomas et al. 14], [Cao et al. 14]
• Malicious web sites and web connections – [Bilge et al. 11], [Antonakakis et al. 12], [Hao et al. 17]
• Predict security events– [Liu et al. 15], [Shen et al. 18]
13
![Page 14: AI in Cybersecurity: Applications, Open Problems, and ... · Intelligent Automation for cyber security •…ut AI becomes a target of attack –Traditional ML and Deep Neural Networks](https://reader035.fdocuments.in/reader035/viewer/2022070912/5fb41e1a9f97053cae6ff45e/html5/thumbnails/14.jpg)
Security Breaches
2011 2014 2017
- Exfiltration of sensitive information- Loss of intellectual property- Financial losses
2013
Source: Verizon DBIR
14
![Page 15: AI in Cybersecurity: Applications, Open Problems, and ... · Intelligent Automation for cyber security •…ut AI becomes a target of attack –Traditional ML and Deep Neural Networks](https://reader035.fdocuments.in/reader035/viewer/2022070912/5fb41e1a9f97053cae6ff45e/html5/thumbnails/15.jpg)
Defenses in Enterprise Networks
Firewall
Internet
WebEmailProxy
DMZ
Internal LAN
FirewallProxyEmail
Endpoint
Data collection
• Security controls deployed for network and host protection• Security logs mostly used for forensic investigation• How can we detect and predict breaches using security logs?
15
![Page 16: AI in Cybersecurity: Applications, Open Problems, and ... · Intelligent Automation for cyber security •…ut AI becomes a target of attack –Traditional ML and Deep Neural Networks](https://reader035.fdocuments.in/reader035/viewer/2022070912/5fb41e1a9f97053cae6ff45e/html5/thumbnails/16.jpg)
Challenges of AI in Security
Limited success of machine learning for security in operational environments [Sommer and Paxson 2010]
• AI is successful in many domains
– Product recommendation, NLP, speech recognition
• What is different in cyber security?
1. High cost of errors (both false positives and false negatives)
2. Variability of user activity under normal conditions
3. Interpretability of results to facilitate manual investigation
4. Resilience against advanced adversaries
16
![Page 17: AI in Cybersecurity: Applications, Open Problems, and ... · Intelligent Automation for cyber security •…ut AI becomes a target of attack –Traditional ML and Deep Neural Networks](https://reader035.fdocuments.in/reader035/viewer/2022070912/5fb41e1a9f97053cae6ff45e/html5/thumbnails/17.jpg)
RSA Analytics Framework
Classification
Outlier detectionClustering
Graph InferenceMachine Learning
Web VPN Firewall AV
Data normalization
Feature extraction
Data Collection
Regression
Incident Response
EnterpriseNetwork
Alerts
Feedback
Behavioral Profiles
Endpoint
Supervised
Semi-Supervised
Unsupervised
17
![Page 18: AI in Cybersecurity: Applications, Open Problems, and ... · Intelligent Automation for cyber security •…ut AI becomes a target of attack –Traditional ML and Deep Neural Networks](https://reader035.fdocuments.in/reader035/viewer/2022070912/5fb41e1a9f97053cae6ff45e/html5/thumbnails/18.jpg)
Key Ideas
• Design ML modules for specific attack patterns
– E.g., C&C, lateral movement, data exfiltration
– Maximize precision and reduce false positive rates
– Combine multiple models for increased recall of malicious activities
• Continuous interaction with EMC CIRC over several years
• Leverage ground truth from existing security products and previous incidents investigated by CIRC
• Interpretability of results
18
Recommendations by [Sommer and Paxson 2010]
![Page 19: AI in Cybersecurity: Applications, Open Problems, and ... · Intelligent Automation for cyber security •…ut AI becomes a target of attack –Traditional ML and Deep Neural Networks](https://reader035.fdocuments.in/reader035/viewer/2022070912/5fb41e1a9f97053cae6ff45e/html5/thumbnails/19.jpg)
MADE▪ Goals
– Identify HTTP Command-and-Control (C&C) communication
▪ Approach
– Use 10 categories of generic and enterprise features (89 total features)
– Enterprise-specific profiles of domains and user-agent strings
– Supervised learning (classification)
▪ Output
– Prioritized list of external C&C domains
19
New findings
A. Oprea, Z. Li, R. Norris, K. Bowers. MADE: Security Analytics for Enterprise Threat Detection. ACSAC 2018.
![Page 20: AI in Cybersecurity: Applications, Open Problems, and ... · Intelligent Automation for cyber security •…ut AI becomes a target of attack –Traditional ML and Deep Neural Networks](https://reader035.fdocuments.in/reader035/viewer/2022070912/5fb41e1a9f97053cae6ff45e/html5/thumbnails/20.jpg)
▪ Goals
– Detect all domains and hosts involved in multi-stage campaigns
▪ Approach
– Semi-supervised learning
– Construct bipartite communication graph
– Label C&C domains as seeds
– Propagate risk with belief propagation
▪ Output
– Prioritized list of malicious domains
– Compromised hosts
C&C
Delivery
Internal hosts
External Destinations
C&C
Delivery
Exfiltration
Multi-Stage Attacks
20
A. Oprea, Z. Li, T.-F. Yen, S. Chin, S. Alrwais. Detection of Early-Stage Enterprise Infection byMining Large-Scale Log Data. DSN 2015.
![Page 21: AI in Cybersecurity: Applications, Open Problems, and ... · Intelligent Automation for cyber security •…ut AI becomes a target of attack –Traditional ML and Deep Neural Networks](https://reader035.fdocuments.in/reader035/viewer/2022070912/5fb41e1a9f97053cae6ff45e/html5/thumbnails/21.jpg)
• Dataset
• 20 TB
• Precision (confirmed malicious)
• 97%
• False positive rates:
• 6x10-3 %
• New detections in one month
• 18 domains
Command-and-Control (C&C)
• Dataset
• 38 TB
• Precision (confirmed malicious)
• 85%
• False positive rates:
• 8.58x10-4 %
• New detections in one month
• 152 domains
• 945 compromised hosts
Deployment Statistics
Multi-Stage Attacks
21
![Page 22: AI in Cybersecurity: Applications, Open Problems, and ... · Intelligent Automation for cyber security •…ut AI becomes a target of attack –Traditional ML and Deep Neural Networks](https://reader035.fdocuments.in/reader035/viewer/2022070912/5fb41e1a9f97053cae6ff45e/html5/thumbnails/22.jpg)
Open Problems: Interpretable Models for Security
• Why does the ML model predict something as attack?
• What type of attack it is?• Is it similar to known attacks?• Is it a new attack/zero-day?• What is the root cause?
![Page 23: AI in Cybersecurity: Applications, Open Problems, and ... · Intelligent Automation for cyber security •…ut AI becomes a target of attack –Traditional ML and Deep Neural Networks](https://reader035.fdocuments.in/reader035/viewer/2022070912/5fb41e1a9f97053cae6ff45e/html5/thumbnails/23.jpg)
Open Problems: Measurable Security
• What are the right metrics in cyber security?• How do we compare different models?• What are some good benchmarks?
![Page 24: AI in Cybersecurity: Applications, Open Problems, and ... · Intelligent Automation for cyber security •…ut AI becomes a target of attack –Traditional ML and Deep Neural Networks](https://reader035.fdocuments.in/reader035/viewer/2022070912/5fb41e1a9f97053cae6ff45e/html5/thumbnails/24.jpg)
Open Problem: Intelligent Automation
Machine Learning
EnvironmentEnterprise
Cloud
AI Defensive Strategy
DefenseManualAutomated
![Page 25: AI in Cybersecurity: Applications, Open Problems, and ... · Intelligent Automation for cyber security •…ut AI becomes a target of attack –Traditional ML and Deep Neural Networks](https://reader035.fdocuments.in/reader035/viewer/2022070912/5fb41e1a9f97053cae6ff45e/html5/thumbnails/25.jpg)
Implications for Cyber Security
• AI has potential in security applications– Complement traditional defenses (crypto, multi-factor
authentication, trusted hardware)
– Design intelligent and adaptive defense algorithms
• …But AI becomes a target of attack– Deep Neural Networks are not resilient to adversarial
manipulations• [Szegedy et al. 13]: “Intriguing properties of neural networks”
– Many critical real-world applications are vulnerable
– New adversarially-resilient algorithms are needed!
AI
25
![Page 26: AI in Cybersecurity: Applications, Open Problems, and ... · Intelligent Automation for cyber security •…ut AI becomes a target of attack –Traditional ML and Deep Neural Networks](https://reader035.fdocuments.in/reader035/viewer/2022070912/5fb41e1a9f97053cae6ff45e/html5/thumbnails/26.jpg)
Security of AI
Can AI Be Secured?
26
![Page 27: AI in Cybersecurity: Applications, Open Problems, and ... · Intelligent Automation for cyber security •…ut AI becomes a target of attack –Traditional ML and Deep Neural Networks](https://reader035.fdocuments.in/reader035/viewer/2022070912/5fb41e1a9f97053cae6ff45e/html5/thumbnails/27.jpg)
Adversarial Machine Learning: Taxonomy
TargetedTarget small set of
points
AvailabilityTarget majority of
points
PrivacyLearn sensitive
information
Training Targeted PoisoningBackdoor
Trojan Attacks
Poisoning Availability
-
Testing Evasion AttacksAdversarial Examples
- Model ExtractionModel Inversion
Attacker’s Objective
Lear
nin
g st
age
27
![Page 28: AI in Cybersecurity: Applications, Open Problems, and ... · Intelligent Automation for cyber security •…ut AI becomes a target of attack –Traditional ML and Deep Neural Networks](https://reader035.fdocuments.in/reader035/viewer/2022070912/5fb41e1a9f97053cae6ff45e/html5/thumbnails/28.jpg)
Adversarial Machine Learning: Taxonomy
TargetedTarget small set of
points
AvailabilityTarget majority of
points
PrivacyLearn sensitive
information
Training Targeted PoisoningBackdoor
Trojan Attacks
Poisoning Availability
-
Testing Evasion AttacksAdversarial Examples
- Model ExtractionModel Inversion
Attacker’s Objective
Lear
nin
g st
age
28
![Page 29: AI in Cybersecurity: Applications, Open Problems, and ... · Intelligent Automation for cyber security •…ut AI becomes a target of attack –Traditional ML and Deep Neural Networks](https://reader035.fdocuments.in/reader035/viewer/2022070912/5fb41e1a9f97053cae6ff45e/html5/thumbnails/29.jpg)
Evasion Attacks
• [Szegedy et al. 13] Intriguing properties of neural networks• [Biggio et al. 13] Evasion Attacks against Machine Learning at Test Time• [Goodfellow et al. 14] Explaining and Harnessing Adversarial Examples• [Carlini, Wagner 17] Towards Evaluating the Robustness of Neural Networks• [Madry et al. 17] Towards Deep Learning Models Resistant to Adversarial Attacks• [Kannan et al. 18] Adversarial Logit Pairing• …
29
Adversarial example
![Page 30: AI in Cybersecurity: Applications, Open Problems, and ... · Intelligent Automation for cyber security •…ut AI becomes a target of attack –Traditional ML and Deep Neural Networks](https://reader035.fdocuments.in/reader035/viewer/2022070912/5fb41e1a9f97053cae6ff45e/html5/thumbnails/30.jpg)
Evasion Attacks For Neural Networks
Z(x)
Softmax
[Carlini and Wagner 2017] Penalty method[Biggio et al. 2013, Madry et al. 2018] Projected Gradient Descent
Input: Images represented as feature vectors
Given input 𝑥Find adversarial example
𝑥′ = 𝑥 + 𝛿
min𝛿
𝑐 𝛿2
2+ 𝑍𝑡(𝑥 + 𝛿)
Optimization Formulation
Min distance Change class
30
![Page 31: AI in Cybersecurity: Applications, Open Problems, and ... · Intelligent Automation for cyber security •…ut AI becomes a target of attack –Traditional ML and Deep Neural Networks](https://reader035.fdocuments.in/reader035/viewer/2022070912/5fb41e1a9f97053cae6ff45e/html5/thumbnails/31.jpg)
Evasion Attacks for Security
Pr[y=1|x]Raw Data
Feature Extraction
Challenge• Attacks in feature space are not feasible in raw data spaceSolution• New iterative attack algorithm taking into account feature constraints
Network Connection
Malicious Benign
31
![Page 32: AI in Cybersecurity: Applications, Open Problems, and ... · Intelligent Automation for cyber security •…ut AI becomes a target of attack –Traditional ML and Deep Neural Networks](https://reader035.fdocuments.in/reader035/viewer/2022070912/5fb41e1a9f97053cae6ff45e/html5/thumbnails/32.jpg)
How Effective are Evasion Attacks in Security?
32
Perfect accuracy (No attack)
Significant degradation under attack
Feed-Forward Neural Network83 features
![Page 33: AI in Cybersecurity: Applications, Open Problems, and ... · Intelligent Automation for cyber security •…ut AI becomes a target of attack –Traditional ML and Deep Neural Networks](https://reader035.fdocuments.in/reader035/viewer/2022070912/5fb41e1a9f97053cae6ff45e/html5/thumbnails/33.jpg)
Evasion Attacks in Connected Cars
Udacity Challenge
• Public competition and dataset 2014
• Steering angle prediction from camera image
Predict direction: Straight, Left, Right33
![Page 34: AI in Cybersecurity: Applications, Open Problems, and ... · Intelligent Automation for cyber security •…ut AI becomes a target of attack –Traditional ML and Deep Neural Networks](https://reader035.fdocuments.in/reader035/viewer/2022070912/5fb41e1a9f97053cae6ff45e/html5/thumbnails/34.jpg)
How Effective are Evasion Attacks in Connected Cars?
Perfect accuracy
(no attack)
Significant degradation under attack
Convolutional Neural Network25 million parameters
34
![Page 35: AI in Cybersecurity: Applications, Open Problems, and ... · Intelligent Automation for cyber security •…ut AI becomes a target of attack –Traditional ML and Deep Neural Networks](https://reader035.fdocuments.in/reader035/viewer/2022070912/5fb41e1a9f97053cae6ff45e/html5/thumbnails/35.jpg)
Adversarial Examples
Original ImageClass “Straight”
Adversarial ImageClass “Right”
Adversarial ImageClass “Left”
35
![Page 36: AI in Cybersecurity: Applications, Open Problems, and ... · Intelligent Automation for cyber security •…ut AI becomes a target of attack –Traditional ML and Deep Neural Networks](https://reader035.fdocuments.in/reader035/viewer/2022070912/5fb41e1a9f97053cae6ff45e/html5/thumbnails/36.jpg)
Adversarial Examples
Original ImageClass “Left”
Adversarial ImageClass “Straight”
Adversarial ImageClass “Right”
36
![Page 37: AI in Cybersecurity: Applications, Open Problems, and ... · Intelligent Automation for cyber security •…ut AI becomes a target of attack –Traditional ML and Deep Neural Networks](https://reader035.fdocuments.in/reader035/viewer/2022070912/5fb41e1a9f97053cae6ff45e/html5/thumbnails/37.jpg)
Taxonomy
TargetedTarget small set of
points
AvailabilityTarget majority of
points
PrivacyLearn sensitive
information
Training Targeted PoisoningBackdoor
Trojan Attacks
Poisoning Availability
-
Testing Evasion AttacksAdversarial Examples
- Model ExtractionModel Inversion
Attacker’s Objective
Lear
nin
g st
age
37
![Page 38: AI in Cybersecurity: Applications, Open Problems, and ... · Intelligent Automation for cyber security •…ut AI becomes a target of attack –Traditional ML and Deep Neural Networks](https://reader035.fdocuments.in/reader035/viewer/2022070912/5fb41e1a9f97053cae6ff45e/html5/thumbnails/38.jpg)
Training-Time Attacks
• ML is trained by crowdsourcing data in many applications
• Cannot fully trust training data!
• Social networks• News articles• Tweets
• Navigation systems• Face recognition• Mobile sensors
38
![Page 39: AI in Cybersecurity: Applications, Open Problems, and ... · Intelligent Automation for cyber security •…ut AI becomes a target of attack –Traditional ML and Deep Neural Networks](https://reader035.fdocuments.in/reader035/viewer/2022070912/5fb41e1a9f97053cae6ff45e/html5/thumbnails/39.jpg)
Poisoning Availability Attacks
Data
Labels
Plane
MLmodel
ML AlgorithmBird
Testing Data
• Attacker Objective:– Corrupt the predictions by the ML model significantly
• Attacker Capability: – Insert fraction of poisoning points in training
M. Jagielski, A. Oprea, B. Biggio, C. Liu, C. Nita-Rotaru, and B. Li. Manipulating Machine Learning: Poisoning Attacks and Countermeasures for Regression Learning. In IEEE S&P 2018
Poisoned Data
39
![Page 40: AI in Cybersecurity: Applications, Open Problems, and ... · Intelligent Automation for cyber security •…ut AI becomes a target of attack –Traditional ML and Deep Neural Networks](https://reader035.fdocuments.in/reader035/viewer/2022070912/5fb41e1a9f97053cae6ff45e/html5/thumbnails/40.jpg)
Optimization Formulation
argmax𝐷𝑝
𝐴(𝐷𝑣𝑎𝑙, 𝜽𝑝) 𝑠. 𝑡.
𝜽𝑝 ∈ argmin𝜽
𝐿(𝐷 ∪ 𝐷𝑝, 𝜽)
Given a training set 𝐷 find a set of poisoning data points 𝐷𝑝
that maximizes the adversary objective 𝐴 on validation set 𝐷𝑣𝑎𝑙
where corrupted model 𝜽𝑝 is learned by minimizing the loss 𝐿 on 𝐷 ∪ 𝐷𝑝
Bilevel Optimization NP-Hard!
40
First white-box attack for regression [Jagielski et al. 18]
• Determine optimal poisoning point (𝒙𝑐,𝑦𝑐)
• Optimize by both 𝒙𝑐 and 𝑦𝑐
![Page 41: AI in Cybersecurity: Applications, Open Problems, and ... · Intelligent Automation for cyber security •…ut AI becomes a target of attack –Traditional ML and Deep Neural Networks](https://reader035.fdocuments.in/reader035/viewer/2022070912/5fb41e1a9f97053cae6ff45e/html5/thumbnails/41.jpg)
How Effective are Poisoning Attacks?• Improve existing attacks by a factor of 6.83
Existing attack
Novel attacks
Predict loan rate with Ridge regression (i.e. with L2 regularization)
Stronger attack
41
![Page 42: AI in Cybersecurity: Applications, Open Problems, and ... · Intelligent Automation for cyber security •…ut AI becomes a target of attack –Traditional ML and Deep Neural Networks](https://reader035.fdocuments.in/reader035/viewer/2022070912/5fb41e1a9f97053cae6ff45e/html5/thumbnails/42.jpg)
Is It Really a Threat?• Case study on healthcare dataset (predict Warfarin medicine dosage )
• At 20% poisoning rate
– Modifies 75% of patients’ dosages by 93.49% for LASSO
– Modifies 10% of patients’ dosages by a factor of 4.59 for Ridge
• At 8% poisoning rate
– Modifies 50% of the patients’ dosages by 75.06%
Quntile Initial Dosage Ridge Difference LASSO Difference
0.1 15.5 mg/wk 31.54% 37.20%
0.25 21 mg/wk 87.50% 93.49%
0.5 30 mg/wk 150.99% 139.31%
0.75 41.53 mg/wk 274.18% 224.08%
0.9 52.5 mg/wk 459.63% 358.89%
42
![Page 43: AI in Cybersecurity: Applications, Open Problems, and ... · Intelligent Automation for cyber security •…ut AI becomes a target of attack –Traditional ML and Deep Neural Networks](https://reader035.fdocuments.in/reader035/viewer/2022070912/5fb41e1a9f97053cae6ff45e/html5/thumbnails/43.jpg)
Open Problem: Understand AI Threat Surface
TargetedTarget small set of
points
AvailabilityTarget majority of
points
PrivacyLearn sensitive
information
Training Targeted PoisoningBackdoor
Trojan Attacks
Poisoning Availability
-
Testing Evasion AttacksAdversarial Examples
- Model ExtractionModel Inversion
Attacker’s Objective
Lear
nin
g st
age
• Application-specific attacks with realistic constraints• How secure is my AI application?
![Page 44: AI in Cybersecurity: Applications, Open Problems, and ... · Intelligent Automation for cyber security •…ut AI becomes a target of attack –Traditional ML and Deep Neural Networks](https://reader035.fdocuments.in/reader035/viewer/2022070912/5fb41e1a9f97053cae6ff45e/html5/thumbnails/44.jpg)
Open Problem: Design Robust AI
• Most AI models are vulnerable in face of attacks!– Evasion (testing-time) attacks – Poisoning (training-time) attacks– Privacy attacks
• How to make AI more robust to attacks?
![Page 45: AI in Cybersecurity: Applications, Open Problems, and ... · Intelligent Automation for cyber security •…ut AI becomes a target of attack –Traditional ML and Deep Neural Networks](https://reader035.fdocuments.in/reader035/viewer/2022070912/5fb41e1a9f97053cae6ff45e/html5/thumbnails/45.jpg)
Takeaways
• AI has potential in security applications– Design intelligent and adaptive defense algorithms
– Open problems: Interpretable models; Measurable security; Intelligent Automation for cyber security
• …But AI becomes a target of attack– Traditional ML and Deep Neural Networks are not resilient
to adversarial manipulations
– Open problem: Understand threat surface for critical real-world applications in systematic way
– Open problem: Design robust AI algorithms in face of attacks
45
![Page 46: AI in Cybersecurity: Applications, Open Problems, and ... · Intelligent Automation for cyber security •…ut AI becomes a target of attack –Traditional ML and Deep Neural Networks](https://reader035.fdocuments.in/reader035/viewer/2022070912/5fb41e1a9f97053cae6ff45e/html5/thumbnails/46.jpg)
Alina [email protected]
Northeastern University Cybersecurity & Privacy Institute
Acknowledgements
46