Anomaly Detection for Global Scale at Netflix

17
GLOBAL SCALE ANOMALY DETECTION FOR

Transcript of Anomaly Detection for Global Scale at Netflix

Page 1: Anomaly Detection for Global Scale at Netflix

GLOBAL SCALEANOMALY DETECTION FOR

Page 2: Anomaly Detection for Global Scale at Netflix

DIRECTOR, DATA ENGINEERING & ANALYTICSPAUL ELLWOOD

Page 3: Anomaly Detection for Global Scale at Netflix

1 INTERNATIONAL

EXPANSION

Page 4: Anomaly Detection for Global Scale at Netflix
Page 5: Anomaly Detection for Global Scale at Netflix

2 FINITE ANALYST

CAPACITY

Page 6: Anomaly Detection for Global Scale at Netflix

UNREALISTIC EXPECTATIONS

TEST ANALYSIS DASHBOARD

▸ One test

▸ One report (of the five)

▸ Ignoring multi-select combinations

▸ Ignoring date range selections

Page 7: Anomaly Detection for Global Scale at Netflix

16,821,043,200 UNIQUE VIEWS

Page 8: Anomaly Detection for Global Scale at Netflix

3 ANOMALY

DETECTION

Page 9: Anomaly Detection for Global Scale at Netflix
Page 10: Anomaly Detection for Global Scale at Netflix
Page 11: Anomaly Detection for Global Scale at Netflix

SMARTER ANALYTIC PRODUCTS

Page 12: Anomaly Detection for Global Scale at Netflix

SMARTER ANALYTIC PRODUCTS

▸ Statistical Anomalies

▸ Personalized

▸ Filtered & Ranked by Impact

▸ Usable

▸ Ability to Dig Deeper

Page 13: Anomaly Detection for Global Scale at Netflix

CASE STUDY

PAYMENT PROCESSING

▸ Multiple dimensions

▸ High cardinality dimensions

▸ Hundreds of thousands of time series

▸ Increasing volumes

▸ High business impact

Page 14: Anomaly Detection for Global Scale at Netflix
Page 15: Anomaly Detection for Global Scale at Netflix
Page 16: Anomaly Detection for Global Scale at Netflix

QUESTIONS?

Page 17: Anomaly Detection for Global Scale at Netflix

DIRECTOR, DATA ENGINEERING & ANALYTICSPAUL ELLWOOD

@pellwoodhttps://www.linkedin.com/in/pellwood

https://www.linkedin.com/in/pellwood

Anomaly Detection and Mitigation. Outline DoS and DDoS Anomaly Detection and Mitigation Systems Cisco DDoS Anomaly Detection and Mitigation Solutions.

Anomaly Detection and Mitigation. Outline DoS and DDoS Anomaly Detection and Mitigation Systems Cisco DDoS Anomaly Detection and Mitigation Solutions.

Chap10 Anomaly Detection

Chap10 Anomaly Detection

Anomaly Detection Technology Using BigGraph€¦ · Anomaly Detection Technology Using BigGraph ... detection Several anomaly detection techniques have been developed and applied

Anomaly Detection Technology Using BigGraph€¦ · Anomaly Detection Technology Using BigGraph ... detection Several anomaly detection techniques have been developed and applied

Anomaly Detection: Principles, Benchmarking, Explanation ...web.engr.oregonstate.edu/~tgd/...anomaly-detection... · Towards a Theory of Anomaly Detection [Siddiqui, et al.; UAI 2016]

Anomaly Detection: Principles, Benchmarking, Explanation ...web.engr.oregonstate.edu/~tgd/...anomaly-detection... · Towards a Theory of Anomaly Detection [Siddiqui, et al.; UAI 2016]

Deep Learning based Anomaly Detection in€¦ · Deep Learning based Anomaly Detection in Multivariate Time-Series Data 1. Background Anomaly detection, also known as outlier detection,

Deep Learning based Anomaly Detection in€¦ · Deep Learning based Anomaly Detection in Multivariate Time-Series Data 1. Background Anomaly detection, also known as outlier detection,

Network Traffic Anomaly Detection and Characterizationcs.northwestern.edu/~ajb200/anomaly detection paper 1.0.pdf · Network Traffic Anomaly Detection and Characterization Aaron Beach

Network Traffic Anomaly Detection and Characterizationcs.northwestern.edu/~ajb200/anomaly detection paper 1.0.pdf · Network Traffic Anomaly Detection and Characterization Aaron Beach

Probabilistic Anomaly Detection in Dynamic Systemspapers.nips.cc/paper/805-probabilistic-anomaly-detection-in... · Probabilistic Anomaly Detection in Dynamic Systems 827 model (HMM)

Probabilistic Anomaly Detection in Dynamic Systemspapers.nips.cc/paper/805-probabilistic-anomaly-detection-in... · Probabilistic Anomaly Detection in Dynamic Systems 827 model (HMM)

Anomaly detection final

Anomaly detection final

Anomaly Detection - University of Edinburghudrc.eng.ed.ac.uk/sites/udrc.eng.ed.ac.uk/files... · Anomaly detection system architecture ! Incongruence detection ! Dempster Shaffer

Anomaly Detection - University of Edinburghudrc.eng.ed.ac.uk/sites/udrc.eng.ed.ac.uk/files... · Anomaly detection system architecture ! Incongruence detection ! Dempster Shaffer

Unsupervised Anomaly Detection in Large Datacenters › ~mgabel › papers › MSc Thesis - Final.pdf · 2014-06-14 · Unsupervised Anomaly Detection ... Unsupervised Anomaly Detection

Unsupervised Anomaly Detection in Large Datacenters › ~mgabel › papers › MSc Thesis - Final.pdf · 2014-06-14 · Unsupervised Anomaly Detection ... Unsupervised Anomaly Detection

Anomaly Detection Techniques

Anomaly Detection Techniques

Intrusion & Anomaly Detection & Preventiongauss.ececs.uc.edu/Courses/c6056/pdf/intrusion.pdfIntrusion & Anomaly Detection & Prevention Intrusion Detection: Monitor events, analyze

Intrusion & Anomaly Detection & Preventiongauss.ececs.uc.edu/Courses/c6056/pdf/intrusion.pdfIntrusion & Anomaly Detection & Prevention Intrusion Detection: Monitor events, analyze

Anomaly Detection Survey

Anomaly Detection Survey

Anomaly Detection in Predictive Maintenance - KNIME · 1 Anomaly Detection in Predictive Maintenance Time Alignment and Visualization for Anomaly Detection Rosaria Silipo Rosaria.Silipo@knime.com

Anomaly Detection in Predictive Maintenance - KNIME · 1 Anomaly Detection in Predictive Maintenance Time Alignment and Visualization for Anomaly Detection Rosaria Silipo [email protected]

BGP Anomaly Detection

BGP Anomaly Detection

Anomaly Detection and Attribution Using Bayesian … · UNCLASSIFIED DSTO{TR{2975 Anomaly Detection and Attribution Using Bayesian Networks Executive Summary Anomaly detection techniques

Anomaly Detection and Attribution Using Bayesian … · UNCLASSIFIED DSTO{TR{2975 Anomaly Detection and Attribution Using Bayesian Networks Executive Summary Anomaly detection techniques

Anomaly Detection RapidMiner

Anomaly Detection RapidMiner

Real-Time Anomaly Detection in Water Distribution Networks ... Anomaly Detection in Water Distribution Networks using Spark Streaming ... 2.3 Anomaly Detection in Water Distribution

Real-Time Anomaly Detection in Water Distribution Networks ... Anomaly Detection in Water Distribution Networks using Spark Streaming ... 2.3 Anomaly Detection in Water Distribution

Taxonomy of Anomaly Based Intrusion Detection System… · Figure 1: Taxonomy of Anomaly based Intrusion Detection System. III. STATISTICAL ANOMALY BASED INTRUSION DETECTION SYSTEM

Taxonomy of Anomaly Based Intrusion Detection System… · Figure 1: Taxonomy of Anomaly based Intrusion Detection System. III. STATISTICAL ANOMALY BASED INTRUSION DETECTION SYSTEM

Anomaly Detection using Outlier Detection Schemes

Anomaly Detection using Outlier Detection Schemes