Demystifying Deep Learning - Roberto Paredes Palacios @ PAPIs Connect
Practical machine learning models to prevent revenue loss — Eiti Kimura and Flávio Clésio...
-
Upload
papisio -
Category
Technology
-
view
297 -
download
0
Transcript of Practical machine learning models to prevent revenue loss — Eiti Kimura and Flávio Clésio...
![Page 1: Practical machine learning models to prevent revenue loss — Eiti Kimura and Flávio Clésio (movile) @PAPIs Connect — São Paulo 2017](https://reader033.fdocuments.in/reader033/viewer/2022052915/5a67f3f77f8b9a8a068b46a3/html5/thumbnails/1.jpg)
Eiti Kimura / Flávio Clésio@Movile Brazil, June
PRACTICAL MACHINE LEARNING MODELS TO PREVENT REVENUE LOSS
PAPIS.io Connect 2017
![Page 2: Practical machine learning models to prevent revenue loss — Eiti Kimura and Flávio Clésio (movile) @PAPIs Connect — São Paulo 2017](https://reader033.fdocuments.in/reader033/viewer/2022052915/5a67f3f77f8b9a8a068b46a3/html5/thumbnails/2.jpg)
ABOUT US
Flávio Clésio
• Core Machine Learning at Movile• MSc. in Production Engineering (Machine Learning in Credit Derivatives/NPL) • Specialist in Database Engineering and Business Intelligence• Blogger at Mineração de Dados (Data Mining) - http://mineracaodedados.wordpress.com• Strata Hadoop World Singapore Speaker
flavioclesio
![Page 3: Practical machine learning models to prevent revenue loss — Eiti Kimura and Flávio Clésio (movile) @PAPIs Connect — São Paulo 2017](https://reader033.fdocuments.in/reader033/viewer/2022052915/5a67f3f77f8b9a8a068b46a3/html5/thumbnails/3.jpg)
ABOUT US
• Software Architect and TI Coordinator at Movile• Msc. in Electrical Engineering• Apache Cassandra MVP (2014/2015 e 2015/2016)• Apache Cassandra Contributor (2015)• Cassandra Summit Speaker (2014 e 2015)• Strata Hadoop World Singapore Speaker
Eiti Kimura
eitikimura
![Page 4: Practical machine learning models to prevent revenue loss — Eiti Kimura and Flávio Clésio (movile) @PAPIs Connect — São Paulo 2017](https://reader033.fdocuments.in/reader033/viewer/2022052915/5a67f3f77f8b9a8a068b46a3/html5/thumbnails/4.jpg)
Movile is the company behind of several apps that makes the life easier
WE MAKE LIFE BETTER THROUGH OUR APPS
![Page 5: Practical machine learning models to prevent revenue loss — Eiti Kimura and Flávio Clésio (movile) @PAPIs Connect — São Paulo 2017](https://reader033.fdocuments.in/reader033/viewer/2022052915/5a67f3f77f8b9a8a068b46a3/html5/thumbnails/5.jpg)
● The Movile's Platform Case
● Practical Machine Learning Model Training
● Results and Conclusions
AGENDA
![Page 6: Practical machine learning models to prevent revenue loss — Eiti Kimura and Flávio Clésio (movile) @PAPIs Connect — São Paulo 2017](https://reader033.fdocuments.in/reader033/viewer/2022052915/5a67f3f77f8b9a8a068b46a3/html5/thumbnails/6.jpg)
MOVILE'S SUBSCRIPTIONAND BILLING PLATFORM IN ITS SIMPLEST FORM● A distributed platform
● User's subscription management
● MISSION CRITICAL platform: can not stop under any circumstance
![Page 7: Practical machine learning models to prevent revenue loss — Eiti Kimura and Flávio Clésio (movile) @PAPIs Connect — São Paulo 2017](https://reader033.fdocuments.in/reader033/viewer/2022052915/5a67f3f77f8b9a8a068b46a3/html5/thumbnails/7.jpg)
REVENUE LEAKAGE LOSSES IMPACT ALONG THE DAY
● 1 Hour Outage: Dozens of thousands of BRL
● 3 Hour Outage: Some hundreds of thousands of BRL
● 12 Hour Outage: Millions of BRL
![Page 8: Practical machine learning models to prevent revenue loss — Eiti Kimura and Flávio Clésio (movile) @PAPIs Connect — São Paulo 2017](https://reader033.fdocuments.in/reader033/viewer/2022052915/5a67f3f77f8b9a8a068b46a3/html5/thumbnails/8.jpg)
How can we check if platform is fully functional based on data analysis only?
MAIN PROBLEM: MONITORING
Tip: what if we ask help to an intelligent system?
![Page 9: Practical machine learning models to prevent revenue loss — Eiti Kimura and Flávio Clésio (movile) @PAPIs Connect — São Paulo 2017](https://reader033.fdocuments.in/reader033/viewer/2022052915/5a67f3f77f8b9a8a068b46a3/html5/thumbnails/9.jpg)
● 230 Millions + of billing requests attempt a day● 4 main mobile carriers drive the operational work
HOW DATA LOOKS LIKE?
![Page 10: Practical machine learning models to prevent revenue loss — Eiti Kimura and Flávio Clésio (movile) @PAPIs Connect — São Paulo 2017](https://reader033.fdocuments.in/reader033/viewer/2022052915/5a67f3f77f8b9a8a068b46a3/html5/thumbnails/10.jpg)
DATA AND ALGORITHM MODELING
Sample of data (predicting the number of success)
featureslabel/target
# success carrier_weight hour week response_time #no_credit #errors # attempts
61.083, [4.0, 17h, 3.0, 1259.0, 24.751.650, 2.193.67, 26.314.551]
SUPERVISED LEARNING
Linear Regression
![Page 11: Practical machine learning models to prevent revenue loss — Eiti Kimura and Flávio Clésio (movile) @PAPIs Connect — São Paulo 2017](https://reader033.fdocuments.in/reader033/viewer/2022052915/5a67f3f77f8b9a8a068b46a3/html5/thumbnails/11.jpg)
● Linear Model with Stochastic Gradient (SDG)
● Lasso with SGD Model (L1 Regularization)
● Ridge Regression with SGD Model (L2 Regularization)
● Decision Tree with Regression Properties (Not CART)
TESTED ALGORITHMS USING SPARK MLLIB
![Page 12: Practical machine learning models to prevent revenue loss — Eiti Kimura and Flávio Clésio (movile) @PAPIs Connect — São Paulo 2017](https://reader033.fdocuments.in/reader033/viewer/2022052915/5a67f3f77f8b9a8a068b46a3/html5/thumbnails/12.jpg)
MODELING LIFECYCLE
Training Data
Testing Data
Feature Extraction
Train
Score
Model
Evaluation
Dataset
![Page 13: Practical machine learning models to prevent revenue loss — Eiti Kimura and Flávio Clésio (movile) @PAPIs Connect — São Paulo 2017](https://reader033.fdocuments.in/reader033/viewer/2022052915/5a67f3f77f8b9a8a068b46a3/html5/thumbnails/13.jpg)
http://spark-notebook.io/
SPARK NOTEBOOK
![Page 14: Practical machine learning models to prevent revenue loss — Eiti Kimura and Flávio Clésio (movile) @PAPIs Connect — São Paulo 2017](https://reader033.fdocuments.in/reader033/viewer/2022052915/5a67f3f77f8b9a8a068b46a3/html5/thumbnails/14.jpg)
Machine Learning Tested Model Accuracy RMSE
Linear Model With SGD 44% 1.64
Lasso with SGD Model 43% 1.65
Ridge Regression with SGD Model 43% 1.66
Decision Tree Model 96% 0.20
TIME TO EVALUATE THE RESULTS
![Page 15: Practical machine learning models to prevent revenue loss — Eiti Kimura and Flávio Clésio (movile) @PAPIs Connect — São Paulo 2017](https://reader033.fdocuments.in/reader033/viewer/2022052915/5a67f3f77f8b9a8a068b46a3/html5/thumbnails/15.jpg)
WATCHER-AI INTRODUCTION
Hi I'm Watcher-ai!It is nice to see you here
![Page 16: Practical machine learning models to prevent revenue loss — Eiti Kimura and Flávio Clésio (movile) @PAPIs Connect — São Paulo 2017](https://reader033.fdocuments.in/reader033/viewer/2022052915/5a67f3f77f8b9a8a068b46a3/html5/thumbnails/16.jpg)
WATCHER-AI ARCHITECTURE
![Page 17: Practical machine learning models to prevent revenue loss — Eiti Kimura and Flávio Clésio (movile) @PAPIs Connect — São Paulo 2017](https://reader033.fdocuments.in/reader033/viewer/2022052915/5a67f3f77f8b9a8a068b46a3/html5/thumbnails/17.jpg)
Avoid to Lose more than U$ 3 Million Dollars preventing leakage
Saved more than 500 working hours
Recovery Time drops from 6 hours to 1 hour
![Page 18: Practical machine learning models to prevent revenue loss — Eiti Kimura and Flávio Clésio (movile) @PAPIs Connect — São Paulo 2017](https://reader033.fdocuments.in/reader033/viewer/2022052915/5a67f3f77f8b9a8a068b46a3/html5/thumbnails/18.jpg)
• Automatic refeed and training using collected data. Analyse more data to predict
possible errors with carrier
• Notify more people and specific teams (more complex problems)
• Refactory to be more generic, so other teams can add their own algorithm
• Why not interact with Watcher to guide the analysis?
• Don't limit your mind, there is a lot to keep improving...
IT IS JUST A WARM UP
![Page 19: Practical machine learning models to prevent revenue loss — Eiti Kimura and Flávio Clésio (movile) @PAPIs Connect — São Paulo 2017](https://reader033.fdocuments.in/reader033/viewer/2022052915/5a67f3f77f8b9a8a068b46a3/html5/thumbnails/19.jpg)
THANK YOU!
@eitikimura eiti-kimura-movile [email protected]
@flavioclesio fclesio [email protected]
github.com/fclesio/watcher-ai-samplespapis-2017-brazil