How to deploy machine learning models in the Cloud
-
Upload
alex-casalboni -
Category
Technology
-
view
1.925 -
download
3
Transcript of How to deploy machine learning models in the Cloud
![Page 1: How to deploy machine learning models in the Cloud](https://reader033.fdocuments.in/reader033/viewer/2022042619/586fb4a61a28abe57d8b72bd/html5/thumbnails/1.jpg)
How to deploy Machine Learning models in the Cloud
Alex Casalboni
PYCON SETTE @ Firenze
clda.co/pycon7-alex
![Page 2: How to deploy machine learning models in the Cloud](https://reader033.fdocuments.in/reader033/viewer/2022042619/586fb4a61a28abe57d8b72bd/html5/thumbnails/2.jpg)
About Me
PYCON SETTE @ FIRENZE
@alex_casalboni
clda.co/pycon7-alex
Bachelor in Computer Science
Master in Sound & Music Engineering
6+ years of Web Development
3+ years of Python
![Page 3: How to deploy machine learning models in the Cloud](https://reader033.fdocuments.in/reader033/viewer/2022042619/586fb4a61a28abe57d8b72bd/html5/thumbnails/3.jpg)
Have you ever met a Data Scien@st?
PYCON SETTE @ FIRENZE
Bob
Bob leads a skilled team of data scienCsts
Bob has built a great ML model
Bob loves Python
Bob likes user-‐centered ML
clda.co/pycon7-alex
![Page 4: How to deploy machine learning models in the Cloud](https://reader033.fdocuments.in/reader033/viewer/2022042619/586fb4a61a28abe57d8b72bd/html5/thumbnails/4.jpg)
How did Bob build his model in Python?
PYCON SETTE @ FIRENZE
clda.co/pycon7-code
clda.co/pycon7-alex
Bob
![Page 5: How to deploy machine learning models in the Cloud](https://reader033.fdocuments.in/reader033/viewer/2022042619/586fb4a61a28abe57d8b72bd/html5/thumbnails/5.jpg)
PYCON SETTE @ FIRENZE
GPU availability
distributed compuCng
infrastructure-‐aware code
1. Bob needs Elas@city
clda.co/pycon7-alex
And why is deploying a challenge?
Bob
![Page 6: How to deploy machine learning models in the Cloud](https://reader033.fdocuments.in/reader033/viewer/2022042619/586fb4a61a28abe57d8b72bd/html5/thumbnails/6.jpg)
PYCON SETTE @ FIRENZE
MulC-‐model architecture
A/B tesCng & versioning
RESTful API
2. Bob needs plenty of Features
Auth layer
clda.co/pycon7-alex
And why is deploying a challenge?
Bob
![Page 7: How to deploy machine learning models in the Cloud](https://reader033.fdocuments.in/reader033/viewer/2022042619/586fb4a61a28abe57d8b72bd/html5/thumbnails/7.jpg)
PYCON SETTE @ FIRENZE
infrastructure maintenance
conCnuous delivery & deployments
automaCon, performance & monitoring
3. Data Scien@sts ain’t DevOps
≠
clda.co/pycon7-alex
And why is deploying a challenge?
Data ScienCst DevOps
![Page 8: How to deploy machine learning models in the Cloud](https://reader033.fdocuments.in/reader033/viewer/2022042619/586fb4a61a28abe57d8b72bd/html5/thumbnails/8.jpg)
PYCON SETTE @ FIRENZE
model design, training & tesCng
code reviews & debugging
data exploraCon
4. DevOps ain’t Data Scien@sts
≠
clda.co/pycon7-alex
And why is deploying a challenge?
Data ScienCstDevOps
![Page 9: How to deploy machine learning models in the Cloud](https://reader033.fdocuments.in/reader033/viewer/2022042619/586fb4a61a28abe57d8b72bd/html5/thumbnails/9.jpg)
Deployment Strategies
PYCON SETTE @ FIRENZE
1. Web-‐app controller
clda.co/pycon7-alex
how will Bob update his model(s)?
same website auth layer?
shared (limited) uWSGI processes?
It’s disarmingly simple, but…
Bob
![Page 10: How to deploy machine learning models in the Cloud](https://reader033.fdocuments.in/reader033/viewer/2022042619/586fb4a61a28abe57d8b72bd/html5/thumbnails/10.jpg)
Deployment Strategies
PYCON SETTE @ FIRENZE
2. Fleet of servers
clda.co/pycon7-alex
same problems as before
Bob has many more machines to maintain
sCll no elasCcity (over-‐provisioning)
Bigger capacity and no code changes, but…
Bob Bob2 Bob3
Load Balancing
![Page 11: How to deploy machine learning models in the Cloud](https://reader033.fdocuments.in/reader033/viewer/2022042619/586fb4a61a28abe57d8b72bd/html5/thumbnails/11.jpg)
Deployment Strategies
PYCON SETTE @ FIRENZE
3. Auto Scaling
clda.co/pycon7-alex
<Bob>
sCll shared resources? containers?
even bigger lack of ownership
what about caching, versioning and auth?
Bob achieved elasCcity and availability , but…
AWS ELB + Auto Scaling (or maybe ElasCc Beanstalk?)
![Page 12: How to deploy machine learning models in the Cloud](https://reader033.fdocuments.in/reader033/viewer/2022042619/586fb4a61a28abe57d8b72bd/html5/thumbnails/12.jpg)
Deployment Strategies
PYCON SETTE @ FIRENZE
4. Serverless
clda.co/pycon7-alex
+Amazon API Gateway
AWS Lambda
Happy Bob
Infinite scalability, global CDN and caching
Transparent elasCcity (and PAYG)
RESTful & auth layer + versioning (for free!)
Bob doesn’t need to be a DevOps anymore!
![Page 13: How to deploy machine learning models in the Cloud](https://reader033.fdocuments.in/reader033/viewer/2022042619/586fb4a61a28abe57d8b72bd/html5/thumbnails/13.jpg)
The Serverless Cloud
PYCON SETTE @ FIRENZEclda.co/pycon7-alex
There is always a server somewhere, you just don't have to worry about it :)
![Page 14: How to deploy machine learning models in the Cloud](https://reader033.fdocuments.in/reader033/viewer/2022042619/586fb4a61a28abe57d8b72bd/html5/thumbnails/14.jpg)
Does Bob’s code change a lot?
PYCON SETTE @ FIRENZEclda.co/pycon7-alex
Basically, no change!
Bob
![Page 15: How to deploy machine learning models in the Cloud](https://reader033.fdocuments.in/reader033/viewer/2022042619/586fb4a61a28abe57d8b72bd/html5/thumbnails/15.jpg)
How does Bob keep the model updated?
PYCON SETTE @ FIRENZEclda.co/pycon7-alex
Bob
Hint: use gzip!
![Page 16: How to deploy machine learning models in the Cloud](https://reader033.fdocuments.in/reader033/viewer/2022042619/586fb4a61a28abe57d8b72bd/html5/thumbnails/16.jpg)
PYCON SETTE @ FIRENZE
clda.co/pycon7-lambda
clda.co/pycon7-alex
![Page 17: How to deploy machine learning models in the Cloud](https://reader033.fdocuments.in/reader033/viewer/2022042619/586fb4a61a28abe57d8b72bd/html5/thumbnails/17.jpg)
Serverless Cloud Domina@on
PYCON SETTE @ FIRENZE
Amazon Web Services (Lambda)
Microso` Azure (FuncCons)
Google Cloud Plaborm (FuncCons)
IBM Bluemix (OpenWhisk)
hook.io
Cloud PlaSorms Hybrid & Microservices
stackhut.com
webtask.io
iron.io
clda.co/pycon7-alex