ModelOps | Operationalising AI Dr Iain Brown · Scale pilot projects to enterprise level Integrate...
17
Copyright © SAS Institute Inc. All rights reserved. ModelOps | Operationalising AI Dr Iain Brown | Head of Data Science, SAS UK & I | Adjunct Professor, University of Southampton
Transcript of ModelOps | Operationalising AI Dr Iain Brown · Scale pilot projects to enterprise level Integrate...
Copyright © SAS Inst itute Inc. A l l r ights reserved.
ModelOps | Operationalising AI
Dr Iain Brown | Head of Data Science, SAS UK & I
| Adjunct Professor, University of Southampton
Copyright © SAS Inst itute Inc. A l l r ights reserved.
Over 60% of models developed with
the intention of operationalizing them
were never actually
The “Last Mile”: How to Consistently Extract
Value from Data Analytics
“The inability to integrate analytic solutions into workflows
and achieve frontline adoption is the number one inhibitor to
why data and analytics initiatives fail.”
70% of enterprises view advanced analytics as a critical
strategic priority, but only 10% actually believe they're
achieving it.
Copyright © SAS Inst itute Inc. A l l r ights reserved.
AI Project
Ref: “Hidden Technical Debt in Machine Learning Systems”, Google Inc.
1
Copyright © SAS Inst itute Inc. A l l r ights reserved.
CHALLENGES
NAVIGATING THE CHAOS
Complex analytic ecosystem
Too many choices
TIP OF THE ICEBERG
Poor time to value
Specialized resources
TURNING THE SHIP
Resistance to change
Lack of KPIs
Copyright © SAS Inst itute Inc. A l l r ights reserved.
TIME
VALUE OF ANALYTICS
Prepare Data
Explore Build Models
Rewrite toDeploy Model
Deploy ModelManually
PoorInsights
No GovernanceModel Decay
LOST BUSINESS OPPORTUNITY
ANALYTIC’S LAST MILE
Manual Retraining
Manual Model DeploymentLost Opportunities
Copyright © SAS Inst itute Inc. A l l r ights reserved.
TIME
VALUE OF ANALYTICS
Prepare Data
Explore Build Models
Rewrite toDeploy Model
Deploy ModelManually
PoorInsightsBuild better
models faster
More Data, Intelligent Data
Preparation Automated Model
Deployment
Monitor & Manage Models
Automated Decisioning Model Governance
Improved insights
Realized Business
Value
Ongoing Business impact
No GovernanceModel Decay
ANALYTIC’S LAST MILE
Operationalized AnalyticsFaster, Greater Business Value
Copyright © SAS Inst itute Inc. A l l r ights reserved.
Development – Historical Data
ProblemDefinition
TransformAnd Select
TrainModel
EvaluateModel
RetrainModel
MonitorResults
ServeModel
AccumulateData
DataQualityAnalysis
Production – New Live Data
Data Engineer• Data Preparation• Deployment services• Report administration
Data Scientist• Exploratory analysis• Feature engineering• Model development
Business Manager• Manages campaigns• Domain expert• Evaluates processes
and ROI
ModelOpsFrom the Lab to Production
Copyright © SAS Inst itute Inc. A l l r ights reserved.
Data Replication Required for Model Serving
Model Training – Historical Data
TransformAnd Select
ModelFormulate
ModelValidation
ServeModel
MonitorResults
RetrainModel
AccumulateData
DataQualityAnalysis
Model Serving – New Data
Data Scientist
Data Engineer
Copyright © SAS Inst itute Inc. A l l r ights reserved.
…To Infinity and Beyond
Operationalizing AI
Copyright © SAS Inst itute Inc. A l l r ights reserved.
Build once and deploy rapidly
anywhere
ModelGovernance
Central Repository
Centralise and Deploy
Deployment
Copyright © SAS Inst itute Inc. A l l r ights reserved.
Decisioning
Deployment
Automate high volumeinteractions
Manage decisions
Business ruleand analytical
model execution
Copyright © SAS Inst itute Inc. A l l r ights reserved.
Monitor and Improve
Deployment
Monitor Performance
Improve
Copyright © SAS Inst itute Inc. A l l r ights reserved.
REST APIs
Batch
ContainerStreaming,
Edge Devices
RecodingAlignment
with DevOps
Copyright © SAS Inst itute Inc. A l l r ights reserved.
Study: The AI DividendThe views of AI experts from data scientists to the boardroom
12 focus groups
54 participants
Strong optimism around the
potential of AI, though students were
the most fearful about the future.
Healthcare seen as the
model for ‘AI done right’
AI viewed as a powerful source
of competitive advantage by
business leaders.
Obstacles to consumer trust:
- Unconscious bias and inaccuracy
- Lack of data responsibility and privacy
- Little transparency or explainability
- Disregard of ethics by creators and
businesses
https://www.sas.com/en_gb/whitepapers/artificial-intelligence-searching-for-the-ai-dividend.html
Copyright © SAS Inst itute Inc. A l l r ights reserved.
T EAF
Copyright © SAS Inst itute Inc. A l l r ights reserved.
Machine Learning Model(new)
MemberData
Accurate pre-approvals for PLs
Application Data
Transactional
Behaviour
12months PL
applications
+
Validation Oversight Team
(existing)
+
Robust over time?
Robust under
heavy sampling?
Explain the
unexplainable?
=
Multiple approaches
compared (NN’s, RF’s and
SGB)
Stochastic Gradient
Boosting selected
10%
Copyright © SAS Inst itute Inc. A l l r ights reserved.
VALUE OF MODELOPS
Do not miss opportunities
Drive Additional Business Value
Manage Risk and Compliance
Deliver Insightsat Scale
Gain trust and transparency
Scale pilot projects to enterprise level
Integrate models with rules for best actions to take, at scale
Monitor model effectiveness and decay
Centralized governance of analytic assets
