Trends Affecting System Architectures: Mobile …...Ka;a Streams Akka Streams … Sessions Streams...
Transcript of Trends Affecting System Architectures: Mobile …...Ka;a Streams Akka Streams … Sessions Streams...
![Page 1: Trends Affecting System Architectures: Mobile …...Ka;a Streams Akka Streams … Sessions Streams Storage Device 1 Telemetry 2. → Model Training 3. ← New models 2, 3 4. ← Model](https://reader036.fdocuments.in/reader036/viewer/2022062505/5ece3dbe3111ee591f38650b/html5/thumbnails/1.jpg)
Dean Wampler, Ph.D. [email protected] polyglotprogramming.com/talks
Trends Affecting System Architectures: Mobile Engagement, Streaming Data, and Machine Learning
©DeanWampler&Lightbend,2007-2019 @deanwampler
![Page 2: Trends Affecting System Architectures: Mobile …...Ka;a Streams Akka Streams … Sessions Streams Storage Device 1 Telemetry 2. → Model Training 3. ← New models 2, 3 4. ← Model](https://reader036.fdocuments.in/reader036/viewer/2022062505/5ece3dbe3111ee591f38650b/html5/thumbnails/2.jpg)
lbnd.io/fast-data-ebook
Data Streaming Architectures
@deanwampler
![Page 4: Trends Affecting System Architectures: Mobile …...Ka;a Streams Akka Streams … Sessions Streams Storage Device 1 Telemetry 2. → Model Training 3. ← New models 2, 3 4. ← Model](https://reader036.fdocuments.in/reader036/viewer/2022062505/5ece3dbe3111ee591f38650b/html5/thumbnails/4.jpg)
But first, introductions…
What we’re up to at Lightbend… lightbend.com/lightbend-pipelines-demo
![Page 5: Trends Affecting System Architectures: Mobile …...Ka;a Streams Akka Streams … Sessions Streams Storage Device 1 Telemetry 2. → Model Training 3. ← New models 2, 3 4. ← Model](https://reader036.fdocuments.in/reader036/viewer/2022062505/5ece3dbe3111ee591f38650b/html5/thumbnails/5.jpg)
What We’ll Discuss
![Page 6: Trends Affecting System Architectures: Mobile …...Ka;a Streams Akka Streams … Sessions Streams Storage Device 1 Telemetry 2. → Model Training 3. ← New models 2, 3 4. ← Model](https://reader036.fdocuments.in/reader036/viewer/2022062505/5ece3dbe3111ee591f38650b/html5/thumbnails/6.jpg)
What We’ll Discuss
• Forces driving us towards stream processing• Streaming architectures • Requirements• An example
• Mixing data science and data engineering • Other production concerns
![Page 7: Trends Affecting System Architectures: Mobile …...Ka;a Streams Akka Streams … Sessions Streams Storage Device 1 Telemetry 2. → Model Training 3. ← New models 2, 3 4. ← Model](https://reader036.fdocuments.in/reader036/viewer/2022062505/5ece3dbe3111ee591f38650b/html5/thumbnails/7.jpg)
Forces driving us towards stream processing
![Page 8: Trends Affecting System Architectures: Mobile …...Ka;a Streams Akka Streams … Sessions Streams Storage Device 1 Telemetry 2. → Model Training 3. ← New models 2, 3 4. ← Model](https://reader036.fdocuments.in/reader036/viewer/2022062505/5ece3dbe3111ee591f38650b/html5/thumbnails/8.jpg)
EnergyMedical
Finance
… and IoT
Telecom
MobileStateoftheartphone!
![Page 9: Trends Affecting System Architectures: Mobile …...Ka;a Streams Akka Streams … Sessions Streams Storage Device 1 Telemetry 2. → Model Training 3. ← New models 2, 3 4. ← Model](https://reader036.fdocuments.in/reader036/viewer/2022062505/5ece3dbe3111ee591f38650b/html5/thumbnails/9.jpg)
EnergyMedical
Finance
… and IoT
Telecom
Mobile@deanwampler
Information value has a half life;
it decays with time
![Page 10: Trends Affecting System Architectures: Mobile …...Ka;a Streams Akka Streams … Sessions Streams Storage Device 1 Telemetry 2. → Model Training 3. ← New models 2, 3 4. ← Model](https://reader036.fdocuments.in/reader036/viewer/2022062505/5ece3dbe3111ee591f38650b/html5/thumbnails/10.jpg)
EnergyMedical
Finance
… and IoT
Telecom
Mobile
Historically, software developers have downplayed the
importance of data.We can’t anymore.
@deanwampler
![Page 11: Trends Affecting System Architectures: Mobile …...Ka;a Streams Akka Streams … Sessions Streams Storage Device 1 Telemetry 2. → Model Training 3. ← New models 2, 3 4. ← Model](https://reader036.fdocuments.in/reader036/viewer/2022062505/5ece3dbe3111ee591f38650b/html5/thumbnails/11.jpg)
@deanwampler
Streaming Architectures: Requirements
![Page 12: Trends Affecting System Architectures: Mobile …...Ka;a Streams Akka Streams … Sessions Streams Storage Device 1 Telemetry 2. → Model Training 3. ← New models 2, 3 4. ← Model](https://reader036.fdocuments.in/reader036/viewer/2022062505/5ece3dbe3111ee591f38650b/html5/thumbnails/12.jpg)
@deanwampler
• Reliability - fault and “surprise” tolerant• Availability - “always on”• Low latency - for some definition of “low”• Scalability - up and down• Adaptability - ideally without restarts
![Page 13: Trends Affecting System Architectures: Mobile …...Ka;a Streams Akka Streams … Sessions Streams Storage Device 1 Telemetry 2. → Model Training 3. ← New models 2, 3 4. ← Model](https://reader036.fdocuments.in/reader036/viewer/2022062505/5ece3dbe3111ee591f38650b/html5/thumbnails/13.jpg)
@deanwampler
Reminds me of microservices
…
Browse
REST
AccountOrders
ShoppingCart
API Gateway
Inventory
![Page 14: Trends Affecting System Architectures: Mobile …...Ka;a Streams Akka Streams … Sessions Streams Storage Device 1 Telemetry 2. → Model Training 3. ← New models 2, 3 4. ← Model](https://reader036.fdocuments.in/reader036/viewer/2022062505/5ece3dbe3111ee591f38650b/html5/thumbnails/14.jpg)
@deanwampler
Streaming Architectures: An Example
![Page 15: Trends Affecting System Architectures: Mobile …...Ka;a Streams Akka Streams … Sessions Streams Storage Device 1 Telemetry 2. → Model Training 3. ← New models 2, 3 4. ← Model](https://reader036.fdocuments.in/reader036/viewer/2022062505/5ece3dbe3111ee591f38650b/html5/thumbnails/15.jpg)
DataCenter
Mini-batch,Batch
Spark
LowLatencyMicroservices
Ka;aStreamsAkkaStreams
…
Sessions
Streams
Storage
Device
Telemetry1
2. → Model Training3. ← New models
2, 3
4. ← Model Storage5. → Boot up, historical data
4, 5
6. → Data Pipeline 7. → Model Serving 8. ← Anomalies
6, 7, 8Ingest Scores9
Corrective Action
10
Spark
MicroserviceMicroserviceMicroservice
DeviceSessionMicroservices
Persistence
Ka+a Cluster
Broker
@deanwampler
Detect anomalies in IoT devices
![Page 16: Trends Affecting System Architectures: Mobile …...Ka;a Streams Akka Streams … Sessions Streams Storage Device 1 Telemetry 2. → Model Training 3. ← New models 2, 3 4. ← Model](https://reader036.fdocuments.in/reader036/viewer/2022062505/5ece3dbe3111ee591f38650b/html5/thumbnails/16.jpg)
DataCenter
Mini-batch,Batch
Spark
LowLatencyMicroservices
Ka;aStreamsAkkaStreams
…
Sessions
Streams
Storage
Device
Telemetry1
2. → Model Training3. ← New models
2, 3
4. ← Model Storage5. → Boot up, historical data
4, 5
6. → Data Pipeline 7. → Model Serving 8. ← Anomalies
6, 7, 8Ingest Scores9
Corrective Action
10
Spark
MicroserviceMicroserviceMicroservice
DeviceSessionMicroservices
Persistence
Ka+a Cluster
Broker
@deanwampler
Sessionmanagement,RESTmicroservices Model
Scoring
ModelTraining
Threegroupsoffunctionality
![Page 17: Trends Affecting System Architectures: Mobile …...Ka;a Streams Akka Streams … Sessions Streams Storage Device 1 Telemetry 2. → Model Training 3. ← New models 2, 3 4. ← Model](https://reader036.fdocuments.in/reader036/viewer/2022062505/5ece3dbe3111ee591f38650b/html5/thumbnails/17.jpg)
DataCenter
Mini-batch,Batch
Spark
LowLatencyMicroservices
Ka;aStreamsAkkaStreams
…
Sessions
Streams
Storage
Device
Telemetry1
2. → Model Training3. ← New models
2, 3
4. ← Model Storage5. → Boot up, historical data
4, 5
6. → Data Pipeline 7. → Model Serving 8. ← Anomalies
6, 7, 8Ingest Scores9
Corrective Action
10
Spark
MicroserviceMicroserviceMicroservice
DeviceSessionMicroservices
Persistence
Ka+a Cluster
Broker
@deanwampler
IngestdevicetelemetrytoKafka
![Page 18: Trends Affecting System Architectures: Mobile …...Ka;a Streams Akka Streams … Sessions Streams Storage Device 1 Telemetry 2. → Model Training 3. ← New models 2, 3 4. ← Model](https://reader036.fdocuments.in/reader036/viewer/2022062505/5ece3dbe3111ee591f38650b/html5/thumbnails/18.jpg)
DataCenter
Mini-batch,Batch
Spark
LowLatencyMicroservices
Ka;aStreamsAkkaStreams
…
Sessions
Streams
Storage
Device
Telemetry1
2. → Model Training3. ← New models
2, 3
4. ← Model Storage5. → Boot up, historical data
4, 5
6. → Data Pipeline 7. → Model Serving 8. ← Anomalies
6, 7, 8Ingest Scores9
Corrective Action
10
Spark
MicroserviceMicroserviceMicroservice
DeviceSessionMicroservices
Persistence
Ka+a Cluster
Broker
@deanwampler
IngestdevicetelemetrytoKafka
Analogs
At restStreaming
Microservices
Ka+aStreamsAkkaStreams
…
Ka#a Cluster
Broker
Mini-batch
SparkSparkFlink…
HDFS
Batch
SparkMapReduce
…
![Page 19: Trends Affecting System Architectures: Mobile …...Ka;a Streams Akka Streams … Sessions Streams Storage Device 1 Telemetry 2. → Model Training 3. ← New models 2, 3 4. ← Model](https://reader036.fdocuments.in/reader036/viewer/2022062505/5ece3dbe3111ee591f38650b/html5/thumbnails/19.jpg)
DataCenter
Mini-batch,Batch
Spark
LowLatencyMicroservices
Ka;aStreamsAkkaStreams
…
Sessions
Streams
Storage
Device
Telemetry1
2. → Model Training3. ← New models
2, 3
4. ← Model Storage5. → Boot up, historical data
4, 5
6. → Data Pipeline 7. → Model Serving 8. ← Anomalies
6, 7, 8Ingest Scores9
Corrective Action
10
Spark
MicroserviceMicroserviceMicroservice
DeviceSessionMicroservices
Persistence
Ka+a Cluster
Broker
@deanwampler
ReadtelemetryintoaperiodicSparkjobformodeltrainingtodetectanomalies
Largedatavolume,Longlatency(seconds-days)
![Page 20: Trends Affecting System Architectures: Mobile …...Ka;a Streams Akka Streams … Sessions Streams Storage Device 1 Telemetry 2. → Model Training 3. ← New models 2, 3 4. ← Model](https://reader036.fdocuments.in/reader036/viewer/2022062505/5ece3dbe3111ee591f38650b/html5/thumbnails/20.jpg)
DataCenter
Mini-batch,Batch
Spark
LowLatencyMicroservices
Ka;aStreamsAkkaStreams
…
Sessions
Streams
Storage
Device
Telemetry1
2. → Model Training3. ← New models
2, 3
4. ← Model Storage5. → Boot up, historical data
4, 5
6. → Data Pipeline 7. → Model Serving 8. ← Anomalies
6, 7, 8Ingest Scores9
Corrective Action
10
Spark
MicroserviceMicroserviceMicroservice
DeviceSessionMicroservices
Persistence
Ka+a Cluster
Broker
@deanwampler
UpdatedmodelparametersarewrittenbacktoKafkainanewtopic
![Page 21: Trends Affecting System Architectures: Mobile …...Ka;a Streams Akka Streams … Sessions Streams Storage Device 1 Telemetry 2. → Model Training 3. ← New models 2, 3 4. ← Model](https://reader036.fdocuments.in/reader036/viewer/2022062505/5ece3dbe3111ee591f38650b/html5/thumbnails/21.jpg)
DataCenter
Mini-batch,Batch
Spark
LowLatencyMicroservices
Ka;aStreamsAkkaStreams
…
Sessions
Streams
Storage
Device
Telemetry1
2. → Model Training3. ← New models
2, 3
4. ← Model Storage5. → Boot up, historical data
4, 5
6. → Data Pipeline 7. → Model Serving 8. ← Anomalies
6, 7, 8Ingest Scores9
Corrective Action
10
Spark
MicroserviceMicroserviceMicroservice
DeviceSessionMicroservices
Persistence
Ka+a Cluster
Broker
@deanwampler
Updatedmodelparametersarealsowrittentolonger-termstorage(forsystemrestarts,auditing,…)
![Page 22: Trends Affecting System Architectures: Mobile …...Ka;a Streams Akka Streams … Sessions Streams Storage Device 1 Telemetry 2. → Model Training 3. ← New models 2, 3 4. ← Model](https://reader036.fdocuments.in/reader036/viewer/2022062505/5ece3dbe3111ee591f38650b/html5/thumbnails/22.jpg)
DataCenter
Mini-batch,Batch
Spark
LowLatencyMicroservices
Ka;aStreamsAkkaStreams
…
Sessions
Streams
Storage
Device
Telemetry1
2. → Model Training3. ← New models
2, 3
4. ← Model Storage5. → Boot up, historical data
4, 5
6. → Data Pipeline 7. → Model Serving 8. ← Anomalies
6, 7, 8Ingest Scores9
Corrective Action
10
Spark
MicroserviceMicroserviceMicroservice
DeviceSessionMicroservices
Persistence
Ka+a Cluster
Broker
@deanwampler
Ingestthetelemetrydatatoscoreit,lookingforanomalies
![Page 23: Trends Affecting System Architectures: Mobile …...Ka;a Streams Akka Streams … Sessions Streams Storage Device 1 Telemetry 2. → Model Training 3. ← New models 2, 3 4. ← Model](https://reader036.fdocuments.in/reader036/viewer/2022062505/5ece3dbe3111ee591f38650b/html5/thumbnails/23.jpg)
DataCenter
Mini-batch,Batch
Spark
LowLatencyMicroservices
Ka;aStreamsAkkaStreams
…
Sessions
Streams
Storage
Device
Telemetry1
2. → Model Training3. ← New models
2, 3
4. ← Model Storage5. → Boot up, historical data
4, 5
6. → Data Pipeline 7. → Model Serving 8. ← Anomalies
6, 7, 8Ingest Scores9
Corrective Action
10
Spark
MicroserviceMicroserviceMicroservice
DeviceSessionMicroservices
Persistence
Ka+a Cluster
Broker
@deanwampler
Smalldatavolume,Lowlatency(milliseconds-…)
Ingestmodelparameterstoupdatemodelinthelow-latencymicroservicesformodelserving
![Page 24: Trends Affecting System Architectures: Mobile …...Ka;a Streams Akka Streams … Sessions Streams Storage Device 1 Telemetry 2. → Model Training 3. ← New models 2, 3 4. ← Model](https://reader036.fdocuments.in/reader036/viewer/2022062505/5ece3dbe3111ee591f38650b/html5/thumbnails/24.jpg)
DataCenter
Mini-batch,Batch
Spark
LowLatencyMicroservices
Ka;aStreamsAkkaStreams
…
Sessions
Streams
Storage
Device
Telemetry1
2. → Model Training3. ← New models
2, 3
4. ← Model Storage5. → Boot up, historical data
4, 5
6. → Data Pipeline 7. → Model Serving 8. ← Anomalies
6, 7, 8Ingest Scores9
Corrective Action
10
Spark
MicroserviceMicroserviceMicroservice
DeviceSessionMicroservices
Persistence
Ka+a Cluster
Broker
@deanwampler
WritethedetectedanomaliesbacktoKafkainanewtopic
![Page 25: Trends Affecting System Architectures: Mobile …...Ka;a Streams Akka Streams … Sessions Streams Storage Device 1 Telemetry 2. → Model Training 3. ← New models 2, 3 4. ← Model](https://reader036.fdocuments.in/reader036/viewer/2022062505/5ece3dbe3111ee591f38650b/html5/thumbnails/25.jpg)
DataCenter
Mini-batch,Batch
Spark
LowLatencyMicroservices
Ka;aStreamsAkkaStreams
…
Sessions
Streams
Storage
Device
Telemetry1
2. → Model Training3. ← New models
2, 3
4. ← Model Storage5. → Boot up, historical data
4, 5
6. → Data Pipeline 7. → Model Serving 8. ← Anomalies
6, 7, 8Ingest Scores9
Corrective Action
10
Spark
MicroserviceMicroserviceMicroservice
DeviceSessionMicroservices
Persistence
Ka+a Cluster
Broker
@deanwampler
Readanomalyinformationintomicroservicesthatmanagethedevicesremotely
![Page 26: Trends Affecting System Architectures: Mobile …...Ka;a Streams Akka Streams … Sessions Streams Storage Device 1 Telemetry 2. → Model Training 3. ← New models 2, 3 4. ← Model](https://reader036.fdocuments.in/reader036/viewer/2022062505/5ece3dbe3111ee591f38650b/html5/thumbnails/26.jpg)
DataCenter
Mini-batch,Batch
Spark
LowLatencyMicroservices
Ka;aStreamsAkkaStreams
…
Sessions
Streams
Storage
Device
Telemetry1
2. → Model Training3. ← New models
2, 3
4. ← Model Storage5. → Boot up, historical data
4, 5
6. → Data Pipeline 7. → Model Serving 8. ← Anomalies
6, 7, 8Ingest Scores9
Corrective Action
10
Spark
MicroserviceMicroserviceMicroservice
DeviceSessionMicroservices
Persistence
Ka+a Cluster
Broker
@deanwampler
Takecorrectiveaction,e.g.,uploadmoreinformation,downloadpatches,control-alt-delete,…
![Page 27: Trends Affecting System Architectures: Mobile …...Ka;a Streams Akka Streams … Sessions Streams Storage Device 1 Telemetry 2. → Model Training 3. ← New models 2, 3 4. ← Model](https://reader036.fdocuments.in/reader036/viewer/2022062505/5ece3dbe3111ee591f38650b/html5/thumbnails/27.jpg)
DataCenter
Mini-batch,Batch
Spark
LowLatencyMicroservices
Ka;aStreamsAkkaStreams
…
Sessions
Streams
Storage
Device
Telemetry1
2. → Model Training3. ← New models
2, 3
4. ← Model Storage5. → Boot up, historical data
4, 5
6. → Data Pipeline 7. → Model Serving 8. ← Anomalies
6, 7, 8Ingest Scores9
Corrective Action
10
Spark
MicroserviceMicroserviceMicroservice
DeviceSessionMicroservices
Persistence
Ka+a Cluster
Broker
@deanwampler
Sessionmanagement,RESTmicroservices Model
Scoring
ModelTraining
Threegroupsoffunctionality
![Page 28: Trends Affecting System Architectures: Mobile …...Ka;a Streams Akka Streams … Sessions Streams Storage Device 1 Telemetry 2. → Model Training 3. ← New models 2, 3 4. ← Model](https://reader036.fdocuments.in/reader036/viewer/2022062505/5ece3dbe3111ee591f38650b/html5/thumbnails/28.jpg)
@deanwampler
Streaming Architectures: An Example
Variation: Model Serving as a Service
![Page 29: Trends Affecting System Architectures: Mobile …...Ka;a Streams Akka Streams … Sessions Streams Storage Device 1 Telemetry 2. → Model Training 3. ← New models 2, 3 4. ← Model](https://reader036.fdocuments.in/reader036/viewer/2022062505/5ece3dbe3111ee591f38650b/html5/thumbnails/29.jpg)
DataCenter
Mini-batch,Batch
Spark
LowLatencyMicroservices
Ka;aStreamsAkkaStreams
…
Sessions
Streams
Storage
Device
Telemetry1
2. → Model Training3. ← New models
2, 3
4. ← Model Storage5. → Boot up, historical data
4, 5
6. → Data Pipeline 7. → Model Serving 8. ← Anomalies
6, 7, 8Ingest Scores9
Corrective Action
10
Spark
MicroserviceMicroserviceMicroservice
DeviceSessionMicroservices
Persistence
Ka+a Cluster
Broker
@deanwampler
From this…
![Page 30: Trends Affecting System Architectures: Mobile …...Ka;a Streams Akka Streams … Sessions Streams Storage Device 1 Telemetry 2. → Model Training 3. ← New models 2, 3 4. ← Model](https://reader036.fdocuments.in/reader036/viewer/2022062505/5ece3dbe3111ee591f38650b/html5/thumbnails/30.jpg)
@deanwampler
DataCenter
REST,gRPC,…
Spark
LowLatency
Ka9aStreamsAkkaStreams
…
Sessions
Streams
Storage
Device
Telemetry1
2. → Model Training
3
3. ← Model Storage4. → Boot up, historical data
3, 4
5. → Data Pipeline 7. ← Anomalies
5, 7Ingest Scores8
Corrective Action
9
TensorFlow,…
MicroserviceMicroserviceMicroservice
DeviceSessionMicroservices
Persistence
Ka+a Cluster
Broker 66. Model Serving
Bothtrainingandservingdonebyaseparateservice,likeTensorFlow
To this…
![Page 31: Trends Affecting System Architectures: Mobile …...Ka;a Streams Akka Streams … Sessions Streams Storage Device 1 Telemetry 2. → Model Training 3. ← New models 2, 3 4. ← Model](https://reader036.fdocuments.in/reader036/viewer/2022062505/5ece3dbe3111ee591f38650b/html5/thumbnails/31.jpg)
@deanwampler
Streaming Architectures: An Example
Variation: Model Serving on the Edge
![Page 32: Trends Affecting System Architectures: Mobile …...Ka;a Streams Akka Streams … Sessions Streams Storage Device 1 Telemetry 2. → Model Training 3. ← New models 2, 3 4. ← Model](https://reader036.fdocuments.in/reader036/viewer/2022062505/5ece3dbe3111ee591f38650b/html5/thumbnails/32.jpg)
DataCenter
Mini-batch,Batch
Spark
LowLatencyMicroservices
Ka;aStreamsAkkaStreams
…
Sessions
Streams
Storage
Device
Telemetry1
2. → Model Training3. ← New models
2, 3
4. ← Model Storage5. → Boot up, historical data
4, 5
6. → Data Pipeline 7. → Model Serving 8. ← Anomalies
6, 7, 8Ingest Scores9
Corrective Action
10
Spark
MicroserviceMicroserviceMicroservice
DeviceSessionMicroservices
Persistence
Ka+a Cluster
Broker
@deanwampler
From this…
![Page 33: Trends Affecting System Architectures: Mobile …...Ka;a Streams Akka Streams … Sessions Streams Storage Device 1 Telemetry 2. → Model Training 3. ← New models 2, 3 4. ← Model](https://reader036.fdocuments.in/reader036/viewer/2022062505/5ece3dbe3111ee591f38650b/html5/thumbnails/33.jpg)
DataCenter
Mini-batch,Batch
SparkSessions
Streams
Storage
Device
Telemetry1
2. → Model Training3. ← New models
2, 3
4. ← Model Storage5. → Boot up, historical data
4, 5
Corrective Action
8
Spark
MicroserviceMicroserviceMicroservice
DeviceSessionMicroservices
Persistence
Ka+a Cluster
Model Parameters6Model Parameters7
Broker
@deanwampler
Pushmodelupdatestothedeviceandscorethere.Eliminatesround-tripoverheadforscoring.
To this…
![Page 34: Trends Affecting System Architectures: Mobile …...Ka;a Streams Akka Streams … Sessions Streams Storage Device 1 Telemetry 2. → Model Training 3. ← New models 2, 3 4. ← Model](https://reader036.fdocuments.in/reader036/viewer/2022062505/5ece3dbe3111ee591f38650b/html5/thumbnails/34.jpg)
@deanwampler
Mixing data science and data engineering
![Page 35: Trends Affecting System Architectures: Mobile …...Ka;a Streams Akka Streams … Sessions Streams Storage Device 1 Telemetry 2. → Model Training 3. ← New models 2, 3 4. ← Model](https://reader036.fdocuments.in/reader036/viewer/2022062505/5ece3dbe3111ee591f38650b/html5/thumbnails/35.jpg)
@deanwampler
SoftwareEngineering toolbox
Data Science toolbox
Tools
![Page 36: Trends Affecting System Architectures: Mobile …...Ka;a Streams Akka Streams … Sessions Streams Storage Device 1 Telemetry 2. → Model Training 3. ← New models 2, 3 4. ← Model](https://reader036.fdocuments.in/reader036/viewer/2022062505/5ece3dbe3111ee591f38650b/html5/thumbnails/36.jpg)
@deanwampler
Data Science:• Less process oriented• Iterative, experimental• … but still within the
Scientific Method
Data Engineering:• Process oriented• Agile Manifesto• … which does not
mention data!
Process
https://derwen.ai/s/6fqt
![Page 37: Trends Affecting System Architectures: Mobile …...Ka;a Streams Akka Streams … Sessions Streams Storage Device 1 Telemetry 2. → Model Training 3. ← New models 2, 3 4. ← Model](https://reader036.fdocuments.in/reader036/viewer/2022062505/5ece3dbe3111ee591f38650b/html5/thumbnails/37.jpg)
@deanwampler
Data Science:• Comfortable with
uncertainty• Probabilities and
Statistics
Data Engineering:• Uncomfortable with
uncertainty• Prefer determinism• … while admitting that
distributed systems aren’t deterministic
Philosophy
![Page 38: Trends Affecting System Architectures: Mobile …...Ka;a Streams Akka Streams … Sessions Streams Storage Device 1 Telemetry 2. → Model Training 3. ← New models 2, 3 4. ← Model](https://reader036.fdocuments.in/reader036/viewer/2022062505/5ece3dbe3111ee591f38650b/html5/thumbnails/38.jpg)
@deanwampler
Other Production Concerns
![Page 39: Trends Affecting System Architectures: Mobile …...Ka;a Streams Akka Streams … Sessions Streams Storage Device 1 Telemetry 2. → Model Training 3. ← New models 2, 3 4. ← Model](https://reader036.fdocuments.in/reader036/viewer/2022062505/5ece3dbe3111ee591f38650b/html5/thumbnails/39.jpg)
@deanwampler
Models Are
Data
![Page 40: Trends Affecting System Architectures: Mobile …...Ka;a Streams Akka Streams … Sessions Streams Storage Device 1 Telemetry 2. → Model Training 3. ← New models 2, 3 4. ← Model](https://reader036.fdocuments.in/reader036/viewer/2022062505/5ece3dbe3111ee591f38650b/html5/thumbnails/40.jpg)
@deanwampler
Auditing• Kind of Model• Parameters and hyperparameters•When trained• Data used for training•When deployed, undeployed, etc.•…
![Page 41: Trends Affecting System Architectures: Mobile …...Ka;a Streams Akka Streams … Sessions Streams Storage Device 1 Telemetry 2. → Model Training 3. ← New models 2, 3 4. ← Model](https://reader036.fdocuments.in/reader036/viewer/2022062505/5ece3dbe3111ee591f38650b/html5/thumbnails/41.jpg)
@deanwampler
Auditing• Quality metrics• Serving metrics (how many records, scoring
times…)• Provenance of decision to retrain• The metrics gathered above that were used to
decide when to retrain
![Page 42: Trends Affecting System Architectures: Mobile …...Ka;a Streams Akka Streams … Sessions Streams Storage Device 1 Telemetry 2. → Model Training 3. ← New models 2, 3 4. ← Model](https://reader036.fdocuments.in/reader036/viewer/2022062505/5ece3dbe3111ee591f38650b/html5/thumbnails/42.jpg)
@deanwampler
Auditing•… and maybe most important:• What model was used to score a particular
record??
![Page 43: Trends Affecting System Architectures: Mobile …...Ka;a Streams Akka Streams … Sessions Streams Storage Device 1 Telemetry 2. → Model Training 3. ← New models 2, 3 4. ← Model](https://reader036.fdocuments.in/reader036/viewer/2022062505/5ece3dbe3111ee591f38650b/html5/thumbnails/43.jpg)
@deanwampler
Model Updates
Concept Drift
Models have a half life, too
![Page 44: Trends Affecting System Architectures: Mobile …...Ka;a Streams Akka Streams … Sessions Streams Storage Device 1 Telemetry 2. → Model Training 3. ← New models 2, 3 4. ← Model](https://reader036.fdocuments.in/reader036/viewer/2022062505/5ece3dbe3111ee591f38650b/html5/thumbnails/44.jpg)
@deanwampler
DataCenter
Mini-batch,Batch
Spark
LowLatencyMicroservices
Ka;aStreamsAkkaStreams
…
Sessions
Streams
Storage
Device
Telemetry1
2. → Model Training3. ← New models
2, 3
4. ← Model Storage5. → Boot up, historical data
4, 5
6. → Data Pipeline 7. → Model Serving 8. ← Anomalies
6, 7, 8Ingest Scores9
Corrective Action
10
Spark
MicroserviceMicroserviceMicroservice
DeviceSessionMicroservices
Persistence
Ka+a Cluster
Broker
Model Updates
![Page 45: Trends Affecting System Architectures: Mobile …...Ka;a Streams Akka Streams … Sessions Streams Storage Device 1 Telemetry 2. → Model Training 3. ← New models 2, 3 4. ← Model](https://reader036.fdocuments.in/reader036/viewer/2022062505/5ece3dbe3111ee591f38650b/html5/thumbnails/45.jpg)
Recap
![Page 46: Trends Affecting System Architectures: Mobile …...Ka;a Streams Akka Streams … Sessions Streams Storage Device 1 Telemetry 2. → Model Training 3. ← New models 2, 3 4. ← Model](https://reader036.fdocuments.in/reader036/viewer/2022062505/5ece3dbe3111ee591f38650b/html5/thumbnails/46.jpg)
Information value has a half life;
it decays with time
![Page 47: Trends Affecting System Architectures: Mobile …...Ka;a Streams Akka Streams … Sessions Streams Storage Device 1 Telemetry 2. → Model Training 3. ← New models 2, 3 4. ← Model](https://reader036.fdocuments.in/reader036/viewer/2022062505/5ece3dbe3111ee591f38650b/html5/thumbnails/47.jpg)
Streaming systems have similar requirements as
microservices
DataCenter
Mini-batch,Batch
Spark
LowLatencyMicroservices
Ka;aStreamsAkkaStreams
…
Sessions
Streams
Storage
Device
Telemetry1
2. → Model Training3. ← New models
2, 3
4. ← Model Storage5. → Boot up, historical data
4, 5
6. → Data Pipeline 7. → Model Serving 8. ← Anomalies
6, 7, 8Ingest Scores9
Corrective Action
10
Spark
MicroserviceMicroserviceMicroservice
DeviceSessionMicroservices
Persistence
Ka+a Cluster
Broker
![Page 48: Trends Affecting System Architectures: Mobile …...Ka;a Streams Akka Streams … Sessions Streams Storage Device 1 Telemetry 2. → Model Training 3. ← New models 2, 3 4. ← Model](https://reader036.fdocuments.in/reader036/viewer/2022062505/5ece3dbe3111ee591f38650b/html5/thumbnails/48.jpg)
Integratingdata science anddata engineering
is hard
![Page 49: Trends Affecting System Architectures: Mobile …...Ka;a Streams Akka Streams … Sessions Streams Storage Device 1 Telemetry 2. → Model Training 3. ← New models 2, 3 4. ← Model](https://reader036.fdocuments.in/reader036/viewer/2022062505/5ece3dbe3111ee591f38650b/html5/thumbnails/49.jpg)
Models also have ahalf life;
They have to be updated periodically
![Page 50: Trends Affecting System Architectures: Mobile …...Ka;a Streams Akka Streams … Sessions Streams Storage Device 1 Telemetry 2. → Model Training 3. ← New models 2, 3 4. ← Model](https://reader036.fdocuments.in/reader036/viewer/2022062505/5ece3dbe3111ee591f38650b/html5/thumbnails/50.jpg)
@deanwamplerMarslastSummer
DustyMilkyWay
![Page 51: Trends Affecting System Architectures: Mobile …...Ka;a Streams Akka Streams … Sessions Streams Storage Device 1 Telemetry 2. → Model Training 3. ← New models 2, 3 4. ← Model](https://reader036.fdocuments.in/reader036/viewer/2022062505/5ece3dbe3111ee591f38650b/html5/thumbnails/51.jpg)
References• Ideas:• O’Reilly Radar: Data, AI, others• distill.pub• The Algorithm• The Gradient
@deanwampler
![Page 52: Trends Affecting System Architectures: Mobile …...Ka;a Streams Akka Streams … Sessions Streams Storage Device 1 Telemetry 2. → Model Training 3. ← New models 2, 3 4. ← Model](https://reader036.fdocuments.in/reader036/viewer/2022062505/5ece3dbe3111ee591f38650b/html5/thumbnails/52.jpg)
References• General Information about Stream Processing• My O'Reilly Report on Architectures• Streaming Systems Book• Stream Processing with Apache Spark• Designing Data-Intensive Apps book
@deanwampler
![Page 53: Trends Affecting System Architectures: Mobile …...Ka;a Streams Akka Streams … Sessions Streams Storage Device 1 Telemetry 2. → Model Training 3. ← New models 2, 3 4. ← Model](https://reader036.fdocuments.in/reader036/viewer/2022062505/5ece3dbe3111ee591f38650b/html5/thumbnails/53.jpg)
References• Other Talks• Strata Talk on ML in a Streaming Context• Stream All the Things! (video)• Streaming Microservices with Akka Streams
and Kafka Streams (video)
@deanwampler
![Page 54: Trends Affecting System Architectures: Mobile …...Ka;a Streams Akka Streams … Sessions Streams Storage Device 1 Telemetry 2. → Model Training 3. ← New models 2, 3 4. ← Model](https://reader036.fdocuments.in/reader036/viewer/2022062505/5ece3dbe3111ee591f38650b/html5/thumbnails/54.jpg)
References• Tutorials• Model serving in streams• Stream processing with Kafka and
microservices
@deanwampler
![Page 55: Trends Affecting System Architectures: Mobile …...Ka;a Streams Akka Streams … Sessions Streams Storage Device 1 Telemetry 2. → Model Training 3. ← New models 2, 3 4. ← Model](https://reader036.fdocuments.in/reader036/viewer/2022062505/5ece3dbe3111ee591f38650b/html5/thumbnails/55.jpg)
But first, introductions…
What we’re up to at Lightbend… lightbend.com/lightbend-pipelines-demo
Buy my stuff!!
![Page 56: Trends Affecting System Architectures: Mobile …...Ka;a Streams Akka Streams … Sessions Streams Storage Device 1 Telemetry 2. → Model Training 3. ← New models 2, 3 4. ← Model](https://reader036.fdocuments.in/reader036/viewer/2022062505/5ece3dbe3111ee591f38650b/html5/thumbnails/56.jpg)
@deanwampler
Dean Wampler, Ph.D. [email protected]
@deanwampler lightbend.com/lightbend-platform
polyglotprogramming.com/talks
Questions?