50 Shades of Data - how, when and why Big,Relational,NoSQL,Elastic,Event,CQRS (Devoxx Maroc,...
-
Upload
lucas-jellema -
Category
Software
-
view
708 -
download
1
Transcript of 50 Shades of Data - how, when and why Big,Relational,NoSQL,Elastic,Event,CQRS (Devoxx Maroc,...
![Page 1: 50 Shades of Data - how, when and why Big,Relational,NoSQL,Elastic,Event,CQRS (Devoxx Maroc, November 2017) - including detailed demo screenshots](https://reader034.fdocuments.in/reader034/viewer/2022051710/5a6489fc7f8b9a27568b5f8b/html5/thumbnails/1.jpg)
50 Shades of Data - how, when and why Big, Fast, Relational, NoSQL, Elastic, Event, CQRS
Lucas Jellema
AMIS (The Netherlands)
@lucasjellema
technology.amis.nl
GitHub:
https://github.com/lucasjellema/order-data-demo-devoxx
#DevoxxMA
![Page 2: 50 Shades of Data - how, when and why Big,Relational,NoSQL,Elastic,Event,CQRS (Devoxx Maroc, November 2017) - including detailed demo screenshots](https://reader034.fdocuments.in/reader034/viewer/2022051710/5a6489fc7f8b9a27568b5f8b/html5/thumbnails/2.jpg)
Who am I?• From The Netherlands, father of two sons
#devoxxma
![Page 3: 50 Shades of Data - how, when and why Big,Relational,NoSQL,Elastic,Event,CQRS (Devoxx Maroc, November 2017) - including detailed demo screenshots](https://reader034.fdocuments.in/reader034/viewer/2022051710/5a6489fc7f8b9a27568b5f8b/html5/thumbnails/3.jpg)
Felicitations!
#devoxxma
![Page 4: 50 Shades of Data - how, when and why Big,Relational,NoSQL,Elastic,Event,CQRS (Devoxx Maroc, November 2017) - including detailed demo screenshots](https://reader034.fdocuments.in/reader034/viewer/2022051710/5a6489fc7f8b9a27568b5f8b/html5/thumbnails/4.jpg)
Who am I?• From The Netherlands, father of two sons
• Masters in Applied Physics
• Started in IT in 1994: Oracle; now CTO of AMIS
• Solution Architect for enterprise IT challenges
• Oracle ACE Director, Oracle Developer Champion, Java Rockstar
• Presenter: Oracle OpenWorld, JavaOne, NLJUG JFall/JSpring, Javapolis/Devoxx, YouTube
• Author of two books on Oracle SOA Suite,1400 blog articles and 7000+ Tweets
#devoxxma
![Page 5: 50 Shades of Data - how, when and why Big,Relational,NoSQL,Elastic,Event,CQRS (Devoxx Maroc, November 2017) - including detailed demo screenshots](https://reader034.fdocuments.in/reader034/viewer/2022051710/5a6489fc7f8b9a27568b5f8b/html5/thumbnails/5.jpg)
What is data?
#devoxxma
![Page 6: 50 Shades of Data - how, when and why Big,Relational,NoSQL,Elastic,Event,CQRS (Devoxx Maroc, November 2017) - including detailed demo screenshots](https://reader034.fdocuments.in/reader034/viewer/2022051710/5a6489fc7f8b9a27568b5f8b/html5/thumbnails/6.jpg)
What is data?• A solidified representation of
• An observation [of a fact]• A concept
• Serialized in order to be• Understood & processed by machines• Reproduced for human consumption
#devoxxma
![Page 7: 50 Shades of Data - how, when and why Big,Relational,NoSQL,Elastic,Event,CQRS (Devoxx Maroc, November 2017) - including detailed demo screenshots](https://reader034.fdocuments.in/reader034/viewer/2022051710/5a6489fc7f8b9a27568b5f8b/html5/thumbnails/7.jpg)
When things weresimple
#devoxxma
RDBMSSQLACID
Data files
Log Files
BackupBackupBackup
SAN
![Page 8: 50 Shades of Data - how, when and why Big,Relational,NoSQL,Elastic,Event,CQRS (Devoxx Maroc, November 2017) - including detailed demo screenshots](https://reader034.fdocuments.in/reader034/viewer/2022051710/5a6489fc7f8b9a27568b5f8b/html5/thumbnails/8.jpg)
And then stuffhappened
#devoxxma
Middle Tier:Java EE (Stateful) application
Client Tier: BrowserClient Tier:
Browser
Client Tier: Browser
Mobile App (offline)
Mobile App (offline)
Mobile App (offline)
Data Warehouse
XML, JSON
Content Management
Big Data
Fast Data
API
APIAPI
µ λ
![Page 9: 50 Shades of Data - how, when and why Big,Relational,NoSQL,Elastic,Event,CQRS (Devoxx Maroc, November 2017) - including detailed demo screenshots](https://reader034.fdocuments.in/reader034/viewer/2022051710/5a6489fc7f8b9a27568b5f8b/html5/thumbnails/9.jpg)
Explosion of Data Store technologies
RDBMSSQLACID
![Page 10: 50 Shades of Data - how, when and why Big,Relational,NoSQL,Elastic,Event,CQRS (Devoxx Maroc, November 2017) - including detailed demo screenshots](https://reader034.fdocuments.in/reader034/viewer/2022051710/5a6489fc7f8b9a27568b5f8b/html5/thumbnails/10.jpg)
V4
#devoxxma
![Page 11: 50 Shades of Data - how, when and why Big,Relational,NoSQL,Elastic,Event,CQRS (Devoxx Maroc, November 2017) - including detailed demo screenshots](https://reader034.fdocuments.in/reader034/viewer/2022051710/5a6489fc7f8b9a27568b5f8b/html5/thumbnails/11.jpg)
Tagcloud
#devoxxma
![Page 12: 50 Shades of Data - how, when and why Big,Relational,NoSQL,Elastic,Event,CQRS (Devoxx Maroc, November 2017) - including detailed demo screenshots](https://reader034.fdocuments.in/reader034/viewer/2022051710/5a6489fc7f8b9a27568b5f8b/html5/thumbnails/12.jpg)
Business Areas
Marketing & Campaigns
External Actors
SupplierGov Agency
ShippingSecurity
FinanceAccounts, Invoices
Supplier & Product Management
Inventory & Warehousing
Output (print & mail, email,
SMS, …)
Sales & Customer Service
Inside the Enterprise
Data DepartmentConsolidation, MI, Reporting,
Analysis and R&D
Customer Management
Order Management
Data providers
Customers
![Page 13: 50 Shades of Data - how, when and why Big,Relational,NoSQL,Elastic,Event,CQRS (Devoxx Maroc, November 2017) - including detailed demo screenshots](https://reader034.fdocuments.in/reader034/viewer/2022051710/5a6489fc7f8b9a27568b5f8b/html5/thumbnails/13.jpg)
Marketing & Campaigns
Public Internet/External Actors
Gov Agency
ShippingSecurity
FinanceAccounts, Invoices
Supplier & Product Management
Inventory & Warehousing
Output (print & mail, email,
SMS, …)
Inside the Enterprise
Data DepartmentConsolidation, MI, Reporting,
Analysis and R&D
Customer Management
Order Management
Data providers
SupplierCustomers
B2B Partner PortalCustomer
Service SaaS
Mobile App
Custom Application for Product Catalog
IoT Gateways & Hub
SaaS ERP
Enterprise Content Management System
Human Workflow Engine
Mail Server
Data Warehouse
SaaS CRM
Custom Order Management Application
B2B APIs
B2B APIs
Open Data APIs
DaaSServices
APIs
SaaS CXCampaigns, Social Media Monitor, 360 Customer
View
LDAP for Users, Roles & Permissions
WebShopPortal
Recommendation Engine
Enterprise Dashboard & BI &
Reporting
Security & Compliance
Monitor
Desktop Tools
Communication & Collaboration tools
Asset Tracker
Business Applications
![Page 14: 50 Shades of Data - how, when and why Big,Relational,NoSQL,Elastic,Event,CQRS (Devoxx Maroc, November 2017) - including detailed demo screenshots](https://reader034.fdocuments.in/reader034/viewer/2022051710/5a6489fc7f8b9a27568b5f8b/html5/thumbnails/14.jpg)
Marketing & Campaigns
Public Internet/External Actors
Gov Agency
ShippingSecurity
FinanceAccounts, Invoices
Supplier & Product Management
Inventory & Warehousing
Output (print & mail, email,
SMS, …)
Inside the Enterprise
Data DepartmentConsolidation, MI, Reporting,
Analysis and R&D
Customer Management
Order Management
Data providers
SupplierCustomers
B2B Partner PortalCustomer
Service SaaS
Mobile App
Big Data Lake
Custom Application for Product Catalog
IoT Gateways & Hub
SaaS ERP
Enterprise Content Management System
Human Workflow Engine
Mail Server
Data Warehouse
SaaS CRM
DaaSServices
SaaS CXCampaigns, Social Media Monitor, 360 Customer
View
LDAP for Users, Roles & Permissions
WebShopPortal
Recommendation Engine
Enterprise Dashboard & BI &
Reporting
Security & Compliance
Monitor
Desktop Tools
Communication & Collaboration tools
Logging Collector & Monitor &
Analyzer
Monitor for Application & Infra
metricsSource Code
Control System
API Gateway
Service Bus
Event Bus
Event Bus
Rule Engine
Desktop Browser
Mobile Devices
Email / Facebook / WhatsApp
Business Applications & IT Systems
Custom Order Management Application
Asset Tracker
Corporate DatabaseFile
Storage
Job Scheduling
B2B APIs
Open Data APIs
APIs
Application Server
Private Blockchain
B2B APIs
Docker Container Registry
![Page 15: 50 Shades of Data - how, when and why Big,Relational,NoSQL,Elastic,Event,CQRS (Devoxx Maroc, November 2017) - including detailed demo screenshots](https://reader034.fdocuments.in/reader034/viewer/2022051710/5a6489fc7f8b9a27568b5f8b/html5/thumbnails/15.jpg)
Marketing & Campaigns
Public Internet/External Actors
Gov Agency
ShippingSecurity
FinanceAccounts, Invoices
Supplier & Product Management
Inventory & Warehousing
Output (print & mail, email,
SMS, …)
Inside the Enterprise
Data DepartmentConsolidation, MI, Reporting,
Analysis and R&D
Customer Management
Order Management
Data providers
SupplierCustomers
B2B Partner PortalCustomer
Service SaaS
Mobile App
Big Data Lake
Custom Application for Product Catalog
IoT Gateways & Hub
SaaS ERP
Enterprise Content Management System
Human Workflow Engine
Mail Server
Data Warehouse
SaaS CRM
DaaSServices
SaaS CXCampaigns, Social Media Monitor, 360 Customer
View
LDAP for Users, Roles & Permissions
WebShopPortal
Recommendation Engine
Enterprise Dashboard & BI &
Reporting
Security & Compliance
Monitor
Desktop Tools
Communication & Collaboration tools
Logging Collector & Monitor &
Analyzer
Monitor for Application & Infra
metricsSource Code
Control System
API Gateway
Service Bus
Event Bus
Event Bus
Rule Engine
Desktop Browser
Mobile Devices
Email / Facebook / WhatsApp
Custom Order Management Application
Asset Tracker
Corporate DatabaseFile
Storage
Job Scheduling
B2B APIs
Open Data APIs
APIs
Application Server
Private Blockchain
B2B APIs
Business & IT - Data List of Products
shown in UIPersonal Profile,
Order and Payments Details
Smart Contracts with supply chain
details
Recent Consumer purchases
informationFootage from
security cameras
Readings from motion
detectors
Emails regarding customer
complaints
Spreadsheets with Sales records
Log-files from IT systems (infra &
platform)
WebShop activity, Social Media
discussions, …
ML Models In Flight
Messages
Events
Job Schedules
Application & Infrastructure source history
Offers, invoices, rewards
messages
Shopping Cart with selected
items
Order Details
API usage, billing, policies
Running & Past workflow instances
Sales Aggregates by Day, Region,
Product Category
Invoices & Payments
Product Manuals
Digital Twin
KPIs & Alerts
Customer Interaction records
Case files (Complaints,
Requests)
Rules & Rule Execution
metrics
Weather, Demographics,
Sports, Social, …
Configdata
Customer Details
Audit Trails, Security Incidents
ML Models
Programming in progress
User Stories, Designs, Discussions
Copy of Production Data
in Acceptance
![Page 16: 50 Shades of Data - how, when and why Big,Relational,NoSQL,Elastic,Event,CQRS (Devoxx Maroc, November 2017) - including detailed demo screenshots](https://reader034.fdocuments.in/reader034/viewer/2022051710/5a6489fc7f8b9a27568b5f8b/html5/thumbnails/16.jpg)
Volume
#devoxxma
![Page 17: 50 Shades of Data - how, when and why Big,Relational,NoSQL,Elastic,Event,CQRS (Devoxx Maroc, November 2017) - including detailed demo screenshots](https://reader034.fdocuments.in/reader034/viewer/2022051710/5a6489fc7f8b9a27568b5f8b/html5/thumbnails/17.jpg)
Marketing & Campaigns
Public Internet/External Actors
Gov Agency
ShippingSecurity
FinanceAccounts, Invoices
Supplier & Product Management
Inventory & Warehousing
Output (print & mail, email,
SMS, …)
Inside the Enterprise
Data DepartmentConsolidation, MI, Reporting,
Analysis and R&D
Customer Management
Order Management
Data providers
SupplierCustomers
B2B Partner PortalCustomer
Service SaaS
Mobile App
Big Data Lake
Custom Application for Product Catalog
IoT Gateways & Hub
SaaS ERP
Enterprise Content Management System
Human Workflow Engine
Mail Server
Data Warehouse
SaaS CRM
DaaSServices
SaaS CXCampaigns, Social Media Monitor, 360 Customer
View
LDAP for Users, Roles & Permissions
WebShopPortal
Recommendation Engine
Enterprise Dashboard & BI &
Reporting
Security & Compliance
Monitor
Desktop Tools
Communication & Collaboration tools
Logging Collector & Monitor &
Analyzer
Monitor for Application & Infra
metricsSource Code
Control System
API Gateway
Service Bus
Event Bus
Event Bus
Rule Engine
Desktop Browser
Mobile Devices
Email / Facebook / WhatsApp
Custom Order Management Application
Asset Tracker
Corporate DatabaseFile
Storage
Job Scheduling
B2B APIs
Open Data APIs
APIs
Application Server
Private Blockchain
B2B APIs
Data VolumePersonal Profile,
Order and Payments Details
Smart Contracts with supply chain
details
Recent Consumer purchases
informationFootage from
security cameras
Emails regarding customer
complaints
Log-files from IT systems (infra &
platform)
ML Models
Job Schedules
Offers, invoices, rewards
messages
Order Details
API usage, billing, policies
Running & Past workflow instances
Invoices & Payments
Product Manuals
Digital Twin
KPIs & Alerts
Customer Interaction records
Case files (Complaints,
Requests)
Rules & Rule Execution
metrics
Weather, Demographics,
Sports, Social, …
Configdata
Customer Details
ML Models
Programming in progress
User Stories, Designs, Discussions
low
Big Data Lake
Machine Learning models
Long term history Data Warehouse
Big Lots of dataSmall chunks of off line data
Piles of log-files
Fine grained events
Gathering –never purging?
Small payloads
Medium size –structured data
Rule meta-data (very small)
![Page 18: 50 Shades of Data - how, when and why Big,Relational,NoSQL,Elastic,Event,CQRS (Devoxx Maroc, November 2017) - including detailed demo screenshots](https://reader034.fdocuments.in/reader034/viewer/2022051710/5a6489fc7f8b9a27568b5f8b/html5/thumbnails/18.jpg)
Compression• . Technical Compression
• Same data, fewer bits to store
• Same time – or even longer - to process
• Logical Compression• Filter (older than, one in X)
• Reduce fine grainedness - helicopterview• Average over geographical area
• Min/Max/Average per minute/hour/day
• Is typically done in data warehouse & digital twin
• Could be done for query stores and even for big data set
#devoxxma
![Page 19: 50 Shades of Data - how, when and why Big,Relational,NoSQL,Elastic,Event,CQRS (Devoxx Maroc, November 2017) - including detailed demo screenshots](https://reader034.fdocuments.in/reader034/viewer/2022051710/5a6489fc7f8b9a27568b5f8b/html5/thumbnails/19.jpg)
80M Pictures of Road
#devoxxma
![Page 20: 50 Shades of Data - how, when and why Big,Relational,NoSQL,Elastic,Event,CQRS (Devoxx Maroc, November 2017) - including detailed demo screenshots](https://reader034.fdocuments.in/reader034/viewer/2022051710/5a6489fc7f8b9a27568b5f8b/html5/thumbnails/20.jpg)
Big Data => Small ML Models
#devoxxma
![Page 21: 50 Shades of Data - how, when and why Big,Relational,NoSQL,Elastic,Event,CQRS (Devoxx Maroc, November 2017) - including detailed demo screenshots](https://reader034.fdocuments.in/reader034/viewer/2022051710/5a6489fc7f8b9a27568b5f8b/html5/thumbnails/21.jpg)
Velocity
#devoxxma
![Page 22: 50 Shades of Data - how, when and why Big,Relational,NoSQL,Elastic,Event,CQRS (Devoxx Maroc, November 2017) - including detailed demo screenshots](https://reader034.fdocuments.in/reader034/viewer/2022051710/5a6489fc7f8b9a27568b5f8b/html5/thumbnails/22.jpg)
![Page 23: 50 Shades of Data - how, when and why Big,Relational,NoSQL,Elastic,Event,CQRS (Devoxx Maroc, November 2017) - including detailed demo screenshots](https://reader034.fdocuments.in/reader034/viewer/2022051710/5a6489fc7f8b9a27568b5f8b/html5/thumbnails/23.jpg)
Fast Data – Fast Insight
#devoxxma
Raw Data
Event Hub
Streaming withHot (Alerting)
and ColdIoT
Device Data Digital Twin
Machine LearningModels to apply todigital twin to predictmaintenance need
![Page 24: 50 Shades of Data - how, when and why Big,Relational,NoSQL,Elastic,Event,CQRS (Devoxx Maroc, November 2017) - including detailed demo screenshots](https://reader034.fdocuments.in/reader034/viewer/2022051710/5a6489fc7f8b9a27568b5f8b/html5/thumbnails/24.jpg)
Marketing & Campaigns
Public Internet/External Actors
Gov Agency
ShippingSecurity
FinanceAccounts, Invoices
Supplier & Product Management
Inventory & Warehousing
Output (print & mail, email,
SMS, …)
Inside the Enterprise
Data DepartmentConsolidation, MI, Reporting,
Analysis and R&D
Customer Management
Order Management
Data providers
SupplierCustomers
B2B Partner PortalCustomer
Service SaaS
Mobile App
Big Data Lake
Custom Application for Product Catalog
IoT Gateways & Hub
SaaS ERP
Enterprise Content Management System
Human Workflow Engine
Mail Server
Data Warehouse
SaaS CRM
DaaSServices
SaaS CXCampaigns, Social Media Monitor, 360 Customer
View
LDAP for Users, Roles & Permissions
WebShopPortal
Recommendation Engine
Enterprise Dashboard & BI &
Reporting
Security & Compliance
Monitor
Desktop Tools
Communication & Collaboration tools
Logging Collector & Monitor &
Analyzer
Monitor for Application & Infra
metricsSource Code
Control System
API Gateway
Service Bus
Event Bus
Event Bus
Rule Engine
Desktop Browser
Mobile Devices
Email / Facebook / WhatsApp
Custom Order Management Application
Asset Tracker
Corporate DatabaseFile
Storage
Job Scheduling
B2B APIs
Open Data APIs
APIs
Application Server
Private Blockchain
B2B APIs
Data Volatility List of Products shown in UI
Personal Profile, Order and
Payments Details
Smart Contracts with supply chain
details
Recent Consumer purchases
informationFootage from
security cameras
Emails regarding customer
complaints
Spreadsheets with Sales records
Log-files from IT systems (infra &
platform)
WebShopactivity, Social
Media discussions, …
ML Models
In Flight Messages
Events
Job Schedules
Application & Infrastructure source history
Offers, invoices, rewards
messages
Shopping Cart with selected
items
Order Details
API usage, billing, policies
Running & Past workflow instances
Invoices & Payments
Product Manuals
Digital Twin
KPIs & Alerts
Customer Interaction records
Case files (Complaints,
Requests)
Rules & Rule Execution
metrics
Weather, Demographics,
Sports, Social, …
Configdata
Customer Details
Audit Trails, Security Incidents
ML Models
Programming in progress
User Stories, Designs, Discussions
high low
Readings from motion
detectors
Sales Aggregates by Day, Region, Product
Category
![Page 25: 50 Shades of Data - how, when and why Big,Relational,NoSQL,Elastic,Event,CQRS (Devoxx Maroc, November 2017) - including detailed demo screenshots](https://reader034.fdocuments.in/reader034/viewer/2022051710/5a6489fc7f8b9a27568b5f8b/html5/thumbnails/25.jpg)
Demo – Cache
#devoxxma
Cache
Application Container Cache
APIUI
Microservice ShoppingCart& Check Out
![Page 26: 50 Shades of Data - how, when and why Big,Relational,NoSQL,Elastic,Event,CQRS (Devoxx Maroc, November 2017) - including detailed demo screenshots](https://reader034.fdocuments.in/reader034/viewer/2022051710/5a6489fc7f8b9a27568b5f8b/html5/thumbnails/26.jpg)
Oracle CloudMicroservice & Cache
#devoxxma
Cache
Application Container Cache
APIUI
Microservice ShoppingCart& Check Out
![Page 27: 50 Shades of Data - how, when and why Big,Relational,NoSQL,Elastic,Event,CQRS (Devoxx Maroc, November 2017) - including detailed demo screenshots](https://reader034.fdocuments.in/reader034/viewer/2022051710/5a6489fc7f8b9a27568b5f8b/html5/thumbnails/27.jpg)
Location
#devoxxma
![Page 28: 50 Shades of Data - how, when and why Big,Relational,NoSQL,Elastic,Event,CQRS (Devoxx Maroc, November 2017) - including detailed demo screenshots](https://reader034.fdocuments.in/reader034/viewer/2022051710/5a6489fc7f8b9a27568b5f8b/html5/thumbnails/28.jpg)
Location of Data
#devoxxma
![Page 29: 50 Shades of Data - how, when and why Big,Relational,NoSQL,Elastic,Event,CQRS (Devoxx Maroc, November 2017) - including detailed demo screenshots](https://reader034.fdocuments.in/reader034/viewer/2022051710/5a6489fc7f8b9a27568b5f8b/html5/thumbnails/29.jpg)
Location of Data
#devoxxma
![Page 30: 50 Shades of Data - how, when and why Big,Relational,NoSQL,Elastic,Event,CQRS (Devoxx Maroc, November 2017) - including detailed demo screenshots](https://reader034.fdocuments.in/reader034/viewer/2022051710/5a6489fc7f8b9a27568b5f8b/html5/thumbnails/30.jpg)
Marketing & Campaigns
Public Internet/External Actors
Gov Agency
ShippingSecurity
FinanceAccounts, Invoices
Supplier & Product Management
Inventory & Warehousing
Output (print & mail, email,
SMS, …)
Inside the Enterprise
Data DepartmentConsolidation, MI, Reporting,
Analysis and R&D
Customer Management
Order Management
Data providers
SupplierCustomers
B2B Partner PortalCustomer
Service SaaS
Mobile App
Big Data Lake
Custom Application for Product Catalog
IoT Gateways & Hub
SaaS ERP
Enterprise Content Management System
Human Workflow Engine
Mail Server
Data Warehouse
SaaS CRM
DaaSServices
SaaS CXCampaigns, Social Media Monitor, 360 Customer
View
LDAP for Users, Roles & Permissions
WebShopPortal
Recommendation Engine
Enterprise Dashboard & BI &
Reporting
Security & Compliance
Monitor
Desktop Tools
Communication & Collaboration tools
Logging Collector & Monitor &
Analyzer
Monitor for Application & Infra
metricsSource Code
Control System
API Gateway
Service Bus
Event Bus
Event Bus
Rule Engine
Desktop Browser
Mobile Devices
Email / Facebook / WhatsApp
Custom Order Management Application
Asset Tracker
Corporate DatabaseFile
Storage
Job Scheduling
B2B APIs
Open Data APIs
APIs
Application Server
Private Blockchain
B2B APIs
Data LocationPersonal Profile,
Order and Payments Details
Smart Contracts with supply chain
details
Recent Consumer purchases
informationFootage from
security cameras
Emails regarding customer
complaints
Log-files from IT systems (infra &
platform)
ML Models
Global Content Delivery Network
Job Schedules
Offers, invoices, rewards
messages
Order Details
API usage, billing, policies
Running & Past workflow instances
Invoices & Payments
Product Manuals
Digital Twin
KPIs & Alerts
Customer Interaction records
Case files (Complaints,
Requests)
Rules & Rule Execution
metrics
Weather, Demographics,
Sports, Social, …
Configdata
Customer Details
ML Models
Programming in progress
User Stories, Designs, Discussions
low Offline Storage in Apps
Third party (SaaS) Git repo
Offsite Standby for Disaster Recovery
SaaS data store in Cloud
DaaS data store in Cloud
Application Server Memory (on site)
Excel Sheets on employee laptops
Local storage on “Things” & Edge devices
Cloud storage for Database backups
Local Database Instance for each region
![Page 31: 50 Shades of Data - how, when and why Big,Relational,NoSQL,Elastic,Event,CQRS (Devoxx Maroc, November 2017) - including detailed demo screenshots](https://reader034.fdocuments.in/reader034/viewer/2022051710/5a6489fc7f8b9a27568b5f8b/html5/thumbnails/31.jpg)
Considerations aroundLocation
• Latency• Latency experienced by end-user is sum of latencies in the chain
• Co-located – systems with chatty interaction• Storage cost• Network Transport costs• Ease of distribution
• Background distribution may be acceptable – provided it happens frequently enough
• Off line usage• Security• Data “en route”
#devoxxma
![Page 32: 50 Shades of Data - how, when and why Big,Relational,NoSQL,Elastic,Event,CQRS (Devoxx Maroc, November 2017) - including detailed demo screenshots](https://reader034.fdocuments.in/reader034/viewer/2022051710/5a6489fc7f8b9a27568b5f8b/html5/thumbnails/32.jpg)
On the move
#devoxxma
![Page 33: 50 Shades of Data - how, when and why Big,Relational,NoSQL,Elastic,Event,CQRS (Devoxx Maroc, November 2017) - including detailed demo screenshots](https://reader034.fdocuments.in/reader034/viewer/2022051710/5a6489fc7f8b9a27568b5f8b/html5/thumbnails/33.jpg)
Marketing & Campaigns
Public Internet/External Actors
Gov Agency
ShippingSecurity
FinanceAccounts, Invoices
Supplier & Product Management
Inventory & Warehousing
Output (print & mail, email,
SMS, …)
Inside the Enterprise
Data DepartmentConsolidation, MI, Reporting,
Analysis and R&D
Customer Management
Order Management
Data providers
SupplierCustomers
B2B Partner PortalCustomer
Service SaaS
Mobile App
Big Data Lake
Custom Application for Product Catalog
IoT Gateways & Hub
SaaS ERP
Enterprise Content Management System
Human Workflow Engine
Mail Server
Data Warehouse
SaaS CRM
DaaSServices
SaaS CXCampaigns, Social Media Monitor, 360 Customer
View
LDAP for Users, Roles & Permissions
WebShopPortal
Recommendation Engine
Enterprise Dashboard & BI &
Reporting
Security & Compliance
Monitor
Desktop Tools
Communication & Collaboration tools
Logging Collector & Monitor &
Analyzer
Monitor for Application & Infra
metricsSource Code
Control System
API Gateway
Service Bus
Event Bus
Event Bus
Rule Engine
Desktop Browser
Mobile Devices
Email / Facebook / WhatsApp
Custom Order Management Application
Asset Tracker
Corporate DatabaseFile
Storage
Job Scheduling
B2B APIs
Open Data APIs
APIs
Application Server
Private Blockchain
B2B APIs
StreamingPersonal Profile,
Order and Payments Details
Smart Contracts with supply chain
details
Recent Consumer purchases
informationFootage from
security cameras
Emails regarding customer
complaints
Log-files from IT systems (infra &
platform)
ML Models
Job Schedules
Offers, invoices, rewards
messages
Order Details
API usage, billing, policies
Running & Past workflow instances
Invoices & Payments
Product Manuals
Digital Twin
KPIs & Alerts
Customer Interaction records
Case files (Complaints,
Requests)
Rules & Rule Execution
metrics
Weather, Demographics,
Sports, Social, …
Configdata
Customer Details
ML Models
Programming in progress
User Stories, Designs, Discussions
low Synchronization of Devices coming online again
Upload of ML Models
Replaying transaction on standby database
Applications being deployed
Update of Datawarehouse
Laptops & USB sticks on the move
Raw IoT => Streaming Analysis => {alerts | digital
twin | big data}
Customer sending complaint by email
Synchronization of SaaS from On Premises
Metrics from Apps | Platform | Infra to Log
Stash & Monitor
Events moving to consumers
UI updates pushed to browser
Task notification sent to employee
Fresh Data pushed to Application Cache
Database Backup moved offsite
![Page 34: 50 Shades of Data - how, when and why Big,Relational,NoSQL,Elastic,Event,CQRS (Devoxx Maroc, November 2017) - including detailed demo screenshots](https://reader034.fdocuments.in/reader034/viewer/2022051710/5a6489fc7f8b9a27568b5f8b/html5/thumbnails/34.jpg)
Cost
#devoxxma
![Page 35: 50 Shades of Data - how, when and why Big,Relational,NoSQL,Elastic,Event,CQRS (Devoxx Maroc, November 2017) - including detailed demo screenshots](https://reader034.fdocuments.in/reader034/viewer/2022051710/5a6489fc7f8b9a27568b5f8b/html5/thumbnails/35.jpg)
TC(D)O –Total Cost of Data Ownership• Business cost (missed opportunity, user dissatisfaction, …) of
not having the data available • at all or fast enough or fresh enough
#devoxxma
SpeedFreshnessAvailable
ComputeStorageNetwork
![Page 36: 50 Shades of Data - how, when and why Big,Relational,NoSQL,Elastic,Event,CQRS (Devoxx Maroc, November 2017) - including detailed demo screenshots](https://reader034.fdocuments.in/reader034/viewer/2022051710/5a6489fc7f8b9a27568b5f8b/html5/thumbnails/36.jpg)
TC(D)O –Total Cost of Data Ownership• Direct cost of
• Acquiring data
• Storing Data• Storage (cheap and slow, expensive and quick)
• Compression (less storage at expense of compute)
• Retrieving Data• Compute resources
• Calculating & Deriving data (DWH, ML Model, CQRS)• Compute resources
• Transporting Data• Network traffic has a price tag
#devoxxma
![Page 37: 50 Shades of Data - how, when and why Big,Relational,NoSQL,Elastic,Event,CQRS (Devoxx Maroc, November 2017) - including detailed demo screenshots](https://reader034.fdocuments.in/reader034/viewer/2022051710/5a6489fc7f8b9a27568b5f8b/html5/thumbnails/37.jpg)
TC(D)O –Total Cost of Data Ownership• Operational costs
• Backup & Recovery
• Security
• Intellectual property
• Life cycle management – slower tier, archive, purge
• “Right to be forgotten”
• Regulatory periods to hang on to data
#devoxxma
![Page 38: 50 Shades of Data - how, when and why Big,Relational,NoSQL,Elastic,Event,CQRS (Devoxx Maroc, November 2017) - including detailed demo screenshots](https://reader034.fdocuments.in/reader034/viewer/2022051710/5a6489fc7f8b9a27568b5f8b/html5/thumbnails/38.jpg)
Open (APIs) & DaaS• Governments and NGOs, scientific
and even commercial organizationsare publishing data
• Inviting anyone who wants to join in to help make sense of the data – understand driving factors, identify categories, help predict
• Many areas• Economy, health, public safety, sports,
traffic & transportation, games,environment, maps, …
#devoxxma
![Page 39: 50 Shades of Data - how, when and why Big,Relational,NoSQL,Elastic,Event,CQRS (Devoxx Maroc, November 2017) - including detailed demo screenshots](https://reader034.fdocuments.in/reader034/viewer/2022051710/5a6489fc7f8b9a27568b5f8b/html5/thumbnails/39.jpg)
Live
#devoxxma
![Page 40: 50 Shades of Data - how, when and why Big,Relational,NoSQL,Elastic,Event,CQRS (Devoxx Maroc, November 2017) - including detailed demo screenshots](https://reader034.fdocuments.in/reader034/viewer/2022051710/5a6489fc7f8b9a27568b5f8b/html5/thumbnails/40.jpg)
Real Life
#devoxxma
Background Batch Process (preparing letters for
customers)
Customers
BPM EngineProcessing cases for
Customers
![Page 41: 50 Shades of Data - how, when and why Big,Relational,NoSQL,Elastic,Event,CQRS (Devoxx Maroc, November 2017) - including detailed demo screenshots](https://reader034.fdocuments.in/reader034/viewer/2022051710/5a6489fc7f8b9a27568b5f8b/html5/thumbnails/41.jpg)
Stale
#devoxxma
![Page 42: 50 Shades of Data - how, when and why Big,Relational,NoSQL,Elastic,Event,CQRS (Devoxx Maroc, November 2017) - including detailed demo screenshots](https://reader034.fdocuments.in/reader034/viewer/2022051710/5a6489fc7f8b9a27568b5f8b/html5/thumbnails/42.jpg)
Stale• Data is a representation of the real world
• All data is inherently stale• Except when it describes something that can not change – and whose
description can not change
• Staleness is probably not a problem• Except in self driving cars…
• Run the end-of-year-report
• Consistency is much more important
#devoxxma
![Page 43: 50 Shades of Data - how, when and why Big,Relational,NoSQL,Elastic,Event,CQRS (Devoxx Maroc, November 2017) - including detailed demo screenshots](https://reader034.fdocuments.in/reader034/viewer/2022051710/5a6489fc7f8b9a27568b5f8b/html5/thumbnails/43.jpg)
Glimpses of the past
#devoxxma
Session 1 Session 2
![Page 44: 50 Shades of Data - how, when and why Big,Relational,NoSQL,Elastic,Event,CQRS (Devoxx Maroc, November 2017) - including detailed demo screenshots](https://reader034.fdocuments.in/reader034/viewer/2022051710/5a6489fc7f8b9a27568b5f8b/html5/thumbnails/44.jpg)
Glimpses of the past
#devoxxma
Session 1 Session 2
![Page 45: 50 Shades of Data - how, when and why Big,Relational,NoSQL,Elastic,Event,CQRS (Devoxx Maroc, November 2017) - including detailed demo screenshots](https://reader034.fdocuments.in/reader034/viewer/2022051710/5a6489fc7f8b9a27568b5f8b/html5/thumbnails/45.jpg)
Flashback to the Past
#devoxxma
![Page 46: 50 Shades of Data - how, when and why Big,Relational,NoSQL,Elastic,Event,CQRS (Devoxx Maroc, November 2017) - including detailed demo screenshots](https://reader034.fdocuments.in/reader034/viewer/2022051710/5a6489fc7f8b9a27568b5f8b/html5/thumbnails/46.jpg)
Powered by Undo
#devoxxma
UNDO
![Page 47: 50 Shades of Data - how, when and why Big,Relational,NoSQL,Elastic,Event,CQRS (Devoxx Maroc, November 2017) - including detailed demo screenshots](https://reader034.fdocuments.in/reader034/viewer/2022051710/5a6489fc7f8b9a27568b5f8b/html5/thumbnails/47.jpg)
Integrity
#devoxxma
![Page 48: 50 Shades of Data - how, when and why Big,Relational,NoSQL,Elastic,Event,CQRS (Devoxx Maroc, November 2017) - including detailed demo screenshots](https://reader034.fdocuments.in/reader034/viewer/2022051710/5a6489fc7f8b9a27568b5f8b/html5/thumbnails/48.jpg)
Data Constraintsto protect integrity
• Allowable values
• Mandatory attributes
• (Foreign Key) References
• NULL
• Constraints on• type• length• format• Spelling• Character encoding
#devoxxma
![Page 49: 50 Shades of Data - how, when and why Big,Relational,NoSQL,Elastic,Event,CQRS (Devoxx Maroc, November 2017) - including detailed demo screenshots](https://reader034.fdocuments.in/reader034/viewer/2022051710/5a6489fc7f8b9a27568b5f8b/html5/thumbnails/49.jpg)
Data is representation of the known real world
• How useful is it to enforce data integrity?
#devoxxma
![Page 50: 50 Shades of Data - how, when and why Big,Relational,NoSQL,Elastic,Event,CQRS (Devoxx Maroc, November 2017) - including detailed demo screenshots](https://reader034.fdocuments.in/reader034/viewer/2022051710/5a6489fc7f8b9a27568b5f8b/html5/thumbnails/50.jpg)
Virtual Reality
#devoxxma
![Page 51: 50 Shades of Data - how, when and why Big,Relational,NoSQL,Elastic,Event,CQRS (Devoxx Maroc, November 2017) - including detailed demo screenshots](https://reader034.fdocuments.in/reader034/viewer/2022051710/5a6489fc7f8b9a27568b5f8b/html5/thumbnails/51.jpg)
Data Integrity• Why?
• Is it about truth?
• About regulations and by-the-book?
• Allow IT systems to run smoothly and not get confused?
• About auditability and non-repudiation?
• What about the real world?• Data in IT is just a representation;
if the world is not by the book – what should IT do?
#devoxxma
![Page 52: 50 Shades of Data - how, when and why Big,Relational,NoSQL,Elastic,Event,CQRS (Devoxx Maroc, November 2017) - including detailed demo screenshots](https://reader034.fdocuments.in/reader034/viewer/2022051710/5a6489fc7f8b9a27568b5f8b/html5/thumbnails/52.jpg)
Blockchain• Distributed
• Across trusted business partners
• Across public, anonymous parties
• Immutable
• Secured
• Smart Contracts• Operations on data (without human intervention)
#devoxxma
![Page 53: 50 Shades of Data - how, when and why Big,Relational,NoSQL,Elastic,Event,CQRS (Devoxx Maroc, November 2017) - including detailed demo screenshots](https://reader034.fdocuments.in/reader034/viewer/2022051710/5a6489fc7f8b9a27568b5f8b/html5/thumbnails/53.jpg)
Format &Technology
#devoxxma
![Page 54: 50 Shades of Data - how, when and why Big,Relational,NoSQL,Elastic,Event,CQRS (Devoxx Maroc, November 2017) - including detailed demo screenshots](https://reader034.fdocuments.in/reader034/viewer/2022051710/5a6489fc7f8b9a27568b5f8b/html5/thumbnails/54.jpg)
Graph Database• Natural fit during development
• Superior (10-1000 times better) performance
#devoxxma
![Page 55: 50 Shades of Data - how, when and why Big,Relational,NoSQL,Elastic,Event,CQRS (Devoxx Maroc, November 2017) - including detailed demo screenshots](https://reader034.fdocuments.in/reader034/viewer/2022051710/5a6489fc7f8b9a27568b5f8b/html5/thumbnails/55.jpg)
From relational SQL to Graph query
#devoxxma
![Page 56: 50 Shades of Data - how, when and why Big,Relational,NoSQL,Elastic,Event,CQRS (Devoxx Maroc, November 2017) - including detailed demo screenshots](https://reader034.fdocuments.in/reader034/viewer/2022051710/5a6489fc7f8b9a27568b5f8b/html5/thumbnails/56.jpg)
SQL vs NoSQL
#devoxxma
![Page 57: 50 Shades of Data - how, when and why Big,Relational,NoSQL,Elastic,Event,CQRS (Devoxx Maroc, November 2017) - including detailed demo screenshots](https://reader034.fdocuments.in/reader034/viewer/2022051710/5a6489fc7f8b9a27568b5f8b/html5/thumbnails/57.jpg)
SQL vs NoSQLACID vs BASERelational vs …
#devoxxma
![Page 58: 50 Shades of Data - how, when and why Big,Relational,NoSQL,Elastic,Event,CQRS (Devoxx Maroc, November 2017) - including detailed demo screenshots](https://reader034.fdocuments.in/reader034/viewer/2022051710/5a6489fc7f8b9a27568b5f8b/html5/thumbnails/58.jpg)
SQL is not good at anything
• But it sucks at nothing
#devoxxma
![Page 59: 50 Shades of Data - how, when and why Big,Relational,NoSQL,Elastic,Event,CQRS (Devoxx Maroc, November 2017) - including detailed demo screenshots](https://reader034.fdocuments.in/reader034/viewer/2022051710/5a6489fc7f8b9a27568b5f8b/html5/thumbnails/59.jpg)
Relational Databases• Based on relational model of data (E.F. Codd), a mathematical
foundation
• Uses SQL for query, DML and DDL
• Transactions are ACID (Atomicity, Consistency, Isolation, Durability)• All or nothing• Constraint Compliant• Individual experience
[in a multi-session environment] (aka concurrency)
• Down does not hurt
#devoxxma
![Page 60: 50 Shades of Data - how, when and why Big,Relational,NoSQL,Elastic,Event,CQRS (Devoxx Maroc, November 2017) - including detailed demo screenshots](https://reader034.fdocuments.in/reader034/viewer/2022051710/5a6489fc7f8b9a27568b5f8b/html5/thumbnails/60.jpg)
ACID comes at a cost• Transaction results have to be persisted
in order to guarantee D
• Concurrency requires some degree of locking (and multi-versioning) in order to have I
• Constraint compliance (unique key, foreign key) means all data hangs together (as do all transactions) in order to have C
• Two-phase commit further introducescomplexity, dependencies and delays
#devoxxma
![Page 61: 50 Shades of Data - how, when and why Big,Relational,NoSQL,Elastic,Event,CQRS (Devoxx Maroc, November 2017) - including detailed demo screenshots](https://reader034.fdocuments.in/reader034/viewer/2022051710/5a6489fc7f8b9a27568b5f8b/html5/thumbnails/61.jpg)
The holy grail of Normalization
• Normalize to prevent• data redundancy
• discrepancies (split brain)
• storage waste
• However: we shouldrecognize the fact thatsome data is read far more frequently than thatit is created and modified
#devoxxma
![Page 62: 50 Shades of Data - how, when and why Big,Relational,NoSQL,Elastic,Event,CQRS (Devoxx Maroc, November 2017) - including detailed demo screenshots](https://reader034.fdocuments.in/reader034/viewer/2022051710/5a6489fc7f8b9a27568b5f8b/html5/thumbnails/62.jpg)
The Relational Model in practice
• Traditional Relational Data Model has severe impact on physical disk performance• Transaction Log => Sequential Write (append to file) • Data Blocks require much more expensive Random Access disk writes
• Indexes (B-Tree, Bitmap, …) are used to speed up query (read) performance• and slow down transactions
• Relational data does not [always] map naturally to the data format required in the application (OO, JSON, XML)
• Capability to join and construct ad-hoc queries across the entire data model is powerful
• Declarative integrity constraints allow for strict enforcement of data quality rules• “the data may be non sensical, but at least it adheres to the rules”
#devoxxma
![Page 63: 50 Shades of Data - how, when and why Big,Relational,NoSQL,Elastic,Event,CQRS (Devoxx Maroc, November 2017) - including detailed demo screenshots](https://reader034.fdocuments.in/reader034/viewer/2022051710/5a6489fc7f8b9a27568b5f8b/html5/thumbnails/63.jpg)
Databases re-evaluated• Not all use cases require ACID (or can afford it)
• Read only (product catalog for web shops)• Inserts only and no (inter-record) constraints• Big Data collected and “dumped” in Data Lake (Hadoop) for subsequent
processing• High performance demands
• Not all data needs structured formats or structured querying andJOINs• Entire documents are stored and retrieved based on a single key
• Sometimes – scalable availability and productivity is more important than Consistency – and ACID is sacrificed• CAP-theorem states: Consistency [across nodes], Availability and Partition
tolerance can not all three be satisfied
#devoxxma
![Page 64: 50 Shades of Data - how, when and why Big,Relational,NoSQL,Elastic,Event,CQRS (Devoxx Maroc, November 2017) - including detailed demo screenshots](https://reader034.fdocuments.in/reader034/viewer/2022051710/5a6489fc7f8b9a27568b5f8b/html5/thumbnails/64.jpg)
NoSQL and BASE• NoSQL arose because of performance and scalability
challenges with Web Scale operations
• NoSQL is a label for a wide variety of databases that lack someaspect of a true relational database• ACID-ness, SQL, relational model, constraints
• The label has been used since 2009• Perhaps NoREL would be more appropriate
• Some well known NoSQL products are• Cassandra, MongoDB, Redis, CouchDB, …
• BASE as alternative to ACID:• basically available, soft state, eventually consistent
(after a short duration)
#devoxxma
![Page 65: 50 Shades of Data - how, when and why Big,Relational,NoSQL,Elastic,Event,CQRS (Devoxx Maroc, November 2017) - including detailed demo screenshots](https://reader034.fdocuments.in/reader034/viewer/2022051710/5a6489fc7f8b9a27568b5f8b/html5/thumbnails/65.jpg)
Typical for NoSQL• Focus on speed, availability and scalability
• Horizontal scale out – distributed with load balancing and fail-over
• No (predefined) Data Structure• Integrity primarily protected by application logic• Open Source (most offerings are, not all: MarkLogic)• Close(r) attention for how the data is used
• Application oriented data format and search paths and specialized database per application (microservice, capability)
• Similar to the switch from SOA to API/Microservice
• Reads (far) more relevant than writes• Data redundancy & denormalization• No data access through SQL – well, …
#devoxxma
![Page 66: 50 Shades of Data - how, when and why Big,Relational,NoSQL,Elastic,Event,CQRS (Devoxx Maroc, November 2017) - including detailed demo screenshots](https://reader034.fdocuments.in/reader034/viewer/2022051710/5a6489fc7f8b9a27568b5f8b/html5/thumbnails/66.jpg)
Types of NoSQL
#devoxxma
![Page 67: 50 Shades of Data - how, when and why Big,Relational,NoSQL,Elastic,Event,CQRS (Devoxx Maroc, November 2017) - including detailed demo screenshots](https://reader034.fdocuments.in/reader034/viewer/2022051710/5a6489fc7f8b9a27568b5f8b/html5/thumbnails/67.jpg)
(leading) NoSQL Database products
• MongoDB is (one of) the most popular (by any measure)
• Cloud (only): • Google BigTable, • AWS Dynamo
• Cache (in memory)• ZooKeeper, Redis,
Memcached, …
• Hadoop/HDFS
• Oracle NoSQL(fka Berkeley DB)
#devoxxma
![Page 68: 50 Shades of Data - how, when and why Big,Relational,NoSQL,Elastic,Event,CQRS (Devoxx Maroc, November 2017) - including detailed demo screenshots](https://reader034.fdocuments.in/reader034/viewer/2022051710/5a6489fc7f8b9a27568b5f8b/html5/thumbnails/68.jpg)
NoSQL means: No Data Access through SQL
• However• Data Professionals and
Developers speak SQL
• Reporting, Dashboarding,ETL, BI tools speak SQL
• There is no common querylanguage across NoSQLproducts
#devoxxma
![Page 69: 50 Shades of Data - how, when and why Big,Relational,NoSQL,Elastic,Event,CQRS (Devoxx Maroc, November 2017) - including detailed demo screenshots](https://reader034.fdocuments.in/reader034/viewer/2022051710/5a6489fc7f8b9a27568b5f8b/html5/thumbnails/69.jpg)
No Data Access through SQL• However
• Data Professionals andDevelopers speak SQL
• Reporting, Dashboarding,ETL, BI tools speak SQL
• There is no common querylanguage across NoSQLproducts
• Attempts from many vendors to create drivers that translate SQL statements into NoSQL commands for the specific target database• To protect existing investments in SQL – skills, tools, applications, reports, ..
#devoxxma
![Page 70: 50 Shades of Data - how, when and why Big,Relational,NoSQL,Elastic,Event,CQRS (Devoxx Maroc, November 2017) - including detailed demo screenshots](https://reader034.fdocuments.in/reader034/viewer/2022051710/5a6489fc7f8b9a27568b5f8b/html5/thumbnails/70.jpg)
SQL vs NoSQL• SQL != RDBMS
• SQL on top of• Hadoop – Spark SQL, Hive, Drill, Impala
• “External Table” Text files, CSV, Excel
• XML, JSON
• KSQL on Kafka events
• Google Spanner, BigQuery
• NoSQL – Berkeley DB, Hbase, Elastic Search, MongoDB, Cassandra
#devoxxma
![Page 71: 50 Shades of Data - how, when and why Big,Relational,NoSQL,Elastic,Event,CQRS (Devoxx Maroc, November 2017) - including detailed demo screenshots](https://reader034.fdocuments.in/reader034/viewer/2022051710/5a6489fc7f8b9a27568b5f8b/html5/thumbnails/71.jpg)
NoSQL (MongoDB) vs SQL (Oracle)
#devoxxma
db.emp.find
( {"JOB":"SALESMAN"}
, { ENAME:1
, SAL:1}
)
.sort
( {'SAL':-1})
.limit(2)
select ename
, sal
from emp
where job = 'SALESMAN'
order
by sal desc
FETCH FIRST 2 ROWS ONLY
![Page 72: 50 Shades of Data - how, when and why Big,Relational,NoSQL,Elastic,Event,CQRS (Devoxx Maroc, November 2017) - including detailed demo screenshots](https://reader034.fdocuments.in/reader034/viewer/2022051710/5a6489fc7f8b9a27568b5f8b/html5/thumbnails/72.jpg)
NoSQL (MongoDB) vs SQL (Oracle)
#devoxxma
db.emp.find
( {"JOB":"SALESMAN"
, $where :
" this.SAL +
(this.COMM != null?
this.COMM: 0)
> 2000"
}
)
select *
from emp
where sal + nvl(comm, 0)
> 2000
![Page 73: 50 Shades of Data - how, when and why Big,Relational,NoSQL,Elastic,Event,CQRS (Devoxx Maroc, November 2017) - including detailed demo screenshots](https://reader034.fdocuments.in/reader034/viewer/2022051710/5a6489fc7f8b9a27568b5f8b/html5/thumbnails/73.jpg)
Distributed
#devoxxma
![Page 74: 50 Shades of Data - how, when and why Big,Relational,NoSQL,Elastic,Event,CQRS (Devoxx Maroc, November 2017) - including detailed demo screenshots](https://reader034.fdocuments.in/reader034/viewer/2022051710/5a6489fc7f8b9a27568b5f8b/html5/thumbnails/74.jpg)
Why distributed?• Because it is
• Business is physically spread out over multiple locations
• To achieve• Scalability
• Performance (parallelism, latency)
• Resilience of the whole – availability (in the face of individual failure)
• (site) Disaster recovery
• Trust (e.g. blockchain)
• Applies to data & processes
#devoxxma
![Page 75: 50 Shades of Data - how, when and why Big,Relational,NoSQL,Elastic,Event,CQRS (Devoxx Maroc, November 2017) - including detailed demo screenshots](https://reader034.fdocuments.in/reader034/viewer/2022051710/5a6489fc7f8b9a27568b5f8b/html5/thumbnails/75.jpg)
Marketing & Campaigns
Public Internet/External Actors
Gov Agency
ShippingSecurity
FinanceAccounts, Invoices
Supplier & Product Management
Inventory & Warehousing
Output (print & mail, email,
SMS, …)
Inside the Enterprise
Data DepartmentConsolidation, MI, Reporting,
Analysis and R&D
Customer Management
Order Management
Data providers
SupplierCustomers
B2B Partner PortalCustomer
Service SaaS
Mobile App
Big Data Lake
Custom Application for Product Catalog
IoT Gateways & Hub
SaaS ERP
Enterprise Content Management System
Human Workflow Engine
Mail Server
Data Warehouse
SaaS CRM
DaaSServices
SaaS CXCampaigns, Social Media Monitor, 360 Customer
View
LDAP for Users, Roles & Permissions
WebShopPortal
Recommendation Engine
Enterprise Dashboard & BI &
Reporting
Security & Compliance
Monitor
Desktop Tools
Communication & Collaboration tools
Logging Collector & Monitor &
Analyzer
Monitor for Application & Infra
metricsSource Code
Control System
API Gateway
Service Bus
Event Bus
Event Bus
Rule Engine
Desktop Browser
Mobile Devices
Email / Facebook / WhatsApp
Custom Order Management Application
Asset Tracker
Corporate DatabaseFile
Storage
Job Scheduling
B2B APIs
Open Data APIs
APIs
Application Server
Private Blockchain
B2B APIs
DistributedPersonal Profile,
Order and Payments Details
Smart Contracts with supply chain
details
Recent Consumer purchases
informationFootage from
security cameras
Emails regarding customer
complaints
Log-files from IT systems (infra &
platform)
ML Models
Global Content Delivery Network
Job Schedules
Offers, invoices, rewards
messages
Order Details
API usage, billing, policies
Running & Past workflow instances
Invoices & Payments
Product Manuals
Digital Twin
KPIs & Alerts
Customer Interaction records
Case files (Complaints,
Requests)
Rules & Rule Execution
metrics
Weather, Demographics,
Sports, Social, …
Configdata
Customer Details
ML Models
Programming in progress
User Stories, Designs, Discussions
low Offline Storage in Apps
Real Application Clusters
Distributed In Memory Cache Hazelcast, MemCached, Redis, Coherence
Java EE Application Server Cluster
SETI
Local storage on “Things” & Edge devices
Active Standby Database
SAN
Cross Cloud/On Premises archive
Distributed Datastore MongoDB, Cassandra, BigTable, HBase
Apache Spark Distributed Data Processing
Logical Data Shards in Oracle Database, MySQL, Elastic
HDFS Hadoop Distributed File System
Kubernetes Distributed Container Platform
Distributed Event Bus: Kafka
Multiple sites around the globe with local …
![Page 76: 50 Shades of Data - how, when and why Big,Relational,NoSQL,Elastic,Event,CQRS (Devoxx Maroc, November 2017) - including detailed demo screenshots](https://reader034.fdocuments.in/reader034/viewer/2022051710/5a6489fc7f8b9a27568b5f8b/html5/thumbnails/76.jpg)
Vertically Distributed Data
#devoxxma
Client Tier: Browser
DOM/UI
MVVM
Middle Tier:Java EE (Stateful) application
APIAPI
APIStateless
![Page 77: 50 Shades of Data - how, when and why Big,Relational,NoSQL,Elastic,Event,CQRS (Devoxx Maroc, November 2017) - including detailed demo screenshots](https://reader034.fdocuments.in/reader034/viewer/2022051710/5a6489fc7f8b9a27568b5f8b/html5/thumbnails/77.jpg)
Vertically Distributed Data
#devoxxma
Client Tier: Browser
DOM/UI
MVVM
Middle Tier:Java EE (Stateful) application
APIAPI
APIStateless
![Page 78: 50 Shades of Data - how, when and why Big,Relational,NoSQL,Elastic,Event,CQRS (Devoxx Maroc, November 2017) - including detailed demo screenshots](https://reader034.fdocuments.in/reader034/viewer/2022051710/5a6489fc7f8b9a27568b5f8b/html5/thumbnails/78.jpg)
Availability
#devoxxma
![Page 79: 50 Shades of Data - how, when and why Big,Relational,NoSQL,Elastic,Event,CQRS (Devoxx Maroc, November 2017) - including detailed demo screenshots](https://reader034.fdocuments.in/reader034/viewer/2022051710/5a6489fc7f8b9a27568b5f8b/html5/thumbnails/79.jpg)
Marketing & Campaigns
Public Internet/External Actors
Gov Agency
ShippingSecurity
FinanceAccounts, Invoices
Supplier & Product Management
Inventory & Warehousing
Output (print & mail, email,
SMS, …)
Inside the Enterprise
Data DepartmentConsolidation, MI, Reporting,
Analysis and R&D
Customer Management
Order Management
Data providers
SupplierCustomers
B2B Partner PortalCustomer
Service SaaS
Mobile App
Big Data Lake
Custom Application for Product Catalog
IoT Gateways & Hub
SaaS ERP
Enterprise Content Management System
Human Workflow Engine
Mail Server
Data Warehouse
SaaS CRM
DaaSServices
SaaS CXCampaigns, Social Media Monitor, 360 Customer
View
LDAP for Users, Roles & Permissions
WebShopPortal
Recommendation Engine
Enterprise Dashboard & BI &
Reporting
Security & Compliance
Monitor
Desktop Tools
Communication & Collaboration tools
Logging Collector & Monitor &
Analyzer
Monitor for Application & Infra
metricsSource Code
Control System
API Gateway
Service Bus
Event Bus
Event Bus
Rule Engine
Desktop Browser
Mobile Devices
Email / Facebook / WhatsApp
Custom Order Management Application
Asset Tracker
Corporate DatabaseFile
Storage
Job Scheduling
B2B APIs
Open Data APIs
APIs
Application Server
Private Blockchain
B2B APIs
AvailabilityPersonal Profile,
Order and Payments Details
Smart Contracts with supply chain
details
Recent Consumer purchases
informationFootage from
security cameras
Emails regarding customer
complaints
Log-files from IT systems (infra &
platform)
ML Models
Global Content Delivery Network
Job Schedules
Offers, invoices, rewards
messages
Order Details
API usage, billing, policies
Running & Past workflow instances
Invoices & Payments
Product Manuals
Digital Twin
KPIs & Alerts
Customer Interaction records
Case files (Complaints,
Requests)
Rules & Rule Execution
metrics
Weather, Demographics,
Sports, Social, …
Configdata
Customer Details
ML Models
Programming in progress
User Stories, Designs, Discussions
low Webshop24/7 on line
Relaxed availability (office hours) for DWH
SaaS CRM less available than desired
Fairly high availability for [clusters of] things – not
for individual things
Active Standby Database
SAN
Cross Cloud/On Premises archive
Low availability demands on Big Data
H/A for Oracle Database
EventBus24/7 on line
H/A for IoT Hub
H/A for LDAP
Fairly high availability for [clusters of] things – not
for individual things
H/A during extended office hours for human
workflow engine
Service Bus 24/7 on line
Some loss or service is acceptable for recommendation engine
![Page 80: 50 Shades of Data - how, when and why Big,Relational,NoSQL,Elastic,Event,CQRS (Devoxx Maroc, November 2017) - including detailed demo screenshots](https://reader034.fdocuments.in/reader034/viewer/2022051710/5a6489fc7f8b9a27568b5f8b/html5/thumbnails/80.jpg)
Availability of Data• Availability:
• unplanned downtime (incident => disaster)• planned (not desired) downtime (upgrade, patch to application, platform,
infra)
• Chain is as strong as the weakest link• Availability is determined by least available component
• Datastore can drive (and help improve) availability of many systems/applications/services• Custom UI on top of SAP requires 99.95% up time – SAP only offers 98%
• Increase availability• H/A architecture – multi-node cluster, hot standby and fail-over, disaster
recovery• Rolling upgrades• Single node for command, multiple (independent) helpers for query
#devoxxma
![Page 81: 50 Shades of Data - how, when and why Big,Relational,NoSQL,Elastic,Event,CQRS (Devoxx Maroc, November 2017) - including detailed demo screenshots](https://reader034.fdocuments.in/reader034/viewer/2022051710/5a6489fc7f8b9a27568b5f8b/html5/thumbnails/81.jpg)
Case of Web Shop• Webshop – 1M visitors per day
• Product catalog consists of 15+ millions of records• The web shop presents: product description, images, reviews, pricing details,
related offerings, stock status
• Some Products are added and updated and removed every day• Although most products do not change very frequently
• Some vendors do bulk manipulation of product details
#devoxxma
Products Product updates
Webshop visits- searches- product details- orders
![Page 82: 50 Shades of Data - how, when and why Big,Relational,NoSQL,Elastic,Event,CQRS (Devoxx Maroc, November 2017) - including detailed demo screenshots](https://reader034.fdocuments.in/reader034/viewer/2022051710/5a6489fc7f8b9a27568b5f8b/html5/thumbnails/82.jpg)
Case of Web Shop –Usage Patterns &
Architecture
#devoxxma
Products Product updates
Webshop visits- searches- product details- orders
firewall
Data manipulationData Quality (enforcement)<10K transactionsBatch jobs next to onlineSpeed is nice
Read onlyOn lineSpeed is crucialXHTML & JSON> 5M visits
![Page 83: 50 Shades of Data - how, when and why Big,Relational,NoSQL,Elastic,Event,CQRS (Devoxx Maroc, November 2017) - including detailed demo screenshots](https://reader034.fdocuments.in/reader034/viewer/2022051710/5a6489fc7f8b9a27568b5f8b/html5/thumbnails/83.jpg)
Case of Web Shop –Technology and Architecture
#devoxxma
ProductsProducts
Products
Webshop visits- searches- product details- orders
firewall
Data manipulationData Quality (enforcement)<10K transactionsBatch jobs next to onlineSpeed is nice
Read onlyOn lineSpeed is crucialXHTML & JSON> 1M visits
DMZ
Read onlyJSON documentsImagesText SearchScale HorizontallyStale but consistent
Products
Nightly generation
Product updates
![Page 84: 50 Shades of Data - how, when and why Big,Relational,NoSQL,Elastic,Event,CQRS (Devoxx Maroc, November 2017) - including detailed demo screenshots](https://reader034.fdocuments.in/reader034/viewer/2022051710/5a6489fc7f8b9a27568b5f8b/html5/thumbnails/84.jpg)
CQRS
#devoxxma
![Page 85: 50 Shades of Data - how, when and why Big,Relational,NoSQL,Elastic,Event,CQRS (Devoxx Maroc, November 2017) - including detailed demo screenshots](https://reader034.fdocuments.in/reader034/viewer/2022051710/5a6489fc7f8b9a27568b5f8b/html5/thumbnails/85.jpg)
CQRS - Command and QueryResponsibility Segragation
#devoxxma
Special Products
Product Clusters
ProductsData Manipulation
Data Retrieval
Food Stuff
ToysQuick Product Search Index
![Page 86: 50 Shades of Data - how, when and why Big,Relational,NoSQL,Elastic,Event,CQRS (Devoxx Maroc, November 2017) - including detailed demo screenshots](https://reader034.fdocuments.in/reader034/viewer/2022051710/5a6489fc7f8b9a27568b5f8b/html5/thumbnails/86.jpg)
CQRS in Oracle Database
#devoxxma
Active Data Guard Standby
SAN
Middleware Middleware Middleware
T T
MVMV
idx idx
IMDB
RAC RAC
Shard(12c R2)
Shard(12c R2)
SAN
SAN
dbf
SGA
RedoLogs
![Page 87: 50 Shades of Data - how, when and why Big,Relational,NoSQL,Elastic,Event,CQRS (Devoxx Maroc, November 2017) - including detailed demo screenshots](https://reader034.fdocuments.in/reader034/viewer/2022051710/5a6489fc7f8b9a27568b5f8b/html5/thumbnails/87.jpg)
CQRS - Command and QueryResponsibility Segragation
• Data manipulation and retrieval in separate places
• (physical data proliferation)
• Query store is optimizedfor consumers
• Level of detail, format,filters applied
• For performance and scalability, independence, productivitylower license fees and lower TCO, security
#devoxxma
![Page 88: 50 Shades of Data - how, when and why Big,Relational,NoSQL,Elastic,Event,CQRS (Devoxx Maroc, November 2017) - including detailed demo screenshots](https://reader034.fdocuments.in/reader034/viewer/2022051710/5a6489fc7f8b9a27568b5f8b/html5/thumbnails/88.jpg)
Synchronizing the Query Stores
#devoxxma
Special Products
Product Clusters
ProductsData Manipulation
Data Retrieval
Food Stuff
ToysQuick Product Search Index
Product Store in SaaS app
![Page 89: 50 Shades of Data - how, when and why Big,Relational,NoSQL,Elastic,Event,CQRS (Devoxx Maroc, November 2017) - including detailed demo screenshots](https://reader034.fdocuments.in/reader034/viewer/2022051710/5a6489fc7f8b9a27568b5f8b/html5/thumbnails/89.jpg)
Synchronizing the Query Stores
• Depends on• Freshness requirements
• Authorization demands
• Cost of synchronizing the query store (full synchronize vs event based)
• Usage pattern for query store
• Facilities available in Command store (and in query stores)
• Relative locations (e.g. cloud & on premises)
• Mechanisms• Importing Database dump-file (periodic, full or partial)
• Direct queries & DML
• Change Data Capture from transaction logs
• Event based
#devoxxma
![Page 90: 50 Shades of Data - how, when and why Big,Relational,NoSQL,Elastic,Event,CQRS (Devoxx Maroc, November 2017) - including detailed demo screenshots](https://reader034.fdocuments.in/reader034/viewer/2022051710/5a6489fc7f8b9a27568b5f8b/html5/thumbnails/90.jpg)
Event Sourcing
#devoxxma
![Page 91: 50 Shades of Data - how, when and why Big,Relational,NoSQL,Elastic,Event,CQRS (Devoxx Maroc, November 2017) - including detailed demo screenshots](https://reader034.fdocuments.in/reader034/viewer/2022051710/5a6489fc7f8b9a27568b5f8b/html5/thumbnails/91.jpg)
State is sum of changes
#devoxxma
Source: https://ookami86.github.io/event-sourcing-in-practice/#how-eventsourcing-works
![Page 92: 50 Shades of Data - how, when and why Big,Relational,NoSQL,Elastic,Event,CQRS (Devoxx Maroc, November 2017) - including detailed demo screenshots](https://reader034.fdocuments.in/reader034/viewer/2022051710/5a6489fc7f8b9a27568b5f8b/html5/thumbnails/92.jpg)
Take the UD out of CRUD
• Introducing the Immu Table• A ledger of entity changes
• With a timestamp or event sequence
• And the entity identifier
• And the new values of the added, changed, erased attributes
• Each event is an immutable record that is appended to the ledger – just simply added to the end• Atomic, very cheap compared to Update and Delete
– does not require a lock- it does require random file access and rearranging blocks on disk
#devoxxma
Bank Account Change Event
Event TypeTimestampAccount IdAmount(New value for) OwnerErased: some attribute
![Page 93: 50 Shades of Data - how, when and why Big,Relational,NoSQL,Elastic,Event,CQRS (Devoxx Maroc, November 2017) - including detailed demo screenshots](https://reader034.fdocuments.in/reader034/viewer/2022051710/5a6489fc7f8b9a27568b5f8b/html5/thumbnails/93.jpg)
Event Log in Event Sourcing
• Primary Data Source is ledger of change events• Not a store of the current state
• However: optionally use snapshots of baseline (state up until time)
• Entity Event Store replaces Table• Offers a simple API for creating and retrieving events
• ‘Entity Change Event’ Producer (to which consumers can subscribe)
• To correct a mistake: • Do not remove the event! (it happened, it may already have been
distributed)
• Instead, create a compensating event (and then it unhappened)
#devoxxma
![Page 94: 50 Shades of Data - how, when and why Big,Relational,NoSQL,Elastic,Event,CQRS (Devoxx Maroc, November 2017) - including detailed demo screenshots](https://reader034.fdocuments.in/reader034/viewer/2022051710/5a6489fc7f8b9a27568b5f8b/html5/thumbnails/94.jpg)
Event Log• Audit Log
• Time travel
• Reconstruct system (application state)• Distributed application state
• Support multiple (read) models
• Easy construct debugging environment – of exact situation and time
• What-if scenarios –take copy, inject event & play forward from there
• State = sum of change events• State = snapshot plus sum of recent events• To synch application state = current state + sum of events after the event
version number on which current state is based
#devoxxma
![Page 95: 50 Shades of Data - how, when and why Big,Relational,NoSQL,Elastic,Event,CQRS (Devoxx Maroc, November 2017) - including detailed demo screenshots](https://reader034.fdocuments.in/reader034/viewer/2022051710/5a6489fc7f8b9a27568b5f8b/html5/thumbnails/95.jpg)
To implementEvent Sourcing
• Take a data store• That is distributed, scalable, available
• For example Apache Cassandra
• Create an Event Log table for each business entity• Create columns for timestamp, event id, change type, entity identifier
• Create columns for all attributes or a single column to hold a document (e.g. JSON)
• A special change type can be ‘snapshot’ to specify a baseline• No older entries are needed in the event log
#devoxxma
![Page 96: 50 Shades of Data - how, when and why Big,Relational,NoSQL,Elastic,Event,CQRS (Devoxx Maroc, November 2017) - including detailed demo screenshots](https://reader034.fdocuments.in/reader034/viewer/2022051710/5a6489fc7f8b9a27568b5f8b/html5/thumbnails/96.jpg)
Event Sourcing driving CQRS
#devoxxma
![Page 97: 50 Shades of Data - how, when and why Big,Relational,NoSQL,Elastic,Event,CQRS (Devoxx Maroc, November 2017) - including detailed demo screenshots](https://reader034.fdocuments.in/reader034/viewer/2022051710/5a6489fc7f8b9a27568b5f8b/html5/thumbnails/97.jpg)
Microservices& Data
#devoxxma
![Page 98: 50 Shades of Data - how, when and why Big,Relational,NoSQL,Elastic,Event,CQRS (Devoxx Maroc, November 2017) - including detailed demo screenshots](https://reader034.fdocuments.in/reader034/viewer/2022051710/5a6489fc7f8b9a27568b5f8b/html5/thumbnails/98.jpg)
What is IT all about?
#devoxxma
Application
Production Runtime
![Page 99: 50 Shades of Data - how, when and why Big,Relational,NoSQL,Elastic,Event,CQRS (Devoxx Maroc, November 2017) - including detailed demo screenshots](https://reader034.fdocuments.in/reader034/viewer/2022051710/5a6489fc7f8b9a27568b5f8b/html5/thumbnails/99.jpg)
What is IT all about?
#devoxxma
Application
Production Runtime
Platform
![Page 100: 50 Shades of Data - how, when and why Big,Relational,NoSQL,Elastic,Event,CQRS (Devoxx Maroc, November 2017) - including detailed demo screenshots](https://reader034.fdocuments.in/reader034/viewer/2022051710/5a6489fc7f8b9a27568b5f8b/html5/thumbnails/100.jpg)
What is IT all about?
#devoxxma
Application
Platform
Production Runtime
Operations
Monitoring & Management
![Page 101: 50 Shades of Data - how, when and why Big,Relational,NoSQL,Elastic,Event,CQRS (Devoxx Maroc, November 2017) - including detailed demo screenshots](https://reader034.fdocuments.in/reader034/viewer/2022051710/5a6489fc7f8b9a27568b5f8b/html5/thumbnails/101.jpg)
One team has Agile responsibilitythrough full lifecyle
#devoxxma
Application
Platform
Production Runtime
Operations
Monitoring & ManagementApplication
Preparation Runtime
Platform
Development
CD
Agile Design, Build, Test
![Page 102: 50 Shades of Data - how, when and why Big,Relational,NoSQL,Elastic,Event,CQRS (Devoxx Maroc, November 2017) - including detailed demo screenshots](https://reader034.fdocuments.in/reader034/viewer/2022051710/5a6489fc7f8b9a27568b5f8b/html5/thumbnails/102.jpg)
One team has Agile responsibilitythrough full lifecyle
#devoxxma
Application
Platform
Production Runtime
Operations
Monitoring & ManagementApplication
Preparation Runtime
Platform
Development
CD
Agile Design, Build, Test
![Page 103: 50 Shades of Data - how, when and why Big,Relational,NoSQL,Elastic,Event,CQRS (Devoxx Maroc, November 2017) - including detailed demo screenshots](https://reader034.fdocuments.in/reader034/viewer/2022051710/5a6489fc7f8b9a27568b5f8b/html5/thumbnails/103.jpg)
One team has Agile responsibilitythrough full lifecyle
#devoxxma
Application
Platform
Application
Platform
![Page 104: 50 Shades of Data - how, when and why Big,Relational,NoSQL,Elastic,Event,CQRS (Devoxx Maroc, November 2017) - including detailed demo screenshots](https://reader034.fdocuments.in/reader034/viewer/2022051710/5a6489fc7f8b9a27568b5f8b/html5/thumbnails/104.jpg)
DevOps team owns and runs one (or more) products
#devoxxma
Application
Platform
Generic Infrastructure Platform for running DevOps Products
Floorspace, Power, Cooling, Storage,
Compute
Monitoring, Management, Cache, Authentication, RDBMS,
Event Hub
![Page 105: 50 Shades of Data - how, when and why Big,Relational,NoSQL,Elastic,Event,CQRS (Devoxx Maroc, November 2017) - including detailed demo screenshots](https://reader034.fdocuments.in/reader034/viewer/2022051710/5a6489fc7f8b9a27568b5f8b/html5/thumbnails/105.jpg)
Multiple products from multiple teamsrun on a shared generic infrastructure
#devoxxma
Generic Infrastructure Platform for running DevOps Products
Floorspace, Power, Cooling, Storage,
Compute
Monitoring, Management, Cache, Authentication, RDBMS,
Event Hub
Application
Platform
Application
Platform
Application
Platform
Application
Platform
Application
Platform
![Page 106: 50 Shades of Data - how, when and why Big,Relational,NoSQL,Elastic,Event,CQRS (Devoxx Maroc, November 2017) - including detailed demo screenshots](https://reader034.fdocuments.in/reader034/viewer/2022051710/5a6489fc7f8b9a27568b5f8b/html5/thumbnails/106.jpg)
App plus platform under DevOps== Microservice
Generic Infrastructure Platform for running DevOps Products
µ µ µ µ µ
![Page 107: 50 Shades of Data - how, when and why Big,Relational,NoSQL,Elastic,Event,CQRS (Devoxx Maroc, November 2017) - including detailed demo screenshots](https://reader034.fdocuments.in/reader034/viewer/2022051710/5a6489fc7f8b9a27568b5f8b/html5/thumbnails/107.jpg)
App plus platform under DevOps== Microservice
Generic Infrastructure Platform for running DevOps Products
µ µ µ µ µ• Stateless
• Horizontally scalable
• Mutually Independent • upgrade, patch, relocate
• Can expose Public API (HTTP/REST) and/or UI
• Communicate with each other through events
• Have their own bounded data context• Do not rely on other microservices [for the data they need]
• Serverless – do not require allocated server, can be fired up
![Page 108: 50 Shades of Data - how, when and why Big,Relational,NoSQL,Elastic,Event,CQRS (Devoxx Maroc, November 2017) - including detailed demo screenshots](https://reader034.fdocuments.in/reader034/viewer/2022051710/5a6489fc7f8b9a27568b5f8b/html5/thumbnails/108.jpg)
Microservices - objectives
• Minimize cost of change
• Maximize agility• Isolate responsibility
• Reduce cohesion by minimizing dependencies• logical, technical and runtime
• only standardized communication/interaction
• Independent, scalable processes
• Choreograhy (broadcast) preferred over Orchestration (direct call)
• Efficient operations
• Comprehendable, controllable IT
#devoxxma
How do we get
from a Monolith
to Microservices?
![Page 109: 50 Shades of Data - how, when and why Big,Relational,NoSQL,Elastic,Event,CQRS (Devoxx Maroc, November 2017) - including detailed demo screenshots](https://reader034.fdocuments.in/reader034/viewer/2022051710/5a6489fc7f8b9a27568b5f8b/html5/thumbnails/109.jpg)
Data in microservices• Microservices are stateless & horizontally scalable
• Microservices are isolated & independent
• Where is their data?
• What about lookup data?• Data not owned by the microservice –
but still required by it to perform its role => bounded context
#devoxxma
![Page 110: 50 Shades of Data - how, when and why Big,Relational,NoSQL,Elastic,Event,CQRS (Devoxx Maroc, November 2017) - including detailed demo screenshots](https://reader034.fdocuments.in/reader034/viewer/2022051710/5a6489fc7f8b9a27568b5f8b/html5/thumbnails/110.jpg)
Microservices State
#devoxxma
Cache
RDBMSDocument
StoreNoSQL
Generic Platform for running microservices
Event Hub
Big Data
Block Storage
LDAP
![Page 111: 50 Shades of Data - how, when and why Big,Relational,NoSQL,Elastic,Event,CQRS (Devoxx Maroc, November 2017) - including detailed demo screenshots](https://reader034.fdocuments.in/reader034/viewer/2022051710/5a6489fc7f8b9a27568b5f8b/html5/thumbnails/111.jpg)
Bounded contextin microservices
• Micoservice needs to be able to run independently
• It needs to contain & own all data required to run• It cannot depend on other microservices
#devoxxma
API
Customer
APIUI
OrderCustomerModified event
![Page 112: 50 Shades of Data - how, when and why Big,Relational,NoSQL,Elastic,Event,CQRS (Devoxx Maroc, November 2017) - including detailed demo screenshots](https://reader034.fdocuments.in/reader034/viewer/2022051710/5a6489fc7f8b9a27568b5f8b/html5/thumbnails/112.jpg)
DemoMicroservices, CQRS & Events
#devoxxma
![Page 113: 50 Shades of Data - how, when and why Big,Relational,NoSQL,Elastic,Event,CQRS (Devoxx Maroc, November 2017) - including detailed demo screenshots](https://reader034.fdocuments.in/reader034/viewer/2022051710/5a6489fc7f8b9a27568b5f8b/html5/thumbnails/113.jpg)
Demo – Shopping Cart & Order microservices plus
state
#devoxxma
APIUI
Order
Cache
Application Container Cache
APIUIRDBMS
Oracle Database Cloud
ShoppingCart& Check Out
![Page 114: 50 Shades of Data - how, when and why Big,Relational,NoSQL,Elastic,Event,CQRS (Devoxx Maroc, November 2017) - including detailed demo screenshots](https://reader034.fdocuments.in/reader034/viewer/2022051710/5a6489fc7f8b9a27568b5f8b/html5/thumbnails/114.jpg)
Demo – Checking Out Cart => create order & publish event
#devoxxma
APIUI
Order
Cache
Oracle Application Container Cache
APIUIRDBMS Oracle Database
Cloud
ShoppingCart& Check Out
EventHubOracle Event Hub Cloud
NewOrder event
![Page 115: 50 Shades of Data - how, when and why Big,Relational,NoSQL,Elastic,Event,CQRS (Devoxx Maroc, November 2017) - including detailed demo screenshots](https://reader034.fdocuments.in/reader034/viewer/2022051710/5a6489fc7f8b9a27568b5f8b/html5/thumbnails/115.jpg)
Demo – Workflow toEvaluate New Order
#devoxxma
Event Bus
Check Total
Cache
Workflow Launcher
API
Check Shipping
API API
Judge Order
![Page 116: 50 Shades of Data - how, when and why Big,Relational,NoSQL,Elastic,Event,CQRS (Devoxx Maroc, November 2017) - including detailed demo screenshots](https://reader034.fdocuments.in/reader034/viewer/2022051710/5a6489fc7f8b9a27568b5f8b/html5/thumbnails/116.jpg)
Demo –Workflow Implementation- Docker Containers on
Kubernetes
#devoxxma
WorkflowLauncher
Check OrderTotal
Check Shipping
Cache CacheInspector
LogMonitor
Event Bus
OrderArbiter
![Page 117: 50 Shades of Data - how, when and why Big,Relational,NoSQL,Elastic,Event,CQRS (Devoxx Maroc, November 2017) - including detailed demo screenshots](https://reader034.fdocuments.in/reader034/viewer/2022051710/5a6489fc7f8b9a27568b5f8b/html5/thumbnails/117.jpg)
Demo – Workflow triggeredby NewOrder event
#devoxxma
Event Bus
Check Total
Cache
EventBridge
Workflow Launcher
API
Check Shipping
API API
Judge Order
NewOrder event
EventHubEvent Hub
Cloud
NewOrder event
![Page 118: 50 Shades of Data - how, when and why Big,Relational,NoSQL,Elastic,Event,CQRS (Devoxx Maroc, November 2017) - including detailed demo screenshots](https://reader034.fdocuments.in/reader034/viewer/2022051710/5a6489fc7f8b9a27568b5f8b/html5/thumbnails/118.jpg)
Demo
#devoxxma
APIUI
Order
Cache
Application Container Cache
APIUIRDBMS
Database Cloud
ShoppingCart& Check Out
Event Bus
Check Total
Cache
EventBridge
Workflow Launcher
API
Check Shipping
API API
Judge Order
EventHubEvent Hub
Cloud
![Page 119: 50 Shades of Data - how, when and why Big,Relational,NoSQL,Elastic,Event,CQRS (Devoxx Maroc, November 2017) - including detailed demo screenshots](https://reader034.fdocuments.in/reader034/viewer/2022051710/5a6489fc7f8b9a27568b5f8b/html5/thumbnails/119.jpg)
Demo
#devoxxma
APIUI
Order
Cache
Application Container Cache
APIUIRDBMS
Database Cloud
ShoppingCart& Check Out
EventBridge
EventHubEvent Hub
Cloud
WorkflowLauncher
Check OrderTotal
Check Shipping
Cache CacheInspector
LogMonitor
Event Bus
OrderArbiter
![Page 120: 50 Shades of Data - how, when and why Big,Relational,NoSQL,Elastic,Event,CQRS (Devoxx Maroc, November 2017) - including detailed demo screenshots](https://reader034.fdocuments.in/reader034/viewer/2022051710/5a6489fc7f8b9a27568b5f8b/html5/thumbnails/120.jpg)
Demo
#devoxxma
APIUI
Order
Cache
Application Container Cache
APIUIRDBMS
Database Cloud
ShoppingCart& Check Out
API Event Bus
Customer
Check Total
Cache
EventBridge
Workflow Launcher
API
Check Shipping
API API
Judge Order
EventHubEvent Hub
Cloud
![Page 121: 50 Shades of Data - how, when and why Big,Relational,NoSQL,Elastic,Event,CQRS (Devoxx Maroc, November 2017) - including detailed demo screenshots](https://reader034.fdocuments.in/reader034/viewer/2022051710/5a6489fc7f8b9a27568b5f8b/html5/thumbnails/121.jpg)
Demo
#devoxxma
APIUI
Order
Cache
Application Container Cache
APIUI
RDBMSOrders &
Customer Bound Context
Database Cloud
ShoppingCart& Check Out
API Event Bus
Customer
Check Total
Cache
EventBridge
Workflow Launcher
API
Check Shipping
API API
Judge Order
EventHubEvent Hub
Cloud
CustomerModified event
![Page 122: 50 Shades of Data - how, when and why Big,Relational,NoSQL,Elastic,Event,CQRS (Devoxx Maroc, November 2017) - including detailed demo screenshots](https://reader034.fdocuments.in/reader034/viewer/2022051710/5a6489fc7f8b9a27568b5f8b/html5/thumbnails/122.jpg)
Wrap Up
#devoxxma
![Page 123: 50 Shades of Data - how, when and why Big,Relational,NoSQL,Elastic,Event,CQRS (Devoxx Maroc, November 2017) - including detailed demo screenshots](https://reader034.fdocuments.in/reader034/viewer/2022051710/5a6489fc7f8b9a27568b5f8b/html5/thumbnails/123.jpg)
Wrap Up• Data used to be like T-Ford
• One model, one color
• And then:
#devoxxma
![Page 124: 50 Shades of Data - how, when and why Big,Relational,NoSQL,Elastic,Event,CQRS (Devoxx Maroc, November 2017) - including detailed demo screenshots](https://reader034.fdocuments.in/reader034/viewer/2022051710/5a6489fc7f8b9a27568b5f8b/html5/thumbnails/124.jpg)
Wrap Up• Data comes in many shades (at least 50) – variations along
many dimensions
#devoxxma
usage
Total Cost of Data Ownership
authorization
distribution
formatvolatility volume
ACID demands availability
freshness requirements (staleness allowance)
location
speed
ownership
required consistency
![Page 125: 50 Shades of Data - how, when and why Big,Relational,NoSQL,Elastic,Event,CQRS (Devoxx Maroc, November 2017) - including detailed demo screenshots](https://reader034.fdocuments.in/reader034/viewer/2022051710/5a6489fc7f8b9a27568b5f8b/html5/thumbnails/125.jpg)
Wrap Up• Some form of CQRS is plain common sense
• Use fitting technology for the query challenge at hand• Graph, Document, Relational, Key/Value, Column, Elastic Index, …
• Every organization will (should) have multiple data stores in various technologies – and not just relational SQL
• Design & implement mechanism to synchronizethe query stores• Events are attractive: decoupled, fine grained and fast
• Devise a purging strategy• Stop carrying around your data legacy
#devoxxma
![Page 126: 50 Shades of Data - how, when and why Big,Relational,NoSQL,Elastic,Event,CQRS (Devoxx Maroc, November 2017) - including detailed demo screenshots](https://reader034.fdocuments.in/reader034/viewer/2022051710/5a6489fc7f8b9a27568b5f8b/html5/thumbnails/126.jpg)
Wrap Up• All data is stale
• Consistency should be your main concern
• Microservices are stateless• They can own state – in their private data store• And maintain derived state – bounded context• Events are published to allow microservices to synch their context
• Event Sourcing reduces complexity• CRUD => CR• Keep a ledger of data changes (book keeping of DML transactions)• Reconstruct state – current or historical – from events
(into query store)
#devoxxma
![Page 127: 50 Shades of Data - how, when and why Big,Relational,NoSQL,Elastic,Event,CQRS (Devoxx Maroc, November 2017) - including detailed demo screenshots](https://reader034.fdocuments.in/reader034/viewer/2022051710/5a6489fc7f8b9a27568b5f8b/html5/thumbnails/127.jpg)
Wrap Up• Data Integrity may be overrated
• Instead of enforcing constraints (reality may not be so clean) –identify anomalies in data and act on them
• SQL sits on top of the world• SQL [like query languages] run against a wide array of data stores,
including Streams, Big Data, NoSQL and CSV / Excel
• People and tools know SQL – make use of that
• Machine Learning and Artificial Intelligence are fueled by data• They make the smallest, rawest, silliest piece of data potentially
valuable
#devoxxma
![Page 128: 50 Shades of Data - how, when and why Big,Relational,NoSQL,Elastic,Event,CQRS (Devoxx Maroc, November 2017) - including detailed demo screenshots](https://reader034.fdocuments.in/reader034/viewer/2022051710/5a6489fc7f8b9a27568b5f8b/html5/thumbnails/128.jpg)
Wrap Up
#devoxxma
![Page 129: 50 Shades of Data - how, when and why Big,Relational,NoSQL,Elastic,Event,CQRS (Devoxx Maroc, November 2017) - including detailed demo screenshots](https://reader034.fdocuments.in/reader034/viewer/2022051710/5a6489fc7f8b9a27568b5f8b/html5/thumbnails/129.jpg)
Wrap Up
#devoxxma
DATA
DATA DATA
![Page 130: 50 Shades of Data - how, when and why Big,Relational,NoSQL,Elastic,Event,CQRS (Devoxx Maroc, November 2017) - including detailed demo screenshots](https://reader034.fdocuments.in/reader034/viewer/2022051710/5a6489fc7f8b9a27568b5f8b/html5/thumbnails/130.jpg)
Thank You!
#devoxxma
Lucas Jellema
AMIS (The Netherlands)
@lucasjellema
technology.amis.nl
GitHub:
https://github.com/lucasjellema/order-data-demo-devoxx
![Page 131: 50 Shades of Data - how, when and why Big,Relational,NoSQL,Elastic,Event,CQRS (Devoxx Maroc, November 2017) - including detailed demo screenshots](https://reader034.fdocuments.in/reader034/viewer/2022051710/5a6489fc7f8b9a27568b5f8b/html5/thumbnails/131.jpg)
And This
#devoxxma
![Page 132: 50 Shades of Data - how, when and why Big,Relational,NoSQL,Elastic,Event,CQRS (Devoxx Maroc, November 2017) - including detailed demo screenshots](https://reader034.fdocuments.in/reader034/viewer/2022051710/5a6489fc7f8b9a27568b5f8b/html5/thumbnails/132.jpg)
Tip: Manage Test Data Setfor (automated) tests
• Build and maintain a functional test dataset with all cases and relevant details
• Dataset inside container image• Run container (start database, load data set)• Execute test
• No set up, no tear down
• Stop & destroy container• Next test – or even in parallel• Recreate the data set & database container image at the end of
each sprint• Note: Docker Snapshots
#devoxxma
![Page 133: 50 Shades of Data - how, when and why Big,Relational,NoSQL,Elastic,Event,CQRS (Devoxx Maroc, November 2017) - including detailed demo screenshots](https://reader034.fdocuments.in/reader034/viewer/2022051710/5a6489fc7f8b9a27568b5f8b/html5/thumbnails/133.jpg)
Smart Data Capture
• Voice Recognition plus NLP
• Optical Character Recognition plus NLP
• Context based enrichment and interpretation
• Machine Learning• Predictive data preparation
• Smart data cleansing
![Page 134: 50 Shades of Data - how, when and why Big,Relational,NoSQL,Elastic,Event,CQRS (Devoxx Maroc, November 2017) - including detailed demo screenshots](https://reader034.fdocuments.in/reader034/viewer/2022051710/5a6489fc7f8b9a27568b5f8b/html5/thumbnails/134.jpg)
Considerations• Right to be forgotten
• But we do not remove events from event sourcing
• And by the way: data is proliferated across many stores
• Enterprise Identifiers
• Pull the Plug Lithmus test: • Kill microservices and have restart
• Kill data store node
• Pull power plug, network plug – anything that could fail
• Destroy derived data store (Q in CQRS is rebuildable – test it)
#devoxxma
![Page 135: 50 Shades of Data - how, when and why Big,Relational,NoSQL,Elastic,Event,CQRS (Devoxx Maroc, November 2017) - including detailed demo screenshots](https://reader034.fdocuments.in/reader034/viewer/2022051710/5a6489fc7f8b9a27568b5f8b/html5/thumbnails/135.jpg)
Considerations for data services
• Batch or Set-wise operations in services and APIs
• Smart operation parameters to leverage strengths• Sort at data store level
• Pagination
• Filter
• Steer depth/breadth of data tree
• Prune attributes
#devoxxma
![Page 136: 50 Shades of Data - how, when and why Big,Relational,NoSQL,Elastic,Event,CQRS (Devoxx Maroc, November 2017) - including detailed demo screenshots](https://reader034.fdocuments.in/reader034/viewer/2022051710/5a6489fc7f8b9a27568b5f8b/html5/thumbnails/136.jpg)
DemoScreenshots
#devoxxma
![Page 137: 50 Shades of Data - how, when and why Big,Relational,NoSQL,Elastic,Event,CQRS (Devoxx Maroc, November 2017) - including detailed demo screenshots](https://reader034.fdocuments.in/reader034/viewer/2022051710/5a6489fc7f8b9a27568b5f8b/html5/thumbnails/137.jpg)
Working withShopping Cart
microservice
#devoxxma
Cache
Application Container Cache
APIUI
Microservice ShoppingCart& Check Out
![Page 138: 50 Shades of Data - how, when and why Big,Relational,NoSQL,Elastic,Event,CQRS (Devoxx Maroc, November 2017) - including detailed demo screenshots](https://reader034.fdocuments.in/reader034/viewer/2022051710/5a6489fc7f8b9a27568b5f8b/html5/thumbnails/138.jpg)
Create a Shopping Cart Instance
#devoxxma
Cache
Application Container Cache
APIUI
Microservice ShoppingCart& Check Out
![Page 139: 50 Shades of Data - how, when and why Big,Relational,NoSQL,Elastic,Event,CQRS (Devoxx Maroc, November 2017) - including detailed demo screenshots](https://reader034.fdocuments.in/reader034/viewer/2022051710/5a6489fc7f8b9a27568b5f8b/html5/thumbnails/139.jpg)
Retrieve a Shopping Cart Instance
#devoxxma
Cache
Application Container Cache
APIUI
Microservice ShoppingCart& Check Out
![Page 140: 50 Shades of Data - how, when and why Big,Relational,NoSQL,Elastic,Event,CQRS (Devoxx Maroc, November 2017) - including detailed demo screenshots](https://reader034.fdocuments.in/reader034/viewer/2022051710/5a6489fc7f8b9a27568b5f8b/html5/thumbnails/140.jpg)
Update a Shopping Cart Instance
#devoxxma
Cache
Application Container Cache
APIUI
Microservice ShoppingCart& Check Out
![Page 141: 50 Shades of Data - how, when and why Big,Relational,NoSQL,Elastic,Event,CQRS (Devoxx Maroc, November 2017) - including detailed demo screenshots](https://reader034.fdocuments.in/reader034/viewer/2022051710/5a6489fc7f8b9a27568b5f8b/html5/thumbnails/141.jpg)
Create a new order throughtheOrder Microservice
#devoxxma
APIUI
Order
RDBMS Oracle Database Cloud
![Page 142: 50 Shades of Data - how, when and why Big,Relational,NoSQL,Elastic,Event,CQRS (Devoxx Maroc, November 2017) - including detailed demo screenshots](https://reader034.fdocuments.in/reader034/viewer/2022051710/5a6489fc7f8b9a27568b5f8b/html5/thumbnails/142.jpg)
Create a new order throughtheOrder Microservice
#devoxxma
APIUI
Order
RDBMS Oracle Database Cloud
![Page 143: 50 Shades of Data - how, when and why Big,Relational,NoSQL,Elastic,Event,CQRS (Devoxx Maroc, November 2017) - including detailed demo screenshots](https://reader034.fdocuments.in/reader034/viewer/2022051710/5a6489fc7f8b9a27568b5f8b/html5/thumbnails/143.jpg)
Create a new order throughtheOrder Microservice
#devoxxma
APIUI
Order
RDBMS Oracle Database Cloud
![Page 144: 50 Shades of Data - how, when and why Big,Relational,NoSQL,Elastic,Event,CQRS (Devoxx Maroc, November 2017) - including detailed demo screenshots](https://reader034.fdocuments.in/reader034/viewer/2022051710/5a6489fc7f8b9a27568b5f8b/html5/thumbnails/144.jpg)
Check database for new order
#devoxxma
APIUI
Order
RDBMS Oracle Database Cloud
![Page 145: 50 Shades of Data - how, when and why Big,Relational,NoSQL,Elastic,Event,CQRS (Devoxx Maroc, November 2017) - including detailed demo screenshots](https://reader034.fdocuments.in/reader034/viewer/2022051710/5a6489fc7f8b9a27568b5f8b/html5/thumbnails/145.jpg)
Check in EventBridge for NewOrder Event
#devoxxma
APIUI
Order
RDBMS Oracle Database Cloud
EventHub Oracle Event Hub
Cloud
NewOrder event
EventBridge
![Page 146: 50 Shades of Data - how, when and why Big,Relational,NoSQL,Elastic,Event,CQRS (Devoxx Maroc, November 2017) - including detailed demo screenshots](https://reader034.fdocuments.in/reader034/viewer/2022051710/5a6489fc7f8b9a27568b5f8b/html5/thumbnails/146.jpg)
Checking Out Shopping Cart => create order & publish event
#devoxxma
APIUI
Order
Cache
Oracle Application Container Cache
APIUIRDBMS Oracle Database
Cloud
ShoppingCart& Check Out
EventHubOracle Event Hub Cloud
NewOrder event
![Page 147: 50 Shades of Data - how, when and why Big,Relational,NoSQL,Elastic,Event,CQRS (Devoxx Maroc, November 2017) - including detailed demo screenshots](https://reader034.fdocuments.in/reader034/viewer/2022051710/5a6489fc7f8b9a27568b5f8b/html5/thumbnails/147.jpg)
Checking Out Shopping Cart => create order & publish event
#devoxxma
APIUI
Order
Cache
Oracle Application Container Cache
APIUIRDBMS Oracle Database
Cloud
ShoppingCart& Check Out
EventHub Oracle Event Hub
Cloud
NewOrder event
![Page 148: 50 Shades of Data - how, when and why Big,Relational,NoSQL,Elastic,Event,CQRS (Devoxx Maroc, November 2017) - including detailed demo screenshots](https://reader034.fdocuments.in/reader034/viewer/2022051710/5a6489fc7f8b9a27568b5f8b/html5/thumbnails/148.jpg)
Checking for New Orderrecordin Database
#devoxxma
APIUI
Order
Cache
Oracle Application Container Cache
APIUIRDBMS Oracle Database
Cloud
ShoppingCart& Check Out
EventHub Oracle Event Hub
Cloud
NewOrder event
![Page 149: 50 Shades of Data - how, when and why Big,Relational,NoSQL,Elastic,Event,CQRS (Devoxx Maroc, November 2017) - including detailed demo screenshots](https://reader034.fdocuments.in/reader034/viewer/2022051710/5a6489fc7f8b9a27568b5f8b/html5/thumbnails/149.jpg)
Check for NewOrder Eventon the EventBridge
#devoxxma
APIUI
Order
Cache
Oracle Application Container Cache
APIUIRDBMS Oracle Database
Cloud
ShoppingCart& Check Out
EventHub Oracle Event Hub
Cloud
NewOrder event
EventBridge
![Page 150: 50 Shades of Data - how, when and why Big,Relational,NoSQL,Elastic,Event,CQRS (Devoxx Maroc, November 2017) - including detailed demo screenshots](https://reader034.fdocuments.in/reader034/viewer/2022051710/5a6489fc7f8b9a27568b5f8b/html5/thumbnails/150.jpg)
Update Customer Name – Havebounded context updated in Order
#devoxxma
APIUI
Order
RDBMSOrders &
Customer Bound Context
Database Cloud
API
Customer
EventHubEvent Hub
Cloud
CustomerModified event
![Page 151: 50 Shades of Data - how, when and why Big,Relational,NoSQL,Elastic,Event,CQRS (Devoxx Maroc, November 2017) - including detailed demo screenshots](https://reader034.fdocuments.in/reader034/viewer/2022051710/5a6489fc7f8b9a27568b5f8b/html5/thumbnails/151.jpg)
Update Customer Name
#devoxxma
APIUI
Order
RDBMSOrders &
Customer Bound Context
Database Cloud
API
Customer
EventHub Event Hub Cloud
CustomerModified event
![Page 152: 50 Shades of Data - how, when and why Big,Relational,NoSQL,Elastic,Event,CQRS (Devoxx Maroc, November 2017) - including detailed demo screenshots](https://reader034.fdocuments.in/reader034/viewer/2022051710/5a6489fc7f8b9a27568b5f8b/html5/thumbnails/152.jpg)
Update Customer Name
#devoxxma
APIUI
Order
RDBMSOrders &
Customer Bound Context
Database Cloud
API
Customer
EventHub Event Hub Cloud
CustomerModified event
![Page 153: 50 Shades of Data - how, when and why Big,Relational,NoSQL,Elastic,Event,CQRS (Devoxx Maroc, November 2017) - including detailed demo screenshots](https://reader034.fdocuments.in/reader034/viewer/2022051710/5a6489fc7f8b9a27568b5f8b/html5/thumbnails/153.jpg)
Update Customer Name
#devoxxma
APIUI
Order
RDBMSOrders &
Customer Bound Context
Database Cloud
API
Customer
EventHub Event Hub Cloud
CustomerModified event
![Page 154: 50 Shades of Data - how, when and why Big,Relational,NoSQL,Elastic,Event,CQRS (Devoxx Maroc, November 2017) - including detailed demo screenshots](https://reader034.fdocuments.in/reader034/viewer/2022051710/5a6489fc7f8b9a27568b5f8b/html5/thumbnails/154.jpg)
Check customer updatethrough customersmicroservice
#devoxxma
APIUI
Order
RDBMSOrders &
Customer Bound Context
Database Cloud
API
Customer
EventHub Event Hub Cloud
![Page 155: 50 Shades of Data - how, when and why Big,Relational,NoSQL,Elastic,Event,CQRS (Devoxx Maroc, November 2017) - including detailed demo screenshots](https://reader034.fdocuments.in/reader034/viewer/2022051710/5a6489fc7f8b9a27568b5f8b/html5/thumbnails/155.jpg)
Verify update customer detailsin orders
#devoxxma
APIUI
Order
RDBMSOrders &
Customer Bound Context
Database Cloud
API
Customer
EventHub Event Hub Cloud
CustomerModified event
![Page 156: 50 Shades of Data - how, when and why Big,Relational,NoSQL,Elastic,Event,CQRS (Devoxx Maroc, November 2017) - including detailed demo screenshots](https://reader034.fdocuments.in/reader034/viewer/2022051710/5a6489fc7f8b9a27568b5f8b/html5/thumbnails/156.jpg)
Verify update bound contextin database
#devoxxma
APIUI
Order
RDBMSOrders &
Customer Bound Context
Database Cloud
API
Customer
EventHub Event Hub Cloud
CustomerModified event
![Page 157: 50 Shades of Data - how, when and why Big,Relational,NoSQL,Elastic,Event,CQRS (Devoxx Maroc, November 2017) - including detailed demo screenshots](https://reader034.fdocuments.in/reader034/viewer/2022051710/5a6489fc7f8b9a27568b5f8b/html5/thumbnails/157.jpg)
Order Workflow to Approve or Reject
new Orders
#devoxxma
APIUI
Order
RDBMS
Oracle Database Cloud
Event Bus
Check Total
Cache
EventBridge
Workflow Launcher
API
Check Shipping
API API
Judge Order
EventHubOracle Event Hub Cloud
NewOrder event
NewOrder event
NewOrder event
![Page 158: 50 Shades of Data - how, when and why Big,Relational,NoSQL,Elastic,Event,CQRS (Devoxx Maroc, November 2017) - including detailed demo screenshots](https://reader034.fdocuments.in/reader034/viewer/2022051710/5a6489fc7f8b9a27568b5f8b/html5/thumbnails/158.jpg)
Order Workflow to Approve or Reject
new Orders
#devoxxma
APIUI
Order
RDBMS
Oracle Database Cloud
Event Bus
Check Total
Cache
EventBridge
Workflow Launcher
API
Check Shipping
API API
Judge Order
EventHubOracle Event Hub Cloud
OrderApproved event
OrderRejected event
NewOrder event
OrderApproved event
OrderRejected event
OrderApproved event
OrderRejected event
![Page 159: 50 Shades of Data - how, when and why Big,Relational,NoSQL,Elastic,Event,CQRS (Devoxx Maroc, November 2017) - including detailed demo screenshots](https://reader034.fdocuments.in/reader034/viewer/2022051710/5a6489fc7f8b9a27568b5f8b/html5/thumbnails/159.jpg)
Create new shopping cart
#devoxxma
![Page 160: 50 Shades of Data - how, when and why Big,Relational,NoSQL,Elastic,Event,CQRS (Devoxx Maroc, November 2017) - including detailed demo screenshots](https://reader034.fdocuments.in/reader034/viewer/2022051710/5a6489fc7f8b9a27568b5f8b/html5/thumbnails/160.jpg)
Check Out Shopping Cart –resulting in a new order record
#devoxxma
APIUI
Order
RDBMS
Oracle Database Cloud
Cache
APIUI
ShoppingCart& Check Out
![Page 161: 50 Shades of Data - how, when and why Big,Relational,NoSQL,Elastic,Event,CQRS (Devoxx Maroc, November 2017) - including detailed demo screenshots](https://reader034.fdocuments.in/reader034/viewer/2022051710/5a6489fc7f8b9a27568b5f8b/html5/thumbnails/161.jpg)
Check Out Shopping Cart –resulting in a new order record
#devoxxma
![Page 162: 50 Shades of Data - how, when and why Big,Relational,NoSQL,Elastic,Event,CQRS (Devoxx Maroc, November 2017) - including detailed demo screenshots](https://reader034.fdocuments.in/reader034/viewer/2022051710/5a6489fc7f8b9a27568b5f8b/html5/thumbnails/162.jpg)
NewOrder Event is published –and transported from cloud to on
premises
#devoxxma
APIUI
Order
RDBMS
Oracle Database Cloud
Event Bus
EventBridge
EventHubOracle Event Hub Cloud
NewOrder eventNewOrder event
![Page 163: 50 Shades of Data - how, when and why Big,Relational,NoSQL,Elastic,Event,CQRS (Devoxx Maroc, November 2017) - including detailed demo screenshots](https://reader034.fdocuments.in/reader034/viewer/2022051710/5a6489fc7f8b9a27568b5f8b/html5/thumbnails/163.jpg)
Order Workflow to Approve or Reject
new orders is kicked off
#devoxxma
Event Bus
Check Total
Cache
Workflow Launcher
API
Check Shipping
API API
Judge Order
NewOrder event
![Page 164: 50 Shades of Data - how, when and why Big,Relational,NoSQL,Elastic,Event,CQRS (Devoxx Maroc, November 2017) - including detailed demo screenshots](https://reader034.fdocuments.in/reader034/viewer/2022051710/5a6489fc7f8b9a27568b5f8b/html5/thumbnails/164.jpg)
Routing slip for Order Workflow
#devoxxma
Check TotalWorkflow Launcher
API
Check Shipping
API API
Judge Order
NewOrder event
![Page 165: 50 Shades of Data - how, when and why Big,Relational,NoSQL,Elastic,Event,CQRS (Devoxx Maroc, November 2017) - including detailed demo screenshots](https://reader034.fdocuments.in/reader034/viewer/2022051710/5a6489fc7f8b9a27568b5f8b/html5/thumbnails/165.jpg)
Routing slip for Order Workflow
#devoxxma
Check TotalWorkflow Launcher
API
Check Shipping
API API
Judge Order
NewOrder event
New Order
Check Shipping
Amsterdam is not allowedas a shipping location
Check Order Total
OrderShippingRejector
When both CheckOrderTotaland CheckShipping have result OK then approve
order
The order total should bebelow 10K
OrderTotalRejector
OrderApprover
OK
OK
NOK
NOK
When CheckShipping has result NOK then reject order
When CheckOrderTotal has result NOK then reject order
OrderApproved event
OrderRejected event
OrderRejected event
![Page 166: 50 Shades of Data - how, when and why Big,Relational,NoSQL,Elastic,Event,CQRS (Devoxx Maroc, November 2017) - including detailed demo screenshots](https://reader034.fdocuments.in/reader034/viewer/2022051710/5a6489fc7f8b9a27568b5f8b/html5/thumbnails/166.jpg)
Shipping Details are Checked
#devoxxma
![Page 167: 50 Shades of Data - how, when and why Big,Relational,NoSQL,Elastic,Event,CQRS (Devoxx Maroc, November 2017) - including detailed demo screenshots](https://reader034.fdocuments.in/reader034/viewer/2022051710/5a6489fc7f8b9a27568b5f8b/html5/thumbnails/167.jpg)
Order Total is Checked
#devoxxma
![Page 168: 50 Shades of Data - how, when and why Big,Relational,NoSQL,Elastic,Event,CQRS (Devoxx Maroc, November 2017) - including detailed demo screenshots](https://reader034.fdocuments.in/reader034/viewer/2022051710/5a6489fc7f8b9a27568b5f8b/html5/thumbnails/168.jpg)
Order Workflow is completed by Order Arbiter – Order is approved
#devoxxma
Event Bus
Check Total
Cache
EventBridge
Workflow Launcher
API
Check Shipping
API API
Judge Order
OrderApproved event
OrderRejected event
NewOrder event
OrderApproved event
OrderRejected event
![Page 169: 50 Shades of Data - how, when and why Big,Relational,NoSQL,Elastic,Event,CQRS (Devoxx Maroc, November 2017) - including detailed demo screenshots](https://reader034.fdocuments.in/reader034/viewer/2022051710/5a6489fc7f8b9a27568b5f8b/html5/thumbnails/169.jpg)
Order Workflow Logging
#devoxxma
Event Bus
Check Total
Cache
Workflow Launcher
API
Check Shipping
API API
Judge Order
NewOrder event
OrderApproved event
OrderRejected event
![Page 170: 50 Shades of Data - how, when and why Big,Relational,NoSQL,Elastic,Event,CQRS (Devoxx Maroc, November 2017) - including detailed demo screenshots](https://reader034.fdocuments.in/reader034/viewer/2022051710/5a6489fc7f8b9a27568b5f8b/html5/thumbnails/170.jpg)
Order Appoved Event is sent to Event Hub,consumed in Order and results in databaserecord being updated
#devoxxma
APIUI
Order
RDBMS
Oracle Database Cloud
EventBridge
EventHubOracle Event Hub Cloud
OrderApproved event
OrderRejected event
OrderApproved event
OrderRejected event
![Page 171: 50 Shades of Data - how, when and why Big,Relational,NoSQL,Elastic,Event,CQRS (Devoxx Maroc, November 2017) - including detailed demo screenshots](https://reader034.fdocuments.in/reader034/viewer/2022051710/5a6489fc7f8b9a27568b5f8b/html5/thumbnails/171.jpg)
Second shopping cartLarge order total and Shipping to
Amsterdam
#devoxxma
![Page 172: 50 Shades of Data - how, when and why Big,Relational,NoSQL,Elastic,Event,CQRS (Devoxx Maroc, November 2017) - including detailed demo screenshots](https://reader034.fdocuments.in/reader034/viewer/2022051710/5a6489fc7f8b9a27568b5f8b/html5/thumbnails/172.jpg)
Second shopping cartLarge order total and Shipping to
Amsterdam
#devoxxma
![Page 173: 50 Shades of Data - how, when and why Big,Relational,NoSQL,Elastic,Event,CQRS (Devoxx Maroc, November 2017) - including detailed demo screenshots](https://reader034.fdocuments.in/reader034/viewer/2022051710/5a6489fc7f8b9a27568b5f8b/html5/thumbnails/173.jpg)
Large order total and Shipping to Amsterdam
- after a little while: order is rejected
#devoxxma
![Page 174: 50 Shades of Data - how, when and why Big,Relational,NoSQL,Elastic,Event,CQRS (Devoxx Maroc, November 2017) - including detailed demo screenshots](https://reader034.fdocuments.in/reader034/viewer/2022051710/5a6489fc7f8b9a27568b5f8b/html5/thumbnails/174.jpg)
Order was rejected
#devoxxma
APIUI
Order
RDBMS
Oracle Database Cloud
Event Bus
Check Total
Cache
EventBridge
Workflow Launcher
API
Check Shipping
API API
Judge Order
EventHubOracle Event Hub Cloud
OrderRejected event
NewOrder event
OrderRejected event
OrderRejected event
![Page 175: 50 Shades of Data - how, when and why Big,Relational,NoSQL,Elastic,Event,CQRS (Devoxx Maroc, November 2017) - including detailed demo screenshots](https://reader034.fdocuments.in/reader034/viewer/2022051710/5a6489fc7f8b9a27568b5f8b/html5/thumbnails/175.jpg)
What happened?
#devoxxma
![Page 176: 50 Shades of Data - how, when and why Big,Relational,NoSQL,Elastic,Event,CQRS (Devoxx Maroc, November 2017) - including detailed demo screenshots](https://reader034.fdocuments.in/reader034/viewer/2022051710/5a6489fc7f8b9a27568b5f8b/html5/thumbnails/176.jpg)
Order was rejected
#devoxxma
APIUI
Order
RDBMS
Oracle Database Cloud
Event Bus
Check Total
Cache
EventBridge
Workflow Launcher
API
Check Shipping
API API
Judge Order
EventHubOracle Event Hub Cloud
OrderRejected event
NewOrder event
OrderRejected event
OrderRejected event
![Page 177: 50 Shades of Data - how, when and why Big,Relational,NoSQL,Elastic,Event,CQRS (Devoxx Maroc, November 2017) - including detailed demo screenshots](https://reader034.fdocuments.in/reader034/viewer/2022051710/5a6489fc7f8b9a27568b5f8b/html5/thumbnails/177.jpg)
Order was rejected
#devoxxma
APIUI
Order
RDBMS
Oracle Database Cloud
Event Bus
Check Total
Cache
EventBridge
Workflow Launcher
API
Check Shipping
API API
Judge Order
EventHubOracle Event Hub Cloud
OrderRejected event
NewOrder event
OrderRejected event
OrderRejected event
![Page 178: 50 Shades of Data - how, when and why Big,Relational,NoSQL,Elastic,Event,CQRS (Devoxx Maroc, November 2017) - including detailed demo screenshots](https://reader034.fdocuments.in/reader034/viewer/2022051710/5a6489fc7f8b9a27568b5f8b/html5/thumbnails/178.jpg)
Order was rejected
#devoxxma
APIUI
Order
RDBMS
Oracle Database Cloud
Event Bus
Check Total
Cache
EventBridge
Workflow Launcher
API
Check Shipping
API API
Judge Order
EventHubOracle Event Hub Cloud
OrderRejected event
NewOrder event
OrderRejected event
OrderRejected event
![Page 179: 50 Shades of Data - how, when and why Big,Relational,NoSQL,Elastic,Event,CQRS (Devoxx Maroc, November 2017) - including detailed demo screenshots](https://reader034.fdocuments.in/reader034/viewer/2022051710/5a6489fc7f8b9a27568b5f8b/html5/thumbnails/179.jpg)
Looking at Workflow Routing Slipin Redis Cache on Kubernetes
#devoxxma
![Page 180: 50 Shades of Data - how, when and why Big,Relational,NoSQL,Elastic,Event,CQRS (Devoxx Maroc, November 2017) - including detailed demo screenshots](https://reader034.fdocuments.in/reader034/viewer/2022051710/5a6489fc7f8b9a27568b5f8b/html5/thumbnails/180.jpg)
Looking at Workflow Routing Slipin Redis Cache on Kubernetes
#devoxxma
![Page 181: 50 Shades of Data - how, when and why Big,Relational,NoSQL,Elastic,Event,CQRS (Devoxx Maroc, November 2017) - including detailed demo screenshots](https://reader034.fdocuments.in/reader034/viewer/2022051710/5a6489fc7f8b9a27568b5f8b/html5/thumbnails/181.jpg)
Looking at Workflow Routing Slipin Redis Cache on Kubernetes
#devoxxma
![Page 182: 50 Shades of Data - how, when and why Big,Relational,NoSQL,Elastic,Event,CQRS (Devoxx Maroc, November 2017) - including detailed demo screenshots](https://reader034.fdocuments.in/reader034/viewer/2022051710/5a6489fc7f8b9a27568b5f8b/html5/thumbnails/182.jpg)
Screenshots of Preparation of Cloud DemoEnvironments
#devoxxma
![Page 183: 50 Shades of Data - how, when and why Big,Relational,NoSQL,Elastic,Event,CQRS (Devoxx Maroc, November 2017) - including detailed demo screenshots](https://reader034.fdocuments.in/reader034/viewer/2022051710/5a6489fc7f8b9a27568b5f8b/html5/thumbnails/183.jpg)
Provision Oracle Database Cloud
#devoxxma
![Page 184: 50 Shades of Data - how, when and why Big,Relational,NoSQL,Elastic,Event,CQRS (Devoxx Maroc, November 2017) - including detailed demo screenshots](https://reader034.fdocuments.in/reader034/viewer/2022051710/5a6489fc7f8b9a27568b5f8b/html5/thumbnails/184.jpg)
Provision Database Cloud
#devoxxma
![Page 185: 50 Shades of Data - how, when and why Big,Relational,NoSQL,Elastic,Event,CQRS (Devoxx Maroc, November 2017) - including detailed demo screenshots](https://reader034.fdocuments.in/reader034/viewer/2022051710/5a6489fc7f8b9a27568b5f8b/html5/thumbnails/185.jpg)
Provision Database Cloud
#devoxxma
![Page 186: 50 Shades of Data - how, when and why Big,Relational,NoSQL,Elastic,Event,CQRS (Devoxx Maroc, November 2017) - including detailed demo screenshots](https://reader034.fdocuments.in/reader034/viewer/2022051710/5a6489fc7f8b9a27568b5f8b/html5/thumbnails/186.jpg)
Provision Database Cloud
#devoxxma
![Page 187: 50 Shades of Data - how, when and why Big,Relational,NoSQL,Elastic,Event,CQRS (Devoxx Maroc, November 2017) - including detailed demo screenshots](https://reader034.fdocuments.in/reader034/viewer/2022051710/5a6489fc7f8b9a27568b5f8b/html5/thumbnails/187.jpg)
Provision Database Cloud
#devoxxma
![Page 188: 50 Shades of Data - how, when and why Big,Relational,NoSQL,Elastic,Event,CQRS (Devoxx Maroc, November 2017) - including detailed demo screenshots](https://reader034.fdocuments.in/reader034/viewer/2022051710/5a6489fc7f8b9a27568b5f8b/html5/thumbnails/188.jpg)
Provision Database Cloud
#devoxxma
![Page 189: 50 Shades of Data - how, when and why Big,Relational,NoSQL,Elastic,Event,CQRS (Devoxx Maroc, November 2017) - including detailed demo screenshots](https://reader034.fdocuments.in/reader034/viewer/2022051710/5a6489fc7f8b9a27568b5f8b/html5/thumbnails/189.jpg)
Provision Database Cloud
#devoxxma
![Page 190: 50 Shades of Data - how, when and why Big,Relational,NoSQL,Elastic,Event,CQRS (Devoxx Maroc, November 2017) - including detailed demo screenshots](https://reader034.fdocuments.in/reader034/viewer/2022051710/5a6489fc7f8b9a27568b5f8b/html5/thumbnails/190.jpg)
Provision Database Cloud
#devoxxma
![Page 191: 50 Shades of Data - how, when and why Big,Relational,NoSQL,Elastic,Event,CQRS (Devoxx Maroc, November 2017) - including detailed demo screenshots](https://reader034.fdocuments.in/reader034/viewer/2022051710/5a6489fc7f8b9a27568b5f8b/html5/thumbnails/191.jpg)
Provision Database Cloud
#devoxxma
![Page 192: 50 Shades of Data - how, when and why Big,Relational,NoSQL,Elastic,Event,CQRS (Devoxx Maroc, November 2017) - including detailed demo screenshots](https://reader034.fdocuments.in/reader034/viewer/2022051710/5a6489fc7f8b9a27568b5f8b/html5/thumbnails/192.jpg)
Set Network Access Rulesfor Database Cloud
#devoxxma
![Page 193: 50 Shades of Data - how, when and why Big,Relational,NoSQL,Elastic,Event,CQRS (Devoxx Maroc, November 2017) - including detailed demo screenshots](https://reader034.fdocuments.in/reader034/viewer/2022051710/5a6489fc7f8b9a27568b5f8b/html5/thumbnails/193.jpg)
Set Network Access Rulesfor Database Cloud
#devoxxma
![Page 194: 50 Shades of Data - how, when and why Big,Relational,NoSQL,Elastic,Event,CQRS (Devoxx Maroc, November 2017) - including detailed demo screenshots](https://reader034.fdocuments.in/reader034/viewer/2022051710/5a6489fc7f8b9a27568b5f8b/html5/thumbnails/194.jpg)
Set Network Access Rulesfor Database Cloud
#devoxxma
![Page 195: 50 Shades of Data - how, when and why Big,Relational,NoSQL,Elastic,Event,CQRS (Devoxx Maroc, November 2017) - including detailed demo screenshots](https://reader034.fdocuments.in/reader034/viewer/2022051710/5a6489fc7f8b9a27568b5f8b/html5/thumbnails/195.jpg)
Access Oracle Databasefrom local SQL Developer
#devoxxma
![Page 196: 50 Shades of Data - how, when and why Big,Relational,NoSQL,Elastic,Event,CQRS (Devoxx Maroc, November 2017) - including detailed demo screenshots](https://reader034.fdocuments.in/reader034/viewer/2022051710/5a6489fc7f8b9a27568b5f8b/html5/thumbnails/196.jpg)
Access Database Cloudfrom local SQLcl
#devoxxma
![Page 197: 50 Shades of Data - how, when and why Big,Relational,NoSQL,Elastic,Event,CQRS (Devoxx Maroc, November 2017) - including detailed demo screenshots](https://reader034.fdocuments.in/reader034/viewer/2022051710/5a6489fc7f8b9a27568b5f8b/html5/thumbnails/197.jpg)
Provisioning ApplicationContainer Cloud
#devoxxma
![Page 198: 50 Shades of Data - how, when and why Big,Relational,NoSQL,Elastic,Event,CQRS (Devoxx Maroc, November 2017) - including detailed demo screenshots](https://reader034.fdocuments.in/reader034/viewer/2022051710/5a6489fc7f8b9a27568b5f8b/html5/thumbnails/198.jpg)
Provisioning ApplicationContainer Cloud
#devoxxma
![Page 199: 50 Shades of Data - how, when and why Big,Relational,NoSQL,Elastic,Event,CQRS (Devoxx Maroc, November 2017) - including detailed demo screenshots](https://reader034.fdocuments.in/reader034/viewer/2022051710/5a6489fc7f8b9a27568b5f8b/html5/thumbnails/199.jpg)
Provisioning ApplicationContainer Cloud
#devoxxma
![Page 200: 50 Shades of Data - how, when and why Big,Relational,NoSQL,Elastic,Event,CQRS (Devoxx Maroc, November 2017) - including detailed demo screenshots](https://reader034.fdocuments.in/reader034/viewer/2022051710/5a6489fc7f8b9a27568b5f8b/html5/thumbnails/200.jpg)
Provisioning ApplicationContainer Cloud
#devoxxma
![Page 201: 50 Shades of Data - how, when and why Big,Relational,NoSQL,Elastic,Event,CQRS (Devoxx Maroc, November 2017) - including detailed demo screenshots](https://reader034.fdocuments.in/reader034/viewer/2022051710/5a6489fc7f8b9a27568b5f8b/html5/thumbnails/201.jpg)
Provisioning ApplicationContainer Cloud
#devoxxma
![Page 202: 50 Shades of Data - how, when and why Big,Relational,NoSQL,Elastic,Event,CQRS (Devoxx Maroc, November 2017) - including detailed demo screenshots](https://reader034.fdocuments.in/reader034/viewer/2022051710/5a6489fc7f8b9a27568b5f8b/html5/thumbnails/202.jpg)
Provisioning ApplicationContainer Cloud
#devoxxma
![Page 203: 50 Shades of Data - how, when and why Big,Relational,NoSQL,Elastic,Event,CQRS (Devoxx Maroc, November 2017) - including detailed demo screenshots](https://reader034.fdocuments.in/reader034/viewer/2022051710/5a6489fc7f8b9a27568b5f8b/html5/thumbnails/203.jpg)
Provisioning ApplicationContainer Cloud
#devoxxma
![Page 204: 50 Shades of Data - how, when and why Big,Relational,NoSQL,Elastic,Event,CQRS (Devoxx Maroc, November 2017) - including detailed demo screenshots](https://reader034.fdocuments.in/reader034/viewer/2022051710/5a6489fc7f8b9a27568b5f8b/html5/thumbnails/204.jpg)
Provisioning ApplicationContainer Cloud
#devoxxma
![Page 205: 50 Shades of Data - how, when and why Big,Relational,NoSQL,Elastic,Event,CQRS (Devoxx Maroc, November 2017) - including detailed demo screenshots](https://reader034.fdocuments.in/reader034/viewer/2022051710/5a6489fc7f8b9a27568b5f8b/html5/thumbnails/205.jpg)
Provisioning ApplicationContainer Cloud
#devoxxma
![Page 206: 50 Shades of Data - how, when and why Big,Relational,NoSQL,Elastic,Event,CQRS (Devoxx Maroc, November 2017) - including detailed demo screenshots](https://reader034.fdocuments.in/reader034/viewer/2022051710/5a6489fc7f8b9a27568b5f8b/html5/thumbnails/206.jpg)
Running a request againstOrder Microservice
#devoxxma
![Page 207: 50 Shades of Data - how, when and why Big,Relational,NoSQL,Elastic,Event,CQRS (Devoxx Maroc, November 2017) - including detailed demo screenshots](https://reader034.fdocuments.in/reader034/viewer/2022051710/5a6489fc7f8b9a27568b5f8b/html5/thumbnails/207.jpg)
Access Application Container logfiles through desktop Cloud Berry
#devoxxma
![Page 208: 50 Shades of Data - how, when and why Big,Relational,NoSQL,Elastic,Event,CQRS (Devoxx Maroc, November 2017) - including detailed demo screenshots](https://reader034.fdocuments.in/reader034/viewer/2022051710/5a6489fc7f8b9a27568b5f8b/html5/thumbnails/208.jpg)
Add Service Binding(to DBaaS)
#devoxxma
![Page 209: 50 Shades of Data - how, when and why Big,Relational,NoSQL,Elastic,Event,CQRS (Devoxx Maroc, November 2017) - including detailed demo screenshots](https://reader034.fdocuments.in/reader034/viewer/2022051710/5a6489fc7f8b9a27568b5f8b/html5/thumbnails/209.jpg)
Add Service Binding(to DBaaS)
#devoxxma
![Page 210: 50 Shades of Data - how, when and why Big,Relational,NoSQL,Elastic,Event,CQRS (Devoxx Maroc, November 2017) - including detailed demo screenshots](https://reader034.fdocuments.in/reader034/viewer/2022051710/5a6489fc7f8b9a27568b5f8b/html5/thumbnails/210.jpg)
After adding Service Binding(to DBaaS)
#devoxxma
![Page 211: 50 Shades of Data - how, when and why Big,Relational,NoSQL,Elastic,Event,CQRS (Devoxx Maroc, November 2017) - including detailed demo screenshots](https://reader034.fdocuments.in/reader034/viewer/2022051710/5a6489fc7f8b9a27568b5f8b/html5/thumbnails/211.jpg)
Demo – Call Ordermicroservice and createstate
#devoxxma
APIUI
Order microservice
RDBMSDVX_ORDERS
tableOracle
Database Cloud
Order JSON
![Page 212: 50 Shades of Data - how, when and why Big,Relational,NoSQL,Elastic,Event,CQRS (Devoxx Maroc, November 2017) - including detailed demo screenshots](https://reader034.fdocuments.in/reader034/viewer/2022051710/5a6489fc7f8b9a27568b5f8b/html5/thumbnails/212.jpg)
Call microservice to createan order
#devoxxma
![Page 213: 50 Shades of Data - how, when and why Big,Relational,NoSQL,Elastic,Event,CQRS (Devoxx Maroc, November 2017) - including detailed demo screenshots](https://reader034.fdocuments.in/reader034/viewer/2022051710/5a6489fc7f8b9a27568b5f8b/html5/thumbnails/213.jpg)
Verify the state of theDVX_ORDERS table
#devoxxma
![Page 214: 50 Shades of Data - how, when and why Big,Relational,NoSQL,Elastic,Event,CQRS (Devoxx Maroc, November 2017) - including detailed demo screenshots](https://reader034.fdocuments.in/reader034/viewer/2022051710/5a6489fc7f8b9a27568b5f8b/html5/thumbnails/214.jpg)
ProvisionedOracle Event Hub Cloud
#devoxxma
![Page 215: 50 Shades of Data - how, when and why Big,Relational,NoSQL,Elastic,Event,CQRS (Devoxx Maroc, November 2017) - including detailed demo screenshots](https://reader034.fdocuments.in/reader034/viewer/2022051710/5a6489fc7f8b9a27568b5f8b/html5/thumbnails/215.jpg)
ProvisionedOracle Event Hub Cloud
#devoxxma
![Page 216: 50 Shades of Data - how, when and why Big,Relational,NoSQL,Elastic,Event,CQRS (Devoxx Maroc, November 2017) - including detailed demo screenshots](https://reader034.fdocuments.in/reader034/viewer/2022051710/5a6489fc7f8b9a27568b5f8b/html5/thumbnails/216.jpg)
Set Network Access Rulesfor Oracle Event Hub Cloud
#devoxxma
![Page 217: 50 Shades of Data - how, when and why Big,Relational,NoSQL,Elastic,Event,CQRS (Devoxx Maroc, November 2017) - including detailed demo screenshots](https://reader034.fdocuments.in/reader034/viewer/2022051710/5a6489fc7f8b9a27568b5f8b/html5/thumbnails/217.jpg)
Set Network Access Rulesfor Oracle Event Hub Cloud
#devoxxma
![Page 218: 50 Shades of Data - how, when and why Big,Relational,NoSQL,Elastic,Event,CQRS (Devoxx Maroc, November 2017) - including detailed demo screenshots](https://reader034.fdocuments.in/reader034/viewer/2022051710/5a6489fc7f8b9a27568b5f8b/html5/thumbnails/218.jpg)
Set Network Access Rulesfor Oracle Event Hub Cloud
#devoxxma
![Page 219: 50 Shades of Data - how, when and why Big,Relational,NoSQL,Elastic,Event,CQRS (Devoxx Maroc, November 2017) - including detailed demo screenshots](https://reader034.fdocuments.in/reader034/viewer/2022051710/5a6489fc7f8b9a27568b5f8b/html5/thumbnails/219.jpg)
Provision Oracle Event Hub Cloud
Topic
#devoxxma
![Page 220: 50 Shades of Data - how, when and why Big,Relational,NoSQL,Elastic,Event,CQRS (Devoxx Maroc, November 2017) - including detailed demo screenshots](https://reader034.fdocuments.in/reader034/viewer/2022051710/5a6489fc7f8b9a27568b5f8b/html5/thumbnails/220.jpg)
Provision Oracle Event Hub Cloud
Topic
#devoxxma
![Page 221: 50 Shades of Data - how, when and why Big,Relational,NoSQL,Elastic,Event,CQRS (Devoxx Maroc, November 2017) - including detailed demo screenshots](https://reader034.fdocuments.in/reader034/viewer/2022051710/5a6489fc7f8b9a27568b5f8b/html5/thumbnails/221.jpg)
Provision Oracle Event Hub Cloud
#devoxxma
![Page 222: 50 Shades of Data - how, when and why Big,Relational,NoSQL,Elastic,Event,CQRS (Devoxx Maroc, November 2017) - including detailed demo screenshots](https://reader034.fdocuments.in/reader034/viewer/2022051710/5a6489fc7f8b9a27568b5f8b/html5/thumbnails/222.jpg)
Provision Oracle Event Hub Cloud
#devoxxma
![Page 223: 50 Shades of Data - how, when and why Big,Relational,NoSQL,Elastic,Event,CQRS (Devoxx Maroc, November 2017) - including detailed demo screenshots](https://reader034.fdocuments.in/reader034/viewer/2022051710/5a6489fc7f8b9a27568b5f8b/html5/thumbnails/223.jpg)
Provision Oracle Event Hub Cloud
#devoxxma
![Page 224: 50 Shades of Data - how, when and why Big,Relational,NoSQL,Elastic,Event,CQRS (Devoxx Maroc, November 2017) - including detailed demo screenshots](https://reader034.fdocuments.in/reader034/viewer/2022051710/5a6489fc7f8b9a27568b5f8b/html5/thumbnails/224.jpg)
Create Service Binding for OrderMicroservice to Event Hub
#devoxxma
APIUI
Order
RDBMS Oracle Database Cloud
EventHubOracle Event Hub Cloud
NewOrder event
![Page 225: 50 Shades of Data - how, when and why Big,Relational,NoSQL,Elastic,Event,CQRS (Devoxx Maroc, November 2017) - including detailed demo screenshots](https://reader034.fdocuments.in/reader034/viewer/2022051710/5a6489fc7f8b9a27568b5f8b/html5/thumbnails/225.jpg)
Create Service Binding for OrderMicroservice to Event Hub
#devoxxma
![Page 226: 50 Shades of Data - how, when and why Big,Relational,NoSQL,Elastic,Event,CQRS (Devoxx Maroc, November 2017) - including detailed demo screenshots](https://reader034.fdocuments.in/reader034/viewer/2022051710/5a6489fc7f8b9a27568b5f8b/html5/thumbnails/226.jpg)
Create Service Binding for OrderMicroservice to Event Hub
#devoxxma
![Page 227: 50 Shades of Data - how, when and why Big,Relational,NoSQL,Elastic,Event,CQRS (Devoxx Maroc, November 2017) - including detailed demo screenshots](https://reader034.fdocuments.in/reader034/viewer/2022051710/5a6489fc7f8b9a27568b5f8b/html5/thumbnails/227.jpg)
Provision Oracle Application Container Cache
#devoxxma
![Page 228: 50 Shades of Data - how, when and why Big,Relational,NoSQL,Elastic,Event,CQRS (Devoxx Maroc, November 2017) - including detailed demo screenshots](https://reader034.fdocuments.in/reader034/viewer/2022051710/5a6489fc7f8b9a27568b5f8b/html5/thumbnails/228.jpg)
Provision Oracle Application Container Cache
#devoxxma
![Page 229: 50 Shades of Data - how, when and why Big,Relational,NoSQL,Elastic,Event,CQRS (Devoxx Maroc, November 2017) - including detailed demo screenshots](https://reader034.fdocuments.in/reader034/viewer/2022051710/5a6489fc7f8b9a27568b5f8b/html5/thumbnails/229.jpg)
Provision Oracle Application Container Cache
#devoxxma
![Page 230: 50 Shades of Data - how, when and why Big,Relational,NoSQL,Elastic,Event,CQRS (Devoxx Maroc, November 2017) - including detailed demo screenshots](https://reader034.fdocuments.in/reader034/viewer/2022051710/5a6489fc7f8b9a27568b5f8b/html5/thumbnails/230.jpg)
Provision Oracle Application Container Cache
#devoxxma
![Page 231: 50 Shades of Data - how, when and why Big,Relational,NoSQL,Elastic,Event,CQRS (Devoxx Maroc, November 2017) - including detailed demo screenshots](https://reader034.fdocuments.in/reader034/viewer/2022051710/5a6489fc7f8b9a27568b5f8b/html5/thumbnails/231.jpg)
Provision Oracle Application Container Cache
#devoxxma
![Page 232: 50 Shades of Data - how, when and why Big,Relational,NoSQL,Elastic,Event,CQRS (Devoxx Maroc, November 2017) - including detailed demo screenshots](https://reader034.fdocuments.in/reader034/viewer/2022051710/5a6489fc7f8b9a27568b5f8b/html5/thumbnails/232.jpg)
Deploying Shopping Cartmicroservice with Application Cache
#devoxxma
Cache
Application Container Cache
APIUI
Microservice ShoppingCart& Check Out
APIUI
Order
RDBMSOracle
Database Cloud
![Page 233: 50 Shades of Data - how, when and why Big,Relational,NoSQL,Elastic,Event,CQRS (Devoxx Maroc, November 2017) - including detailed demo screenshots](https://reader034.fdocuments.in/reader034/viewer/2022051710/5a6489fc7f8b9a27568b5f8b/html5/thumbnails/233.jpg)
Provision Node application onOracle Application Container
#devoxxma
![Page 234: 50 Shades of Data - how, when and why Big,Relational,NoSQL,Elastic,Event,CQRS (Devoxx Maroc, November 2017) - including detailed demo screenshots](https://reader034.fdocuments.in/reader034/viewer/2022051710/5a6489fc7f8b9a27568b5f8b/html5/thumbnails/234.jpg)
Inject Environment Variable intoOracle Application Container app
#devoxxma
![Page 235: 50 Shades of Data - how, when and why Big,Relational,NoSQL,Elastic,Event,CQRS (Devoxx Maroc, November 2017) - including detailed demo screenshots](https://reader034.fdocuments.in/reader034/viewer/2022051710/5a6489fc7f8b9a27568b5f8b/html5/thumbnails/235.jpg)
Inject Environment Variable intoOracle Application Container app
#devoxxma
![Page 236: 50 Shades of Data - how, when and why Big,Relational,NoSQL,Elastic,Event,CQRS (Devoxx Maroc, November 2017) - including detailed demo screenshots](https://reader034.fdocuments.in/reader034/viewer/2022051710/5a6489fc7f8b9a27568b5f8b/html5/thumbnails/236.jpg)
Create MongoDB instance on MLab Cloud
#devoxxma
![Page 237: 50 Shades of Data - how, when and why Big,Relational,NoSQL,Elastic,Event,CQRS (Devoxx Maroc, November 2017) - including detailed demo screenshots](https://reader034.fdocuments.in/reader034/viewer/2022051710/5a6489fc7f8b9a27568b5f8b/html5/thumbnails/237.jpg)
Create MongoDB database& collection Customers
#devoxxma
![Page 238: 50 Shades of Data - how, when and why Big,Relational,NoSQL,Elastic,Event,CQRS (Devoxx Maroc, November 2017) - including detailed demo screenshots](https://reader034.fdocuments.in/reader034/viewer/2022051710/5a6489fc7f8b9a27568b5f8b/html5/thumbnails/238.jpg)
MongoDB collection Customers
#devoxxma
![Page 239: 50 Shades of Data - how, when and why Big,Relational,NoSQL,Elastic,Event,CQRS (Devoxx Maroc, November 2017) - including detailed demo screenshots](https://reader034.fdocuments.in/reader034/viewer/2022051710/5a6489fc7f8b9a27568b5f8b/html5/thumbnails/239.jpg)
Create Customer Microservice
#devoxxma
API
Customer
EventHubOracle Event Hub Cloud
![Page 240: 50 Shades of Data - how, when and why Big,Relational,NoSQL,Elastic,Event,CQRS (Devoxx Maroc, November 2017) - including detailed demo screenshots](https://reader034.fdocuments.in/reader034/viewer/2022051710/5a6489fc7f8b9a27568b5f8b/html5/thumbnails/240.jpg)
Run local Kafka Cluster
#devoxxma
Event Bus
![Page 241: 50 Shades of Data - how, when and why Big,Relational,NoSQL,Elastic,Event,CQRS (Devoxx Maroc, November 2017) - including detailed demo screenshots](https://reader034.fdocuments.in/reader034/viewer/2022051710/5a6489fc7f8b9a27568b5f8b/html5/thumbnails/241.jpg)
Run local Kafka Cluster
#devoxxma
Event Bus
![Page 242: 50 Shades of Data - how, when and why Big,Relational,NoSQL,Elastic,Event,CQRS (Devoxx Maroc, November 2017) - including detailed demo screenshots](https://reader034.fdocuments.in/reader034/viewer/2022051710/5a6489fc7f8b9a27568b5f8b/html5/thumbnails/242.jpg)
Run local KubernetesCluster (Minikube)
#devoxxma
WorkflowLauncher
Check OrderTotal
Check Shipping
Cache CacheInspector
LogMonitorOrderArbiter
![Page 243: 50 Shades of Data - how, when and why Big,Relational,NoSQL,Elastic,Event,CQRS (Devoxx Maroc, November 2017) - including detailed demo screenshots](https://reader034.fdocuments.in/reader034/viewer/2022051710/5a6489fc7f8b9a27568b5f8b/html5/thumbnails/243.jpg)
Run local KubernetesCluster (Minikube)
• Start minikube
• Get minikube IP address
• Apply yaml files for Pods, Deployments and Services
• Open minikube dashboard
• Update minikube IP address and service ports in Postman
#devoxxma
![Page 244: 50 Shades of Data - how, when and why Big,Relational,NoSQL,Elastic,Event,CQRS (Devoxx Maroc, November 2017) - including detailed demo screenshots](https://reader034.fdocuments.in/reader034/viewer/2022051710/5a6489fc7f8b9a27568b5f8b/html5/thumbnails/244.jpg)
Run local KubernetesCluster (Minikube)
#devoxxma
![Page 245: 50 Shades of Data - how, when and why Big,Relational,NoSQL,Elastic,Event,CQRS (Devoxx Maroc, November 2017) - including detailed demo screenshots](https://reader034.fdocuments.in/reader034/viewer/2022051710/5a6489fc7f8b9a27568b5f8b/html5/thumbnails/245.jpg)
Run local KubernetesCluster (Minikube)
#devoxxma
![Page 246: 50 Shades of Data - how, when and why Big,Relational,NoSQL,Elastic,Event,CQRS (Devoxx Maroc, November 2017) - including detailed demo screenshots](https://reader034.fdocuments.in/reader034/viewer/2022051710/5a6489fc7f8b9a27568b5f8b/html5/thumbnails/246.jpg)
Run local KubernetesCluster (Minikube)
#devoxxma
![Page 247: 50 Shades of Data - how, when and why Big,Relational,NoSQL,Elastic,Event,CQRS (Devoxx Maroc, November 2017) - including detailed demo screenshots](https://reader034.fdocuments.in/reader034/viewer/2022051710/5a6489fc7f8b9a27568b5f8b/html5/thumbnails/247.jpg)
Open KubernetesDashboard
#devoxxma
![Page 248: 50 Shades of Data - how, when and why Big,Relational,NoSQL,Elastic,Event,CQRS (Devoxx Maroc, November 2017) - including detailed demo screenshots](https://reader034.fdocuments.in/reader034/viewer/2022051710/5a6489fc7f8b9a27568b5f8b/html5/thumbnails/248.jpg)
Show service endpoints inKubernetes Dashboard
#devoxxma
![Page 249: 50 Shades of Data - how, when and why Big,Relational,NoSQL,Elastic,Event,CQRS (Devoxx Maroc, November 2017) - including detailed demo screenshots](https://reader034.fdocuments.in/reader034/viewer/2022051710/5a6489fc7f8b9a27568b5f8b/html5/thumbnails/249.jpg)
Update Cluster IP and Service Endpoints in Postman
#devoxxma
![Page 250: 50 Shades of Data - how, when and why Big,Relational,NoSQL,Elastic,Event,CQRS (Devoxx Maroc, November 2017) - including detailed demo screenshots](https://reader034.fdocuments.in/reader034/viewer/2022051710/5a6489fc7f8b9a27568b5f8b/html5/thumbnails/250.jpg)
Run EventBridgemicroservice
#devoxxma
Event Bus
EventBridge
EventHubOracle Event Hub Cloud
![Page 251: 50 Shades of Data - how, when and why Big,Relational,NoSQL,Elastic,Event,CQRS (Devoxx Maroc, November 2017) - including detailed demo screenshots](https://reader034.fdocuments.in/reader034/viewer/2022051710/5a6489fc7f8b9a27568b5f8b/html5/thumbnails/251.jpg)
Run Customermicroservice
#devoxxma
API
Customer
EventHubOracle Event Hub Cloud
![Page 252: 50 Shades of Data - how, when and why Big,Relational,NoSQL,Elastic,Event,CQRS (Devoxx Maroc, November 2017) - including detailed demo screenshots](https://reader034.fdocuments.in/reader034/viewer/2022051710/5a6489fc7f8b9a27568b5f8b/html5/thumbnails/252.jpg)
Test Customermicroservice
#devoxxma
API
Customer