Demi Ben-Ari - Monitoring Big Data Systems Done "The Simple Way" - Codemotion Milan 2017
-
Upload
codemotion -
Category
Technology
-
view
82 -
download
0
Transcript of Demi Ben-Ari - Monitoring Big Data Systems Done "The Simple Way" - Codemotion Milan 2017
![Page 1: Demi Ben-Ari - Monitoring Big Data Systems Done "The Simple Way" - Codemotion Milan 2017](https://reader033.fdocuments.in/reader033/viewer/2022051710/5a6479a17f8b9a36568b4793/html5/thumbnails/1.jpg)
Monitoring Big Data Systems - Done “The Simple Way”Demi Ben-Ari - VP R&D @
CODEMOTION MILAN - SPECIAL EDITION 10 – 11 NOVEMBER 2017
![Page 2: Demi Ben-Ari - Monitoring Big Data Systems Done "The Simple Way" - Codemotion Milan 2017](https://reader033.fdocuments.in/reader033/viewer/2022051710/5a6479a17f8b9a36568b4793/html5/thumbnails/2.jpg)
About Me
Demi Ben-Ari, Co-Founder & VP R&D @ Panorays● Google Developer Expert● Co-Founder of Communities:
○ “Big Things” - Big Data, Data Science, DevOps○ Google Developer Group Cloud○ Ofek Alumni Association
In the Past:● Sr. Data Engineer - Windward● Team Leader & Sr. Java Software Engineer,
Missile defense System - “Ofek” – IAF
![Page 3: Demi Ben-Ari - Monitoring Big Data Systems Done "The Simple Way" - Codemotion Milan 2017](https://reader033.fdocuments.in/reader033/viewer/2022051710/5a6479a17f8b9a36568b4793/html5/thumbnails/3.jpg)
Agenda
● A lot of (NOT) funny Jokes● Problem definition and Environment● Monitoring pipeline solutions
○ Metrics○ Datastores○ Dashboards○ Alerting
● Summary● (Not going to address Service discovery and monitoring)
![Page 4: Demi Ben-Ari - Monitoring Big Data Systems Done "The Simple Way" - Codemotion Milan 2017](https://reader033.fdocuments.in/reader033/viewer/2022051710/5a6479a17f8b9a36568b4793/html5/thumbnails/4.jpg)
Say “Distributed”, Say “Big Data”,Say….
![Page 5: Demi Ben-Ari - Monitoring Big Data Systems Done "The Simple Way" - Codemotion Milan 2017](https://reader033.fdocuments.in/reader033/viewer/2022051710/5a6479a17f8b9a36568b4793/html5/thumbnails/5.jpg)
What is Big Data (IMHO)? And What to Monitor?
● Systems involving the “3 Vs”:
What are the right questions we want to ask?
○ Volume - How much?
○ Velocity - How fast?
○ Variety - What kind? (Difference)
![Page 6: Demi Ben-Ari - Monitoring Big Data Systems Done "The Simple Way" - Codemotion Milan 2017](https://reader033.fdocuments.in/reader033/viewer/2022051710/5a6479a17f8b9a36568b4793/html5/thumbnails/6.jpg)
Monitor What?
Product
Data
Infrastructure
Biz
![Page 7: Demi Ben-Ari - Monitoring Big Data Systems Done "The Simple Way" - Codemotion Milan 2017](https://reader033.fdocuments.in/reader033/viewer/2022051710/5a6479a17f8b9a36568b4793/html5/thumbnails/7.jpg)
Monolith Structure
OS CPU Memory Disk
Processes Java Application Server
Database
Web Server
Load Balancer
Users - Other Applications
Monitoring System
UI
Many times...all of this was on a single physical server!
![Page 8: Demi Ben-Ari - Monitoring Big Data Systems Done "The Simple Way" - Codemotion Milan 2017](https://reader033.fdocuments.in/reader033/viewer/2022051710/5a6479a17f8b9a36568b4793/html5/thumbnails/8.jpg)
Distributed Microservices Architecture
Service A
Queue
DB
Service B
DBCache
Cache DBService C
Web Server
DB
Analytics Cluster
Master
Slave Slave Slave
Monitoring System???
![Page 9: Demi Ben-Ari - Monitoring Big Data Systems Done "The Simple Way" - Codemotion Milan 2017](https://reader033.fdocuments.in/reader033/viewer/2022051710/5a6479a17f8b9a36568b4793/html5/thumbnails/9.jpg)
Some basic concepts
![Page 10: Demi Ben-Ari - Monitoring Big Data Systems Done "The Simple Way" - Codemotion Milan 2017](https://reader033.fdocuments.in/reader033/viewer/2022051710/5a6479a17f8b9a36568b4793/html5/thumbnails/10.jpg)
Basic Concepts
● Monitoring● White-box
○ internals● Black-box
○ behavior● Dashboard● Alert
● Root cause● Node and machine● Deploy
○ Any change to a service’s running software or its configuration.
● KPI - Key Performance Indicator
![Page 11: Demi Ben-Ari - Monitoring Big Data Systems Done "The Simple Way" - Codemotion Milan 2017](https://reader033.fdocuments.in/reader033/viewer/2022051710/5a6479a17f8b9a36568b4793/html5/thumbnails/11.jpg)
Data flow and Environment(Use Case)
![Page 12: Demi Ben-Ari - Monitoring Big Data Systems Done "The Simple Way" - Codemotion Milan 2017](https://reader033.fdocuments.in/reader033/viewer/2022051710/5a6479a17f8b9a36568b4793/html5/thumbnails/12.jpg)
Structure of the Data
● Maritime Analytics Platform
● Geo Locations + Metadata
● Arriving over time
● Different types of messages being reported by satellites
● Encoded (For compression reasons)
● Might arrive later than actually transmitted
![Page 13: Demi Ben-Ari - Monitoring Big Data Systems Done "The Simple Way" - Codemotion Milan 2017](https://reader033.fdocuments.in/reader033/viewer/2022051710/5a6479a17f8b9a36568b4793/html5/thumbnails/13.jpg)
Data Flow Diagram
External Data
Source
Analytics Layers
Data Pipeline
Parsed Raw
Entity Resolution Process
Building insightson top of the entities
Data Output Layer
Anomaly Detection
Trends
UI for End Users
![Page 14: Demi Ben-Ari - Monitoring Big Data Systems Done "The Simple Way" - Codemotion Milan 2017](https://reader033.fdocuments.in/reader033/viewer/2022051710/5a6479a17f8b9a36568b4793/html5/thumbnails/14.jpg)
Environment Description
Cluster
Dev Testing Live Staging ProductionEnv
OB1K
RESTful Java Services
![Page 15: Demi Ben-Ari - Monitoring Big Data Systems Done "The Simple Way" - Codemotion Milan 2017](https://reader033.fdocuments.in/reader033/viewer/2022051710/5a6479a17f8b9a36568b4793/html5/thumbnails/15.jpg)
Monitoring Stack - Let’s fill in the blanks
Alerting
Metrics Collection
Datastore
Dashboard
Data Monitoring
Log Monitoring
![Page 16: Demi Ben-Ari - Monitoring Big Data Systems Done "The Simple Way" - Codemotion Milan 2017](https://reader033.fdocuments.in/reader033/viewer/2022051710/5a6479a17f8b9a36568b4793/html5/thumbnails/16.jpg)
Situations & Problems
https://imgflip.com/i/1ap5krhttp://kingofwallpapers.com/otter/otter-004.jpg
![Page 17: Demi Ben-Ari - Monitoring Big Data Systems Done "The Simple Way" - Codemotion Milan 2017](https://reader033.fdocuments.in/reader033/viewer/2022051710/5a6479a17f8b9a36568b4793/html5/thumbnails/17.jpg)
MongoDB + Spark
Worker 1
Worker 2
….
….
…
…Worker N
Spark Cluster
Master
Write
Read
MasterSharded MongoDB
Replica Set
![Page 18: Demi Ben-Ari - Monitoring Big Data Systems Done "The Simple Way" - Codemotion Milan 2017](https://reader033.fdocuments.in/reader033/viewer/2022051710/5a6479a17f8b9a36568b4793/html5/thumbnails/18.jpg)
Cassandra + Spark
Worker 1
Worker 2
….
….
…
…Worker N
Cassandra Cluster
Spark Cluster
Write
Read
![Page 19: Demi Ben-Ari - Monitoring Big Data Systems Done "The Simple Way" - Codemotion Milan 2017](https://reader033.fdocuments.in/reader033/viewer/2022051710/5a6479a17f8b9a36568b4793/html5/thumbnails/19.jpg)
Cassandra + Serving
Cassandra Cluster
Write
Read
UI ClientUI Client
UI ClientUI Client
Web ServiceWeb
ServiceWeb ServiceWeb
Service
![Page 20: Demi Ben-Ari - Monitoring Big Data Systems Done "The Simple Way" - Codemotion Milan 2017](https://reader033.fdocuments.in/reader033/viewer/2022051710/5a6479a17f8b9a36568b4793/html5/thumbnails/20.jpg)
Problems
● Multiple physical servers
● Multiple logical services
● Want Scaling => More Servers
● Even if you had all of the metrics
○ You’ll have an overflow of the data
● Your monitoring becomes a “Big Data” problem itself
![Page 21: Demi Ben-Ari - Monitoring Big Data Systems Done "The Simple Way" - Codemotion Milan 2017](https://reader033.fdocuments.in/reader033/viewer/2022051710/5a6479a17f8b9a36568b4793/html5/thumbnails/21.jpg)
This is what “Distributed” really Means
The DevOps Guy
(It might be you)
![Page 22: Demi Ben-Ari - Monitoring Big Data Systems Done "The Simple Way" - Codemotion Milan 2017](https://reader033.fdocuments.in/reader033/viewer/2022051710/5a6479a17f8b9a36568b4793/html5/thumbnails/22.jpg)
So...SolutionsLet’s Start!
![Page 23: Demi Ben-Ari - Monitoring Big Data Systems Done "The Simple Way" - Codemotion Milan 2017](https://reader033.fdocuments.in/reader033/viewer/2022051710/5a6479a17f8b9a36568b4793/html5/thumbnails/23.jpg)
Monitoring Operation System
![Page 24: Demi Ben-Ari - Monitoring Big Data Systems Done "The Simple Way" - Codemotion Milan 2017](https://reader033.fdocuments.in/reader033/viewer/2022051710/5a6479a17f8b9a36568b4793/html5/thumbnails/24.jpg)
Monitoring Operation System Metrics
● What to measure:○ CPU○ Memory○ Disk Space
● How to measure:○ CollectD or StatsD reporting to Graphite○ New Relic
■ Nice and easy UI■ Even the free account gives great tool■ Alerting of thresholds
![Page 25: Demi Ben-Ari - Monitoring Big Data Systems Done "The Simple Way" - Codemotion Milan 2017](https://reader033.fdocuments.in/reader033/viewer/2022051710/5a6479a17f8b9a36568b4793/html5/thumbnails/25.jpg)
Some help from “the Cloud”
![Page 26: Demi Ben-Ari - Monitoring Big Data Systems Done "The Simple Way" - Codemotion Milan 2017](https://reader033.fdocuments.in/reader033/viewer/2022051710/5a6479a17f8b9a36568b4793/html5/thumbnails/26.jpg)
Google Cloud Platform StackDriver
![Page 27: Demi Ben-Ari - Monitoring Big Data Systems Done "The Simple Way" - Codemotion Milan 2017](https://reader033.fdocuments.in/reader033/viewer/2022051710/5a6479a17f8b9a36568b4793/html5/thumbnails/27.jpg)
Amazon Web Services CloudWatch
![Page 28: Demi Ben-Ari - Monitoring Big Data Systems Done "The Simple Way" - Codemotion Milan 2017](https://reader033.fdocuments.in/reader033/viewer/2022051710/5a6479a17f8b9a36568b4793/html5/thumbnails/28.jpg)
Report to Where?
● We chose: ● Graphite (InfluxDB) + Grafana● Can correlate System and
Application metrics in one place :)
![Page 29: Demi Ben-Ari - Monitoring Big Data Systems Done "The Simple Way" - Codemotion Milan 2017](https://reader033.fdocuments.in/reader033/viewer/2022051710/5a6479a17f8b9a36568b4793/html5/thumbnails/29.jpg)
Report to Where?
● Save DevOps efforts if you’re willing to Pay :)● Hosted Graphite
○ https://www.hostedgraphite.com/● Throwing the “Big Data” volume monitoring problem at
someone else
![Page 30: Demi Ben-Ari - Monitoring Big Data Systems Done "The Simple Way" - Codemotion Milan 2017](https://reader033.fdocuments.in/reader033/viewer/2022051710/5a6479a17f8b9a36568b4793/html5/thumbnails/30.jpg)
Connections Connections…
http://www.mememaker.net/meme/connections-connections-everywhere2/
![Page 31: Demi Ben-Ari - Monitoring Big Data Systems Done "The Simple Way" - Codemotion Milan 2017](https://reader033.fdocuments.in/reader033/viewer/2022051710/5a6479a17f8b9a36568b4793/html5/thumbnails/31.jpg)
Drivers to Datastores
● Actions they usually do:○ Open connection○ Apply actions
■ Select, Insert, Update, Delete○ Close connection
● Do you monitor each?○ Hint: Yes!!!! Hell Yes!!!
● Creating a wrapper in any programming language and reporting the metrics○ Count, execution times, errors…○ Infrastructure code that will give great visibility
![Page 32: Demi Ben-Ari - Monitoring Big Data Systems Done "The Simple Way" - Codemotion Milan 2017](https://reader033.fdocuments.in/reader033/viewer/2022051710/5a6479a17f8b9a36568b4793/html5/thumbnails/32.jpg)
Monitoring Cassandra
![Page 33: Demi Ben-Ari - Monitoring Big Data Systems Done "The Simple Way" - Codemotion Milan 2017](https://reader033.fdocuments.in/reader033/viewer/2022051710/5a6479a17f8b9a36568b4793/html5/thumbnails/33.jpg)
Monitoring Cassandra
● OpsCenter - by DataStax
![Page 34: Demi Ben-Ari - Monitoring Big Data Systems Done "The Simple Way" - Codemotion Milan 2017](https://reader033.fdocuments.in/reader033/viewer/2022051710/5a6479a17f8b9a36568b4793/html5/thumbnails/34.jpg)
Monitoring Cassandra
● Is the enough?...
We can connect it to Graphite also (Blog: “Monitoring the
hell out of Cassandra”)
● Plug & Play the metrics to Graphite - Internal Cassandra
mechanism
● Back to the basics: dstat, iostat, iotop, jstack
![Page 35: Demi Ben-Ari - Monitoring Big Data Systems Done "The Simple Way" - Codemotion Milan 2017](https://reader033.fdocuments.in/reader033/viewer/2022051710/5a6479a17f8b9a36568b4793/html5/thumbnails/35.jpg)
Monitoring Cassandra
![Page 36: Demi Ben-Ari - Monitoring Big Data Systems Done "The Simple Way" - Codemotion Milan 2017](https://reader033.fdocuments.in/reader033/viewer/2022051710/5a6479a17f8b9a36568b4793/html5/thumbnails/36.jpg)
Monitoring Spark
![Page 37: Demi Ben-Ari - Monitoring Big Data Systems Done "The Simple Way" - Codemotion Milan 2017](https://reader033.fdocuments.in/reader033/viewer/2022051710/5a6479a17f8b9a36568b4793/html5/thumbnails/37.jpg)
What to monitor in an Apache Spark Cluster
● Application execution
● Resource consumption and allocation
● Task Failures
● Environment and Amount of servers
● Physical OS metrics
● Infrastructure services
![Page 38: Demi Ben-Ari - Monitoring Big Data Systems Done "The Simple Way" - Codemotion Milan 2017](https://reader033.fdocuments.in/reader033/viewer/2022051710/5a6479a17f8b9a36568b4793/html5/thumbnails/38.jpg)
Ways to Monitoring Spark● Grafana-spark-dashboards
○ Blog: http://www.hammerlab.org/2015/02/27/monitoring-spark-with-graphite-and-grafana/
● Spark UI - Online on each application running● Spark History Server - Offline (After application finishes)● Spark REST API
○ Querying via inner tools to do ad-hoc monitoring● Back to the basics: dstat, iostat, iotop, jstack● Blog post by Tzach Zohar - “Tips from the Trenches”● http://spark.apache.org/docs/latest/monitoring.html
![Page 39: Demi Ben-Ari - Monitoring Big Data Systems Done "The Simple Way" - Codemotion Milan 2017](https://reader033.fdocuments.in/reader033/viewer/2022051710/5a6479a17f8b9a36568b4793/html5/thumbnails/39.jpg)
Monitoring Your Data
https://memegenerator.net/instance/53617544
![Page 40: Demi Ben-Ari - Monitoring Big Data Systems Done "The Simple Way" - Codemotion Milan 2017](https://reader033.fdocuments.in/reader033/viewer/2022051710/5a6479a17f8b9a36568b4793/html5/thumbnails/40.jpg)
Data Questions? What should be measure
● Did all of the computation occur?
○ Are there any data layers missing?
● How much data do we have? (Volume)
● Is all of the data in the Database?
● Data Quality Assurance
![Page 41: Demi Ben-Ari - Monitoring Big Data Systems Done "The Simple Way" - Codemotion Milan 2017](https://reader033.fdocuments.in/reader033/viewer/2022051710/5a6479a17f8b9a36568b4793/html5/thumbnails/41.jpg)
Data Answers!● KISS (Keep it simple stupid)
● Jenkins + Maven (JUnit) for the rescue
● Creating a maven “monitoring” project.
○ Running scheduled tasks, each for the relevant data source
■ Database data existence
■ S3 files existence
■ Data flow that keeps on coming from sensors
■ (Any other data source that you can imagine…)
○ Scheduled task that write amount metrics to Graphite -> Dashboards
○ Report task execution to Graphite
![Page 42: Demi Ben-Ari - Monitoring Big Data Systems Done "The Simple Way" - Codemotion Milan 2017](https://reader033.fdocuments.in/reader033/viewer/2022051710/5a6479a17f8b9a36568b4793/html5/thumbnails/42.jpg)
Data Answers!● The method doesn’t really matter, as long as you:
○ Can follow the results over time
○ Know what your data flow, know what might fail
○ It’s easy for anyone to add more monitoring
(For the ones that add the new data each time…)
○ It don’t trust others to add monitoring
(It will always end up the DevOps’s “fault” -> No monitoring will be
applied)
![Page 43: Demi Ben-Ari - Monitoring Big Data Systems Done "The Simple Way" - Codemotion Milan 2017](https://reader033.fdocuments.in/reader033/viewer/2022051710/5a6479a17f8b9a36568b4793/html5/thumbnails/43.jpg)
Logging?Monitoring?
https://lh4.googleusercontent.com/DFVcH-E5XKj8cbhEtI0qabmf_wwVqWWvk0pK5H5rnC_kVxY2tXClKfzV-LvAH61YRLJUEvtO9amjWfjcY4Z57VBYCuQ95_hdAVEHgLAuepJiArH0wJERWuzzmgnPysCiIA
![Page 46: Demi Ben-Ari - Monitoring Big Data Systems Done "The Simple Way" - Codemotion Milan 2017](https://reader033.fdocuments.in/reader033/viewer/2022051710/5a6479a17f8b9a36568b4793/html5/thumbnails/46.jpg)
● Elastic● Architecture:
Server
Server
Server
ELK - Elasticsearch + Logstash + Kibana
Shippers
Queue
Indexer Web UIStorage
![Page 47: Demi Ben-Ari - Monitoring Big Data Systems Done "The Simple Way" - Codemotion Milan 2017](https://reader033.fdocuments.in/reader033/viewer/2022051710/5a6479a17f8b9a36568b4793/html5/thumbnails/47.jpg)
ELK - Elasticsearch + Logstash + Kibana
http://www.digitalgov.gov/2014/05/07/analyzing-search-data-in-real-time-to-drive-decisions/
![Page 48: Demi Ben-Ari - Monitoring Big Data Systems Done "The Simple Way" - Codemotion Milan 2017](https://reader033.fdocuments.in/reader033/viewer/2022051710/5a6479a17f8b9a36568b4793/html5/thumbnails/48.jpg)
Did someone say “Dashboard”?
http://www.funpic.hu/_files/pictures/original/86/71/27186.jpg
![Page 49: Demi Ben-Ari - Monitoring Big Data Systems Done "The Simple Way" - Codemotion Milan 2017](https://reader033.fdocuments.in/reader033/viewer/2022051710/5a6479a17f8b9a36568b4793/html5/thumbnails/49.jpg)
Redash
● http://redash.io/● Open Source: https://github.com/getredash/redash● Came out as one of many Open source tool by Everything.me● Created and Maintained by Arik Fraimovich (You rock!)● Written in Python● Has an on-premise and hosted solution
● At Panorays we also use if for Alerting via Its integration with Slack
![Page 50: Demi Ben-Ari - Monitoring Big Data Systems Done "The Simple Way" - Codemotion Milan 2017](https://reader033.fdocuments.in/reader033/viewer/2022051710/5a6479a17f8b9a36568b4793/html5/thumbnails/50.jpg)
Redash - Screenshots
![Page 51: Demi Ben-Ari - Monitoring Big Data Systems Done "The Simple Way" - Codemotion Milan 2017](https://reader033.fdocuments.in/reader033/viewer/2022051710/5a6479a17f8b9a36568b4793/html5/thumbnails/51.jpg)
Alerting
![Page 52: Demi Ben-Ari - Monitoring Big Data Systems Done "The Simple Way" - Codemotion Milan 2017](https://reader033.fdocuments.in/reader033/viewer/2022051710/5a6479a17f8b9a36568b4793/html5/thumbnails/52.jpg)
Alerting
● Syren - Open source● Reporting to:
○ Email, Flowdock, HipChat, HTTP,Hubot, IRCcat, PagerDuty, Pushover, SLF4J, Slack, SNMP, Twilio
![Page 53: Demi Ben-Ari - Monitoring Big Data Systems Done "The Simple Way" - Codemotion Milan 2017](https://reader033.fdocuments.in/reader033/viewer/2022051710/5a6479a17f8b9a36568b4793/html5/thumbnails/53.jpg)
Summary - Monitoring Stack
Alerting
Metrics Collection
Datastore
Dashboard
Data Monitoring
Log Monitoring
![Page 54: Demi Ben-Ari - Monitoring Big Data Systems Done "The Simple Way" - Codemotion Milan 2017](https://reader033.fdocuments.in/reader033/viewer/2022051710/5a6479a17f8b9a36568b4793/html5/thumbnails/54.jpg)
Not my problem...or is it?
https://cdn.meme.am/instances/500x/22605665.jpg
![Page 55: Demi Ben-Ari - Monitoring Big Data Systems Done "The Simple Way" - Codemotion Milan 2017](https://reader033.fdocuments.in/reader033/viewer/2022051710/5a6479a17f8b9a36568b4793/html5/thumbnails/55.jpg)
So...Who does the monitoring in our company?
https://imgflip.com/i/18kvv1
![Page 56: Demi Ben-Ari - Monitoring Big Data Systems Done "The Simple Way" - Codemotion Milan 2017](https://reader033.fdocuments.in/reader033/viewer/2022051710/5a6479a17f8b9a36568b4793/html5/thumbnails/56.jpg)
Conclusions
● Correlating Application and System metrics!!!!● Ask the right monitoring questions -> answer with
Dashboards● KISS - simple is key, what’s hard, we tend not to do at all● Alert about what you can actually react to
○ (And to the relevant person)● Measure whatever you can
○ only way to know if you’re improving● Monitor your business KPIs too
![Page 57: Demi Ben-Ari - Monitoring Big Data Systems Done "The Simple Way" - Codemotion Milan 2017](https://reader033.fdocuments.in/reader033/viewer/2022051710/5a6479a17f8b9a36568b4793/html5/thumbnails/57.jpg)
Conclusions
● If all of what I’ve said is not enough…
Graphs are fricking cool!
http://www.rantlifestyle.com/2013/09/23/how-happy-this-baby-is-will-shock-you/
![Page 58: Demi Ben-Ari - Monitoring Big Data Systems Done "The Simple Way" - Codemotion Milan 2017](https://reader033.fdocuments.in/reader033/viewer/2022051710/5a6479a17f8b9a36568b4793/html5/thumbnails/58.jpg)
Questions?
![Page 59: Demi Ben-Ari - Monitoring Big Data Systems Done "The Simple Way" - Codemotion Milan 2017](https://reader033.fdocuments.in/reader033/viewer/2022051710/5a6479a17f8b9a36568b4793/html5/thumbnails/59.jpg)
● “Big Things” Community
�Meetup, YouTube, Facebook, Twitter
● GDG Cloud
● LinkedIn● Twitter: @demibenari● Blog:
http://progexc.blogspot.com/● [email protected]
![Page 60: Demi Ben-Ari - Monitoring Big Data Systems Done "The Simple Way" - Codemotion Milan 2017](https://reader033.fdocuments.in/reader033/viewer/2022051710/5a6479a17f8b9a36568b4793/html5/thumbnails/60.jpg)