Post on 23-Mar-2018
©2015 Trendalyze Decisions LTD
Case Study: Public Health Environmental Analytics
Liverpool Environmental Application Framework (LEAF)
Gaining insights from environmental Internet of Things (IoT) time series, web data and open data using public health and climate change use cases
Apache Spark, Apache Hadoop, Apache Kafka, Node-RED, Mosquitto, Zeppelin and Salt
©2015 Trendalyze Decisions LTD
What is Liverpool Sensor City?
Liverpool Sensor City will support companies as they develop and implement novel sensor systems integrating sensors, firmware and advanced algorithms.
Why Sensors?Sensor market is growing by 10% each yearAround 1.4 million people in the UK are employed in sensor-aligned professionsExports from the UK sensor industry produce around £6 billion ($9B) each year.The UK sensor industry generates £13 billion ($19B) each year and supports 70,000 jobs
https://www.liverpool.ac.uk/sensor-city/
©2015 Trendalyze Decisions LTD
Liverpool Environmental Application Framework (LEAF)
• Spark big data and IoT analytics, predictive models, search and APIs
• Analysis of data from world-class University of Liverpool academic experts
• Pre-built analytics and environmental models
• Tools for Business Users and App Developers
• R, Python, Scala, Java and SQL
• Publish new data sets and APIs
• IoT gateway for citizen sourced data
• Platform as a Service (commercial fee based) and Professional Services
©2015 Trendalyze Decisions LTD
The Challenge Without LEAF
• Locate data sets
• Acquire proprietary data
• Quality assure data
• Link and merge data
• Load data into analytical database
• Build analytics
• Build predictive models
• Map GIS and other visualizations
• Deploy and maintain IoT solution
©2015 Trendalyze Decisions LTD
1 2Identify one pattern (e.g engine failure) Search continuously for similar patterns
Data Samples Modeling Human Skills Timing
IoT Search Analytics Example – Condition Based Maintenance
©2015 Trendalyze Decisions LTD
Mosquitto MQTT Broker
Node-RED/Kafka Queues
Azure IoT Suite
Apache Spark• Transformed data in Parquet• UDFs in Java, Scala, R• Models in SparkSQL and ML• Access remote HDFS files, IoT
streams• Trendalyze Search Algorithms
IBM ODP Hadoop HDFS Cluster
Store detailed level data sets in CSV files, JSON files, NetCDF files on HDFS
Store IOT event data in JSON and Kafka files on HDFS
Batch ETL jobs can be managed
HDFS reads
HDFS writes usingNode-RED connectorand Kafka
WebApplicationsand APIs
Data Viz Tools/Notebooks
IoTWeatherStation
Technical Architecture
R, Python, SQL,Java, Scala
MQTTDevice SDK
REST APIs
©2015 Trendalyze Decisions LTD
Open Source Tools for IoT Projects - Zeppelin Notebook
https://zeppelin.incubator.apache.org/
©2015 Trendalyze Decisions LTD
Open Source Tools for IoT Projects – Freeboard.io Dashboards
http://freeboard.io/
©2015 Trendalyze Decisions LTD
Open Source Tools for IoT Projects - Salt
http://unchartedsoftware.github.io/salt/
©2015 Trendalyze Decisions LTD
Future Implementation – Microsoft Azure IoT Suite
https://azure.microsoft.com/en-gb/solutions/iot-suite/
• HDInsights data analytics
• IoT Hub
• IoT Gateway
• Device SDKs
• Streaming Analytics
• Predictive Analytics
• Business Intelligence
• APIs for Cognitive Services
• Scaleable Azure Cloud Platform
©2015 Trendalyze Decisions LTD
Thank You!
Contact: dave.watson@trendalyze.com
+44 7741 675325+1 786 264 2749