Monitoring and Analyzing London's Air Quality with Pentaho Mark Semenenko, Pentaho Sales Engineer, Hitachi Vantara Anya Rumyantseva, Pentaho Data Scientist, Hitachi Vantara

Pentaho

Agenda

Developing Pentaho Demo showcasing the platform’s end-to-end capabilities

• Pentaho as an end-to-end platform • Use Case Overview • Demonstration • Summary

Pentaho provides flexible management for end-to-end data flows…

Data PrepData Engineering Analytics

Ingestion Processing Blending Data Delivery Data Discovery/ Analysis Analysis

& Dashboards

Administration Security LifecycleManagement Data

Provenance Dynamic Data

Pipeline Monitoring Automation

…and Machine Learning capabilities!

Data Transformation and Machine Learning

Workflows

Full automated production

Dashboard decision support

ypred

Data Engineering Data Preparation Analytics

X

Da ta co

nn ec to rs

Data Source 1

Data Source 2

Data Source n

Aim: develop a demo showing a full spectrum of Pentaho capabilities

Our Use Case “London Air Quality Monitoring”

Why?

• Air quality affects well-being of residents and visitors of the city • 9000 early deaths a year in London due to air pollution • Air pollution monitoring and mitigation are crucial for city councils • The use case is easy to understand and to follow

Data Sources

• London Air Quality Network (96 stations across the city) • Road Traffic Counts (number and vehicle type) • Meteorological Parameters (wind, temperature, humidity)

Collecting, Preparing and Blending

Demonstration

Collecting, Preparing and Blending

Air Quality Prediction

Pentaho Machine Learning Orchestration

1 2

4

3

1

Predictive model for Air Quality was added to the demo.

*Long-Short Term Memory neural network was used to make predictions

Demonstration

Operationalising Predictive Model in Pentaho

What we have shown in the demo:

• Feature engineering and data preparation • Scheduled model training and update • Model application on new data streams • Tracking and storing model performance metrics • Displaying predictions in Pentaho dashboards

Operationalize machine learning models in Pentaho!

Summary

What we covered today:

• Pentaho provides a single consistent experience for developing data products • The platform allows to put machine learning algorithms into production • Informative dashboards for decision making based on advanced analytics and data transformation in the backend