Monitoring and Analyzing London's Air Quality with Pentaho ... · PDF file Monitoring and...

Click here to load reader

  • date post

    24-May-2020
  • Category

    Documents

  • view

    1
  • download

    0

Embed Size (px)

Transcript of Monitoring and Analyzing London's Air Quality with Pentaho ... · PDF file Monitoring and...

  • Monitoring and Analyzing London's Air Quality with Pentaho Mark Semenenko, Pentaho Sales Engineer, Hitachi Vantara Anya Rumyantseva, Pentaho Data Scientist, Hitachi Vantara

  • Pentaho

  • Agenda

    Developing Pentaho Demo showcasing the platform’s end-to-end capabilities

    • Pentaho as an end-to-end platform • Use Case Overview • Demonstration • Summary

  • Pentaho provides flexible management for end-to-end data flows…

    Data PrepData Engineering Analytics

    Ingestion Processing Blending Data Delivery Data Discovery/ Analysis Analysis

    & Dashboards

    Administration Security LifecycleManagement Data

    Provenance Dynamic Data

    Pipeline Monitoring Automation

  • …and Machine Learning capabilities!

    Data Transformation and Machine Learning

    Workflows

    Full automated production

    Dashboard decision support

    ypred

    Data Engineering Data Preparation Analytics

    X

    Da ta co

    nn ec to rs

    Data Source 1

    Data Source 2

    Data Source n

  • Aim: develop a demo showing a full spectrum of Pentaho capabilities

  • Our Use Case “London Air Quality Monitoring”

  • Why?

    • Air quality affects well-being of residents and visitors of the city • 9000 early deaths a year in London due to air pollution • Air pollution monitoring and mitigation are crucial for city councils • The use case is easy to understand and to follow

  • Data Sources

    • London Air Quality Network (96 stations across the city) • Road Traffic Counts (number and vehicle type) • Meteorological Parameters (wind, temperature, humidity)

  • Collecting, Preparing and Blending

  • Demonstration

  • Collecting, Preparing and Blending

  • Air Quality Prediction

  • Pentaho Machine Learning Orchestration

    1 2

    4

    3

    1

    Predictive model for Air Quality was added to the demo.

    *Long-Short Term Memory neural network was used to make predictions

  • Demonstration

  • Operationalising Predictive Model in Pentaho

    What we have shown in the demo:

    • Feature engineering and data preparation • Scheduled model training and update • Model application on new data streams • Tracking and storing model performance metrics • Displaying predictions in Pentaho dashboards

    Operationalize machine learning models in Pentaho!

  • Summary

    What we covered today:

    • Pentaho provides a single consistent experience for developing data products • The platform allows to put machine learning algorithms into production • Informative dashboards for decision making based on advanced analytics and data transformation in the backend