BDE ESD Tool - Big Data Met NORWAY Rasmus Benestad

Utskifting av bakgrunnsbilde:

- Høyreklikk på lysbildet og velg «Slide» -> «Set Background Picture for Slide...»

- Velg ønsket bilde og klikk «Open»

Brussels 2015-06-15

Climate analysis & Big data

Rasmus E. Benestad

Abdelkader Mezghani, & Kajsa M. Parding

esd for retrieving, processing, dissecting, analysing, and visualisation

How do I find useful climate informationin large volumes of data?

Information & Data & Knowledge

● Information "...in its most restricted technical sense, is a sequence of symbols that can be interpreted as a message".(wikipedia)

● Data are measurements, observations, calculations

● Knowledge is expectation about causalities, dependencies, and why?

Norwegian Meteorological Institute

The ultimate objective is to find the answers.

The data is the source and the means finding answers.

Analytical tools designed for data analysis/statistics.

What is the question?

Big data - simple algorithmsFast “distillation” of information

Open source tool ‘esd’http://github.com/metno/esd


esd: open-source and free

Data accessSource of information

Temperature

ECA&D:~1.1 Gb-temperature-precipitation

Temperature

ECA&D +GHCN +MET Norway-temperature

Precipitation

Reanalyses and satellite data

Climate model results


Make use of information hidden in vast archives of climate model results and observations

ExampleExtracting information embedded in global climate models and observations

Climate model results


What is hidden behind the results?


Building block

Empirical-statistical downscaling

Dependencies & connections

Redundancy


Extension of the results to regions


High one-in-twenty mean summer temperature in 2100

InfographicsExposing different aspects

Different sides to information

Changes in rainfall statistics?

Quick look at temperatures

Maps

Correlation

AnalysisPatterns of behaviour

Anomaly with respect to latitude

How similar are the reanalyses? Correlation maps

How good are the models?

Using mathematics to make sense of the data

Is there a change in the precipitation?


How much does it rain?


Global rain gauge data

= huge volume

35,000 rain gauges with daily data

Best way to mine hidden information?


Understand the data

Exponential distribution?

Vast number of data points on top of each other


Extract the essence (cleverly)

Principal component analysis: two main characteristics


E.g. The wet-day 95-percentile for 24-hr precipitation

Calculated Observed


Test results


Changing rainfall patterns


Aggregate

Principal Component Analysis (PCA)

Regression analysis

Quick search

Combination of sources

Mapping/gridding

Statistical distributions

Main instruments


Tricks for efficiency

Structured and standardised metadataCommon information model (CIM)Data reference syntax (DRS)classes and ‘S3’ methodsFast algorithms making use knowns


• Facilitate intercomparisons

• Sharing of generic methods

• Traceability and replicability

• Promotes community building & discussions

Benefits of common standards & structures


Summary

●esd - “easy and simple data” or empirical-statistical downscaling

●Statistics - information from the data

●Aim to address specific questions

Meteorologisk institutt

Thanks for your attention!

BDE ESD Tool - Big Data Met NORWAY Rasmus Benestad

Science

Transcript of BDE ESD Tool - Big Data Met NORWAY Rasmus Benestad