Recent developments and plans at Météo-Fran ce

Recent developments and plans at Météo-France

Frédéric GUILLAUD, Fabien MARTY,

Marie-Françoise VOIDROT

EGOWS Meeting

Ljubljana, June 2008

2

Overview

Two operational tools for “upstream” forecasting at Meteo-France Synergie technical updates and roadmap Oppidum technical updates and roadmap SOA-OGC pilot project

3

Operational tools at Meteo-France

At the present time, two systems forthree levels of forecast

SYNERGIE– National and regional forecasting

(metropolitan and overseas)

– Worldwide usage outside of Meteo-France

OPPIDUM– Local departmental forecasting

(“Prévi-Surveillance” application)

– Outside of Meteo-France.(“Meteo+” application)

4

Synergie

5

Synergie : A little history

First stages of Synergie development started in 1989 First operational release in 1993 More than 15 operational versions since Several Operating System changes Today :

– Linux only

– More than 1,5 million lines of code

– A sum of 160 man-year of development

– A software patchwork of the best tools and libraries of Météo-France and ECMWF

– More than 120 operational client workstations at Météo-France

– More than 70 operational workstations in 25 other countries

6

Synergie development today

A mean of 8 Synergie developers A calendar driven release policy

– A new version by year (June)– Three patches by year

More than 100 new features and bug fixes for each version Many remaining needs !

7


Modern methods and tools– Change and configuration management based on CVS

– Automatic packaging (RPM)

– Bug Tracking System (Flyspray)

– Wiki (dokuwiki)

– Automatic balanced scorecard

8


But :– « Old » technologies

• Mainly « classic » C• MOTIF toolkit• X direct rendering• Oracle• Proprietary client/server communication

– A lot of historical layers

– An impressive software but a « prehistoric » look :• Icons optimised for old SUN graphic cards• Old Window Manager• Some things are far away from interface standards

9


10

Synergie : Technical Audit

2007 : a key year for Synergie– Some political issues :

• Is Synergie deprecated ?• Is it a good idea to continue to implement features in Synergie ?• Do we need to make a sort of new Synergie restarting from scratch ?

– A political decision : make an external audit of Synergie

So during last summer, we ask three questions to an outside consultant :

– Is Synergie technically deprecated ?

– What are main functional needs for next years ?

– What technical changes are needed in Synergie to implement them ?

11

Technical Audit

Is Synergie deprecated ?

– No major points in the next 5 years

But

– Some parts of the architecture will have to be improved or replaced.

12

Synergie : Technical Audit

What technical changes in Synergie ?– Three possibilities :

• (1) We do nothing– Synergie will be dead in (between) 5 or 10 years

• (2) We do a progressive upgrade of technical stakeholders– Synergie will be OK for 10 years

• (3) We restart from scratch (or we hack an existing software)– But are we ready ? (no spec at all !)

Today, no political formal decision

So we follow the scenario (2)

13

Synergie roadmap (4.3 : June 2008)

Oracle => postgreSQL• Mainly because of licence costs • Easier for sysadmins

Introduction of new methods• More secure and easy C with GLIB• An easier « object oriented » C with GOB preprocessor• New libraries from the GNOME zoo

14


Introduction of webservices• Implementation of Wep Map Service (OGC) : Synergie as WMS server• Interoperability with other systems or applications• Mix Synergie georeferenced output with other external datas

15

WMS with Synergie(WMS server) and Google (WMS client)

16

With transparency….hail kernel over Toulouse area

17


Introduction of GTK (to replace MOTIF)

18


– And we take advantage of this replacement to improve ergonomics

19

Synergie Roadmap (4.3 : June 2008)

– But we can also use GTK inside core drawing process

• Today, Synergie computes and renders overlays by itself– It's a low level code– It's hard to keep it bugfree

• With GTK CANVAS objects, we can use a very easy layer system– Less code– Easy to use– Impressive new possibilities

• But – Not as fast as Synergie optimized rendering kernel– Greedy (each layer is in memory)

• So :– We can use it for production windows (one or two at the same time)– But we have to wait for an hardware upgrade for visualisation windows (several

dozens at the same time)

• A full GTK/MAGICS application for next version : « Synergie Cyclone »

20


21


22


23


GDAL and Proj4 projection tools interface

- Facility to export any Synergie visualisation in Standard Geotiff georeferenced format

- Possibility to import any georeferenced image

24

NASA image

25

… zoomed

26

SPOT images (Image simplified resolution of 10 m)

27

Synergie roadmap : Further outlook

– A new window manager ?

28


– New icons ?

29


More GTK and less MOTIF ?– But MOTIF => GTK transition is a huge work...

Replacement of MAGICS by MAGICS++ (ECMWF)

30


More and more Web Services ? The end of our proprietary Client/Server communication protocol ? Continue to work on the WMS service towards a full web front-end to

Synergie ?

31

Oppidum

32

Oppidum : A little history and overview

Developments started in 1996 First operational release in 2001 More than 7 operational releases since Today :

– Windows only (XP, 2003 Server)– A sum of 40 man-year of development– More than 120 operational workstations in Météo-France– More than 200 operational workstations in other companies

Multi-Tier Architecture (data access, business, IHM) Reusable components (Active X) Applications (IHM) are sharing the same components :

– Previ-Surveillance : The forecasting tool in Departmental Offices– Meteoplus : Worksation for external usage– Meteofac : Pre-flight documentation (including automatic generation)– Aspoc+ : providing ATC with thunderstorm diagnostics

C++, C#, MFC, ILOG views, RogueWave DCOM Client/Server Protocol Packaging Install Shield Change and Configuration Management Telelogic

33

Oppidum : System Architecture

SGBD (Oracle)

Applications

(IHM)

Visu

Model

C++

Visu

Satellite

C++

Visu

Radar

C++

Visu

Lightning

C++

Meteo+ (VB) Aspoc+ (C#)

Météofac (C#) Production

(VB, C++, C#)

Met Visualisation Components

Warnings (C++) 2PiR Radar

extrapolation (C++)

Prévi-Surveillance (C++)

Data Acces (C++)

Data Acces

Business Components

Meta Data Acces (C++, C#)

Clie

nt s

ide

Ser

ver

side

LAN DCOM

34

Oppidum technical roadmap 2.0 : June 2008

DCOM => REMOTING .NET– It is now possible to work through a WAN

– Need high speed internet access and broadband (at least 4 Mbit/s)

ORACLE => PostgreSQL (like Synergie)

35


Fine mesh NWP challenge (AROME, ~ 2 km at the present time)– New parameters => new visualisation problems– It become very difficult to plot (contouring and/or shading) some “noisy” fields

like T2M, CAPE, …– Sometimes a disaster : up to 20’ to animate 10 forecasts !– In a few years much fine mesh (=> 500 m) and more and more levels : problems

have to be addressed now …– Those problems have occurred both on our two workstations Synergie and

Oppidum

=> Switch from contouring to pixmap visualisation.

Nowcasting : We had to integrate new Java component

=>Webservices achieve interoperability between JAVA and C# - .NET worlds

36

Oppidum : System Architecture 2.0

SGBD PostgreSQL

Applications

(IHM)

Visu

Model

C++

Visu

Satellite

C++

Visu

Nowcasting

C++

Visu

Radar

C++

Visu

Lightning

C++

Meteo+ (VB) Aspoc+ (C#)

Météofac (C#) Production

(VB, C++, C#)

Met Visualisation Components

Warnings (C++) 2PiR Radar

extrapolation (C++)

Nowcasting Engine

(Java)

Prévi-Surveillance (C++)

Data Acces (C++)Data Acces

Business Components

Meta Data Acces (C++, C#)

Clie

nt s

ide

Ser

ver

side

WEB Remoting HTTP

Proxy NET

Front-End NET

webservice

37


AROME T2M & Simulated Reflectivity as pixmap

38


AROME Cross Section for hydrometeors Integration of nowcasting component (JAVA)

Graupel

SnowRain

Ice-CloudWater-Cloud

39

Towards unique workstation ?

Two systems, two platforms (Linux , Windows) Meet the needs of 3 levels of forecast (national, regional, local) They have been developed and configured for that .But Features have to be implemented on both. Lack in development resources at Meteo-France Potential reorganization : cut down from 100 to 40 offices ?So Our Direction would like to go towards a single workstation for “advanced

forecasting”. A “light” workstation (probably Web-Based) for other needs ? Ideally, sharing the same business server components …

No political formal decision at the present time…

40

Pilot project SOA-OGC

This project has just been launched at the IT department. Because of :

– Normative context for geographic information.• ISOTC/211 and OGC

– International projects choices• INSPIRE, CESAR (WXXM), GEOSS have chosen to follow OGC standards

– Meteo-France context• More and more web services in order to integrate heterogeneous systems and applications

(most of them no standard compliant)• At the present time, no interface nor data standardization (beyond historic WMO formats like

GRIB and BUFR)• Thin experience in GML for meteorological objects and WMS (Synergie server)

Goals :– Improve our expertise on OGC standards and evaluate technologies to implement

them.– Primarily work before main projects

• Re-Architecting finalized production system (SOPRANO step 2)• Web-Based workstation ?

– Better cooperation within meteorological community

41

SOA-OGC Project

Actually, 3 subprojects :

– OGC Web Services

– Service Oriented Architecture : ESB evaluation

– Web-Based Workstation

42

Towards SOA architecture ?

Main goal : interoperability Roadmap towards SOA :

– Define data types and semantics (need collaboration ?)

– Choose formats to encode data (GML)

– Define services and semantics (need collaboration ?)

– Choose Service implementation and protocols

– Publish data types and services

– Register the services in a catalogue

Publish - Find - Bind

43

OGC Web Services subproject (1/2) WFS ( Web Feature Service)

– Version : 1.1.0– Binding REST– Data

• Observations at the surface (SYNOP) and nowcasting objects• GML 2 or GML 3

– Implementation • Geoserver 1.6.3

WCS (Web Coverage Service)– Version : 1.1.0– Binding REST– Data

• Satellite, NWP (Numerical Weather Predictions)

– Output formats• GeoTiff, GRIB 1 (Even if we should have written an application profile for GRIB 1 & 2 WMO

formats …)

– Implementation • Geoserver 1.6.3

44

OGC Web Services (2/2)

WMS– Version : 1.1 & 1.3

– Binding REST

– Implementation : Synergie as WMS

WMS – Version 1.1.0

– Binding REST

– Data : geographic layers (Spot, NOAA …)

– Implementation : GeoServer WMS

45

Service Oriented Architecture More and more Services … OK Everything is a service (most of them web services)…OK But how to avoid … that ?

WS NowcastingMagics++

WCS NWP

ProductionWFS Obs SurfaceOppidum

WS OKAPI

WCS Raster

WMS Synergie

Synergie

Sympo 2

WMS geographic layersWFS Business Post processing

46

Service Oriented Architecture

Light Web application

Geographical information

Met Data Server

Met visualisation

Business algorithms

Other environmemt

visualisation

Environment Data Server

WMS

WCS or WFS

ESB PETALS

47

Web-Based Workstation

No functional specifications yet … But technical requirements (!) :

– Must run in a browser – and if possible most of them (!) – Must use OGC-SOA infrastructure (to be developed)

Tools to be evaluated …– Google GWT – Adobe AIR Flex– Microsoft Silverlight– JavaScript toolkits (OpenLayers ….)

Main goal : What are the limits of web-based meteorological worksations ?– Interactivity ?– Avaibility ?

=> Results : at the next EGOWS …

Conclusion

49

Conclusion

Two successful desktop workstations. Despite of significant updates, the future of Oppidum will depend on

political and strategic decisions Already significant updates of Synergie architecture (4.3)

– Web Services, PostgreSQL, introduction of GTK… Major developments going on (4.4 and further)

– More GTK and less MOTIF, Magics++, Client/Server protocol … The roadmap would be :

– At short term, avoid the “bing-bang” at the client side – Gain experience in OGC standards, SOA, and Web 2 technologies (pilot

projects)– Establish a new server infrastructure “OGC-SOA” based (Soprano Step 2)– If possible, migrate existing applications (workstations, production) within this

architecture (It will depends on the performances …)– Finally, develop (or integrate) new light web-based workstations or business

services to meet the future needs of forecasters and end-users

50

Thank you !

Recent developments and plans at Météo-Fran ce

Documents

Transcript of Recent developments and plans at Météo-Fran ce