IMPROVING RESILIENCE TO EMERGENCIES THROUGH … · Improving Resilience to Emergencies through...

IMPROVINGRESILIENCETOEMERGENCIESTHROUGH

ADVANCEDCYBERTECHNOLOGIES

“ReportonEMS,EuropeanEarlyWarningSystemsandSentineldataintegration”

DeliverableID D3.1

WorkPackageReference WP3

Issue 1.00

DueDateofDeliverable 30/11/2017

SubmissionDate 15/11/2017

DisseminationLevel1 PU

LeadPartner GeoVille

Contributors EOXPLORE,CSI,TUW

GrantAgreementNo 700256

CallID H2020-DRS-1-2015

FundingScheme Collaborative

I-REACT is co-fundedby theHorizon2020FrameworkProgrammeof theEuropeanCommissionundergrantagreementn.700256

1PU=Public,PP=Restrictedtootherprogrammeparticipants(includingtheCommissionServices),RE=Restrictedtoagroupspecifiedbytheconsortium(includingtheCommissionServices),CO=Confidential,onlyformembersoftheconsortium(includingtheCommissionServices)

ImprovingResiliencetoEmergenciesthroughAdvancedCyberTechnologies

Project:I-REACT “ReportonEMS,EuropeanEarlyWarningSystemsandSentineldataintegration”

DeliverableID:D3.1

GrantAgreement:700256 CallID:H2020-DRS-1-2015 Page:2of84

Preparedby Reviewedby Approvedby

W.Stemberger C.Klug C.Rossi

Issue Date Description Author(s)

0.01 13/10/2016 Setupoftemplate W.Stemberger

0.1 10/11/2016 Tableofcontents F.Nagl

0.2 26/03/2017 Provisionofcontentsforchapter2 F.Nagl

0.3 07/11/2017 Provisionofcontentsforchapter5 I.Ali

0.4 08/11/2017 Provisionofcontentsforchapter2 F.Nagl,M.Schwandner

0.5 09/11/2011 Provisionofcontentsforchapter4 M.Leonelli,M.Velluto

0.6 13/11/2017 Provisionofcontentsforchapter3 C.Bielski

1.0 13/11/2017 Reviewandcorrections C.Klug



DeliverableID:D3.1


TABLEOFCONTENTS1 INTRODUCTION...........................................................................................................................9

1.1 PurposeoftheDocument....................................................................................................9

1.2 StructureoftheDocument..................................................................................................9

1.3 AcronymsList.....................................................................................................................10

1.4 ReferenceandapplicableDocuments...............................................................................10

2 COPERNICUSEMSINTEGRATION..............................................................................................12

2.1 ServiceOverviewofCopernicusEMS................................................................................12

2.1.1 RapidMapping...........................................................................................................14

2.1.1.1 ReferenceMaps..................................................................................................15

2.1.1.2 DelineationMaps................................................................................................16

2.1.1.3 GradingMaps......................................................................................................17

2.1.2 RiskandRecoveryMapping.......................................................................................19

2.1.2.1 ReferenceMaps..................................................................................................19

2.1.2.2 Pre-disasterMaps...............................................................................................19

2.1.2.3 Post-disasterMaps..............................................................................................19

2.2 CopernicusEMSDataPortal..............................................................................................20

2.2.1 OverviewonDataProvision.......................................................................................22

2.3 CopernicusEMSIntegration..............................................................................................24

2.3.1 DesignofEMSIntegrationModule............................................................................25

2.3.2 ImplementationofEMSIntegrationModule.............................................................26

3 EUROPEANEARLYWARNINGSYSTEMSINTEGRATION.............................................................29

3.1 ServicesOverviewoftheCopernicusEWS........................................................................29

3.1.1 TheEuropeanFloodAwarenessSystem(EFAS).........................................................29

3.1.2 TheEuropeanForestFireInformationSystem(EFFIS)...............................................30

3.2 CopernicusEFASDataPortal.............................................................................................33

3.3 CopernicusEFASIntegration.............................................................................................34

3.4 I-REACTprocessingofEFASData.......................................................................................35

3.4.1 AffectedCommunitiesBasedontheEFAS5-yearReturnPeriodForecast................36

3.4.2 AffectedCommunitiesBasedontheEFAS20-yearReturnPeriodForecast..............39

3.5 CopernicusEFFISDataPortal.............................................................................................42



DeliverableID:D3.1


3.6 CopernicusEFFISDataPortal.............................................................................................43

3.6.1 TheFireWeatherIndex..............................................................................................43

3.6.2 AffectedCommunitiesBasedontheFWI...................................................................44

3.6.3 AffectedCommunitiesBasedontheFWI–NUTSlevel3...........................................47

3.6.4 FireHotSpotDetectionacrossEurope......................................................................47

3.6.5 AffectedCommunitiesBasedontheVIIRSfirehotspotdata.....................................48

3.7 Discussion..........................................................................................................................52

4 EXISTINGLOCALEMSINTEGRATION.........................................................................................53

4.1 OverviewofLocalEMS......................................................................................................53

4.2 SurveyofexistingEmergencyManagementSystems.......................................................55

4.2.1 Finland........................................................................................................................55

4.2.2 Italy.............................................................................................................................56

4.2.3 Spain...........................................................................................................................60

4.2.4 UnitedKingdom..........................................................................................................61

4.3 DefinitionandDevelopmentofIntegrationServices........................................................63

5 SENTINELDATAPROCESSINGINTEGRATION............................................................................68

5.1 Satellitedataprocurement................................................................................................68

5.1.1 Dataandsensorspecifications...................................................................................69

5.1.2 Datastoragesystem...................................................................................................70

5.1.3 Datacoverageandavailability....................................................................................71

5.1.4 Historicalremotesensingdata...................................................................................72

5.1.5 EODCprocessingenvironmentandplatform.............................................................72

5.1.6 EODCdatastorageandSentinel-1dataflow..............................................................73

5.2 Sentinel-1watermappingalgorithmdesign......................................................................74

5.2.1 Userinterfaceandfloodmonitoringactivationmethod............................................75

5.2.2 Sentinel-1basednear-realtimefloodmappingservicelogicimplementation..........75

5.2.2.1Developmenttools,supportingproductsandqualitycontrol..................................75

A. TUWienSARToolbox.................................................................................................75

B. SupportingProductsandModules.............................................................................76

C. Dataacquisitionandmanagement............................................................................77

5.2.2.2Concepts,RequirementsandInfrastructure.............................................................77



DeliverableID:D3.1


A. Conceptandbigpicture.............................................................................................77

B. Requirements.............................................................................................................78

C. Infrastructureandservicelogicimplementationscheme..........................................78

D. Prototype....................................................................................................................79

6 CONCLUSIONS...........................................................................................................................83

LISTOFFIGURES

Figure2-1:CopernicusEMSserviceoverview..................................................................................13

Figure2-2:Exampleofareferencemapdepictingthepre-eventsituationregardingseverefloodingsinJune2016inBavaria(Germany).Themapdisplaysbasictopographicfeatures.Itwasproducedusingpublicdatasetsandpre-eventimagery..................................................................................16

Figure2-3:ExampleofaflooddelineationintheareaofCarlisle(UnitedKingdom)producedafterStormDesmondcausedsignificantsurfacefloodinginDecember2015.Themapshowstheextentofthefloodedareaattwotimes.Itwasobtainedfrompost-eventimageryusingasemi-automaticapproach..........................................................................................................................................17

Figure2-4:ExampleofadamageassessmentfortheareaofAmatrice(Italy)aftertheearthquakeoccurred on 24th August 2016. The map displays the assessment of damage to buildings andtransportation infrastructureaswellasfurthercrisis information,which indicatesblockedroads,landslides,debrisandtents.Thecrisisinformationwasderivedfrompost-eventsatelliteimagebyvisualinterpretation.........................................................................................................................18

Figure2-5:CopernicusEMSDataPortal;RapidMappingactivationprovidingmetadataandaccesstorasterandvectorproducts..........................................................................................................21

Figure2-6:Mostlyavailablegeographicfeatures,labeledwithbasenames,andcommonattributes...........................................................................................................................................................23

Figure2-7:........................................................................................................................................26

Figure2-8:........................................................................................................................................26

Figure3-1:TheEFASforecastingwebsitethatispubliclyaccessible.Notethatthepublicservicedoesnotprovidecurrentfloodforecasts.................................................................................................30

Figure3-2:TheEFFISCurrentSituationviewer.ThisviewerprovidescurrentseasonwildfiresituationinformationacrossEurope...............................................................................................................31

Figure3-3:TheEFFIShistoricalwildfireviewershowingresultsfor2014.......................................32

Figure3-4:FloodprotectionlevelsacrossEuropebasedontheJongmanetal.(2014)methodology...........................................................................................................................................................35



DeliverableID:D3.1


Figure3-5:DescriptionoftheprocessingchainfortheI-REACTtask3201fortheproductionofthe5-yearreturnperiodfloodhazardforecastoutput..........................................................................37

Figure3-6:Final5-yearfloodhazardforecastoutputinGeoJSONfileformat.Thedarkerthebluecolour,thegreaterthefloodhazardforcast....................................................................................39

Figure3-7:DescriptionoftheprocessingchainfortheI-REACTtask3202fortheproductionofthe20-yearreturnperiodfloodhazardforecastoutput........................................................................39

Figure3-8:Comparinga5-year(leftside)and20-year(rightside)returnperiodfloodhazardforecast.Herethe'raw'dataispresentedatthepixellevel...........................................................................40

Figure3-9:Five-andtwenty-yearreturnperiodsforthefloodhazardforecastoverlapping.PleaserefertoFigure8fortheseparatefloodhazardforecasts................................................................40

Figure3-10:5-and20-yearfloodhazardforecastwithNUTSlevel5datainthebackground........41

Figure3-11:Thefinal20-yearfloodhazardforecastinGeoJSONformat.Theblueshadingprovidesinformation about the number of models forecasting a flood disaster. Darker = more modelsforecastingflood..............................................................................................................................41

Figure3-12:TheprocessingchainforI-REACTtask3203basedontheECMWFFWIdata.............44

Figure3-13:TheI-REACTtask3203outputbasedonECMWFFWIdata.ThetablepresentssomeofthedetailsstoredintheGeoJSONpolygons....................................................................................46

Figure3-14:TheI-REACTtask3206outputbasedonECMWFFWIdata.ThetablepresentssomeofthedetailsstoredintheGeoJSONpolygons.....................................................................................47

Figure3-15:TheprocessingstepsoftheI-REACTtask3204toprocessVIIRSbasedactivefiredatatodelineateaffectedcommunities......................................................................................................48

Figure3-16:TheVIIRSactivefireproductusedtoproducedelineationsoftheNUTS3levelregionswheretheywereidentified.Thetableshowsthateachofthepolygonsprovidesthenumberofhotspotpixelsfoundwithinthatregion(count)....................................................................................50

Figure3-17:TheprocessingstepstoproducetheI-REACTtask3205GeoJSONoutput..................51

Figure3-18:TheGeoJSONoutputoftheI-REACTtask3205showingtheVIIRSactivefiredatapointswithintheNUTSlevelpolygons........................................................................................................51

Figure4-1:WarningsissuedbytheFinnishMeteorologicalInstitute..............................................55

Figure4-2:Meteo/HydrogeologicalwarningbulletinissuedbyARPAPiemonte............................57

Figure4-3:LivestormserviceissuedbyARPAPiemonte.................................................................58

Figure4-4:WarningsonsensorsissuedbyARPAPiemonte............................................................58

Figure4-5:Monitoringhydrostations–RiverPo............................................................................59

Figure4-6:PoriverstretchbetweenFerraraandPanaro................................................................59

Figure4-7:MeteowarningbulletinissuedbyMetOffice................................................................62



DeliverableID:D3.1


Figure4-8:FloodriskwarningsissuedbyEnvironmentAgency......................................................62

Figure4-9:LEMSintegrationmoduleandtheI-REACTsystem........................................................63

Figure4-10:LEMSdatabase.............................................................................................................64

Figure4-11:ARPAHydrogeologicalbulletininI-REACT...................................................................66

Figure 5-1: EODC data acquisition strategy. Satellite data are transferred from the satellite toCollaborative Ground Segment, then to a rolling archive system at ZAMG and stored to EODCinternalstoragesystem....................................................................................................................68

Figure5-2:Sentinel-1AdatacoverageforIWGRDacquisitionmode..............................................71

Figure5-3:Sentinel-1AdatacoverageforIWGRDacquisitionmode..............................................71

Figure5-4:ENVISATASARcoveragemapforWideSwathmode.....................................................72

Figure5-5:EODCinfrastructurecomponents..................................................................................73

Figure5-6:ENVISATASAR/Sentinel-1datapre-processingworkflow............................................74

Figure5-7:ENVISATASAR/Sentinel-1schemeforproductiongeneration.....................................74

Figure5-8:Anexampleoverviewof theSyntheticApertureRadar (SAR)datapre-processingandproductgeneration(NeusiedlLake:Austrian–Hungarianborder)...................................................75

Figure5-9:ApplicationofHandIndexmasktoremovethetopographicnoise...............................76

Figure5-10:AnexampleS1bordernoiseremovalmask.................................................................77

Figure5-11:AnoverviewofI-REACTORframework........................................................................78

Figure 5-12: An overview of infrastructure and service logic for near-real time floodmapping/monitoring........................................................................................................................79

Figure5-13:Anexampleofmulti-temporalfloodfrequencyproduct(Italy)...................................80

Figure5-14:Anexampleofmulti-temporalfloodfrequencyproduct(UnitedKingdom)................80

Figure5-15:ExampleofSentinel-1productuploadedtoI-REACTdatabase...................................81

Figure 5-16: Floodmonitoring framework (top) andprototypeof final product displayedon thefront-endoftheI-REACTwebpage(bottom)...................................................................................81

LISTOFTABLES

Table2.1:Targeteddeliverytimesforthethreemaptypesatthetwoservicelevels....................15

Table2.2:Mainitemsofreferencemaps........................................................................................15

Table2.3:Additionalavailablefeaturesofdelineationmaps..........................................................16

Table2.4:Gradingmap-specificfeatures.........................................................................................18



DeliverableID:D3.1


Table2.5:Countofactivations.........................................................................................................20

Table2.6:Specificationsofprovidedmapproducts........................................................................22

Table3.1:Floodhazardclassificationmatrixbasedontheproceduredescribedinthetext..........34

Table3.2:AvailableEFFISdatathroughWMS..................................................................................42

Table4.1:Connectedsources..........................................................................................................54

Table4.2:Catalunya.........................................................................................................................60

Table4.3:I-REACTDataInterfaceattributes...................................................................................64

Table4.4:GeoJSONattributes.........................................................................................................65

Table4.5:DatastructureofUKserviceprovidingfloodwarnings...................................................67

Table5.1:Sentinel1sensorspecifications.......................................................................................69

Table5-2:Sentinel1:specificationsofdifferentdataacquisitionmodes.......................................70

Table6.1:Verylargetable...................................................................Error!Bookmarknotdefined.



DeliverableID:D3.1


1 INTRODUCTION

1.1 PURPOSEOFTHEDOCUMENT

Thegoalofthisdocumentistoreportonthedesignandimplementationdetailsofinterfacestowardexternal data sources such as Copernicus EMS, European Flood & Fire Systems, local EMS andSentineldatawhicharepartoftheworkpackage3(WP3)“ExternalServicesandDataIntegration”ofthe I-REACTproject.Theoverallobjectiveof thisWP is to integrateexistingdatasourcesandsystems related tonatural hazards, suchas floods and fires, and therefore to “fuel” theoverallsystemwithusefulinformationsources.TheWPissplitintoseveraltasksaccordingtothetypeofdataandsystems,whichareintegratedwithinI-REACT.Ingeneral,datastreamsandserviceswillbeimplementedfollowingtheas-a-serviceapproach,sothatnewdataisingestedautomaticallyintoI-REACTwithouthavingusertoactivateadatastream.

1.2 STRUCTUREOFTHEDOCUMENT

Thedocumentisorganizedasfollows,representingtaskswithindeliverable3.1ofWP3.

• Chapter 1 is an introductory description of the document itself, which should help toorganizereadingincomprehensiveway.

• Chapter2givesadetailedandthoroughdescriptionontheworkingprogressandactivitieswithinTask3.1“COPERNICUSEMSINTEGRATION”.

• Chapter 3 describes indetail the activitieswithin Task3.2 “EUROPEANEARLYWARNINGSYSTEMSINTEGRATION”.

• Chapter4,“EXISTINGLOCALEMSINTEGRATION”comprisesasynopsisofactivitieswithinTask3.4.

• Chapter5isadescriptionofTask3.5“SENTINELDATAPROCESSINGINTEGRATION”.

• Chapter6containstheconclusions.



DeliverableID:D3.1


1.3 ACRONYMSLIST

CAMS CopernicusAtmosphereMonitoringService DSS DecisionSupportSystemEMS EmergencyManagementService EODC EarthObservationDataCentre ERCC EuropeanResponseandCoordinationCentre ESA EuropeanSpaceAgency EWS EarlyWarningSystem EO EarthObservation FWI ForestFireWeatherIndex GWIS Global Wildfire Information System I-REACT ImprovingResiliencetoEmergenciesthroughAdvancedCyberTechnologiesLEMS LocalEmergencyManagementSystemsNRT NearReal-TimeSGRT SARGeophysicalRetrievalToolboxWMS WebMapServiceWP Workpackage ZAMG ZentralanstaltfürMeteorologieundGeodynamik

1.4 REFERENCEANDAPPLICABLEDOCUMENTS

ID Title Revision Date

[RD01] http://emergency.copernicus.eu/mapping/ems/cite-copernicus-

ems-mapping-portal online

accessed 2016

[RD02] https://web.jrc.ec.europa.eu/callsfortender/index.cfm?action=a

pp.tender&id=2588 onlineaccessed 2017

[RD03] http://publications.jrc.ec.europa.eu/repository/bitstream/JRC83

027/lb-na-26072-en-n.pdf onlineaccessed 2017

[RD04]

http://www.copernicus.eu/main/emergency-management onlineaccessed 2017

[RD05]

https://www.efas.eu/efas-archive.html onlineaccessed 2017

[RD06]

http://effis.jrc.ec.europa.eu/about-effis/data-license/ onlineaccessed 2017

[RD07]

http://effis.jrc.ec.europa.eu/applications/fire-history/ onlineaccessed 2017

[RD08]

http://effis.jrc.ec.europa.eu/applications/data-and-services onlineaccessed 2017

[RD09]

http://ec.europa.eu/eurostat/web/nuts onlineaccessed 2017

[RD10]

http://effis.jrc.ec.europa.eu/api-test/#/rest onlineaccessed 2017

[RD11] http://docs.oasis-open.org/emergency/cap/v1.2/CAP-v1.2-os.pdf

onlineaccessed 2017

[RD12]

http://en.ilmatieteenlaitos.fi/about-us online

accessed 2017

[RD13]

http://meteoalarm.eu/ online

accessed 2017



DeliverableID:D3.1


[RD14]

https://www.eodc.eu/ online

accessed 2017

[RD15] Elefante,S.,Wagner,W.,Briese,C.,Cao,S.,&Naeimi,V.:High-

performancecomputingforsoilmoistureestimation.Proceedingsofthe2016conferenceonBigDatafromSpace(BiDS'16).

-- 2016

[RD16]

A.Nobre,L.Cuartas,M.Hodnett,C.Renn,G.Rodrigues,A.Silveira,M.Waterloo,andS.Saleska.Heightabovethenearestdrainageahydrologicallyrelevantnewterrainmodel.JournalofHydrology,vol.404,no.1,pp.13–29,011.http://www.sciencedirect.com/science/article/pii/S0022169411002599.

online

accessed 2017

[RD17]

I. Ali, S. Cao, V.Naeimi, C. Paulik, andW.Wagner.Methods toremovebordernoisefromSentinel-1 interferometricwideswathgroundrangedetectedsyntheticapertureradardata:Implicationsandimportancefortimeseriesanalysis. IEEEJournalofSelectedTopicsinAppliedEarthObservationsandRemoteSensing.

-- 2017

[RD18]

B. Jongman, S. Hochrainer-Stigler, L. Feyen, J.C.J.H. Aerts, R.Mechler, W.J.W. Botzen, L.M. Bouwer, G. Pflug, R. Rojas, P.J.Ward.Increasingstressondisaster-riskfinanceduetolargefloods. Nat.Clim.Change,4,pp.264-268,10.1038/nclimate2124.

-- 2014

[RD19]

V.Naeimi, S. Elefante, S. Cao,W.Wagner,A.Dostalova, andB.Bauer-Marschallinger. Geophysical parameters retrieval fromsentinel-1sardata:Acasestudyforhighperformancecomputingat eodc, Proceedings of the 24thHigh Performance ComputingSymposium, ser. HPC ’16. San Diego, CA, USA: Society forComputerSimulationInternational,2016,pp

-- 2016

[RD20] Details about Equi7 projection:

http://dx.doi.org/10.1016/j.cageo.2014.07.005

onlineaccessed 2017



DeliverableID:D3.1


2 COPERNICUSEMSINTEGRATION

2.1 SERVICEOVERVIEWOFCOPERNICUSEMS

CopernicusEmergencyManagement Service (CopernicusEMS)provides information fordisasterresponse management and supports recovery, disaster risk reduction, prevention, andpreparednessactivities.Theserviceprovidesmapsbasedonsatelliteimagery,addressingvariousemergencysituationsarisingfromnaturalorman-madedisasters,aswellasearlywarningservicesforfloodandfirerisks.MainaimoftheEuropeanUnion(EU)serviceisreinforcingcrisisresponseincaseofnationalorcross-borderdisastersinEuropebutalsoincaseoflarge-scaledisastersoutsideoftheEU[RD01].

CopernicusEMS ispartof theEuropeanCopernicusProgrammeforEarthObservation,aimedatproviding information services on environmental and security issues, based on satellite earthobservation and in situ data. Within this framework, the EMS is operated by the EmergencyResponseCoordinationCentre (ERCC)at theEuropeanCommission'sHumanitarianAidandCivilProtection department (ECHO). Technical assistance in support of service implementation isprovidedbytheJointResearchCentre(JRC).

CopernicusEMSconsistsofdifferentmodulesthatcanbeassignedtotwomaincomponents(Figure2-1)[RD01]:

• EFAS and EFFIS - delivering alerts and risk assessments of floods and forest fires - can besubsumedtoanearlywarningcomponent(describedindetailinchapter3).

• The Copernicus EMS Mapping component provides map products and analysis in case ofdisasters.Itaddressesawiderangeofemergencysituationsresultingfromnaturalorman-madedisasters,coveringinparticularfloods,earthquakes,tsunamis,landslides,severestorms,fires,industrialaccidents,volcaniceruptions,andhumanitariancrises.Theproductsaredeliveredintwodifferentmodesbydifferentmodules:

o RapidMappingprovidesrapidservicedeliveryduringtheemergencyresponsephase(withinhours/days;available24/7/365)

o Risk & Recovery Mapping is designed for on-demand provision of geospatialinformationsupportingrecovery,disasterriskreduction,prevention,andpreparednessactivitiesinpre-orpost-crisissituations(withinweeks/months)

This service structure is complemented by a dedicated validation component, aiming at theimprovementoftheEMSmappingserviceproductsregardingrobustnessandaccuracycriteria.



DeliverableID:D3.1


Figure2-1:CopernicusEMSserviceoverview.

TheoperationalphaseoftheEmergencyManagementService(EMS)startedinApril2012withinthe former framework of the Regulation on GMES (Global Monitoring for Environment andSecurity), which has been updated by the Copernicus Regulation in April 2014. The providedinformationandproductsareaccessibleonafull,openandfreeof-chargebasistothepublic.

CopernicusEMSMapping Services address inparticular theneedsof actors in the fieldof crisismanagementandhumanitarianaidoperations.Relatedtoanemergencyevent“AuthorisedUsers”(AUs) can trigger the service provision process. AUs includemainly the national civil protectionauthoritiesfromtheparticipatingstatesoftheEUCivilProtectionMechanism(NationalFocalPoints,NFPs) as well as EU services. Associated Users (ASCUs), including - amongst others - local andregionalpublicentitiesandgovernmental andnon-governmentalorganisations, can request theservicethroughanAuthorisedUser.Inordertorequestanactivation,aServiceRequestForm(SRF)has to be completed and submitted via e-mail to the European Response Coordination Centre(ERCC).

EachrequestisthenreviewedtoensurethatanactivationiswithinthescopeoftheEMSMappingServiceregardingthetypeandmagnitudeofanevent,therelevanceofimpactandthedegreeofurgency(RapidMappingService)orregardingtherelevanceforDisasterRiskReduction,PreventionandPreparednesspurposes(RiskandRecoveryMappingService).FollowingtheauthorisationoftheactivationrequestbytheERCCmappingproductsaregeneratedbyserviceprovidersunderservicecontracts.Theprocessingofanactivationrequestfollowsadefinedworkflowinvolvingdifferentserviceprocessingstepsincollaborationandexchangebetweenserviceprovider,authoriseduser,JRC,ERCCandESA.JRCisresponsibleforthetechnicalcoordinationoftheEMSMapping,itperformstechnicalandqualitycontrolsthroughoutserviceprocessingandmanagestheserviceevaluationprocessuponanactivation.Furtheritisinchargeofthearchivinganddisseminationofgeneratedproducts. The European Space Agency (ESA) generally coordinates procurement and access to

CopernicusEmergencyManagementService(EMS)

EarlyWarning

EFAS EFFIS

Mapping

RapidMappingRiskandRecoveryMapping



DeliverableID:D3.1


satellite data required for the generation of deliverables. The data provision is based on theCopernicus Space Component Data Access (ESA-CSC-DA) system that offers access to satelliteimagingsensordataofCopernicusSentinelaswellasCopernicusContributingMissions(CCM).

2.1.1 RAPIDMAPPINGSupport of crisismanagement during emergency responsephase is emphasised to be themainpurpose of the rapid mapping service. Earth observation data and disaster related maps areprovidedon-demandwith strict timeliness requirements.Acquisition,processingandanalysisofsatelliteandaerialimageryandothergeospatialrasterandvectordataformthebasisforserviceprovision.

TheCopernicusEMSRapidMappingserviceisoperatedcontinuously,i.e.servicerequestscanbehandled24hoursaday,sevendaysaweek.Thaton-dutyrequesthandlingpermitsashortresponsetime for product supply within the demanded time frame of hours / days. In this regard, theavailabilityofpost-eventsatellite imagery,shortlyafteranemergencyevent, isconsideredakeyrequirement,whichshallbefulfilledwithintheframeworkoftheESACSC-DAmechanism.

Therapidmappingserviceaimsattheprovisionofassessmentsofimpactextentanddamagegradeas well as pre-emergency information particularly on infrastructure of affected areas. Thestandardisedproducts includeprintablemaps supplementedbybrief analysisona case-by-casebasis,aswellasgeospatialdatasetsforusewithinageographicalinformationsystem(GIS).

Dependingontheservicerequest,threecategoriesofmapsareprovided:

• Reference maps: based on satellite imagery acquired prior to the disaster event theydescribethesituationbeforetheemergencysituation

• Delineationmaps:outlinetheextentoftheareaaffectedbytheevent

• Gradingmaps:provideanassessmentoftheimpact/damagegradecausedbyanevent

The differentmap products are provided individually or in combination depending on the userneeds. The service also features updates for delineation and grading maps, allowing for amonitoring.

Tomeettherequirementsgivenbythelevelofurgency,eachmaptypecanberequestedintwodifferentservicelevels(Table2-1):

• Service Level 1 targets a rapid product delivery within a few hours after the receipt ofbaselinesatellitedata.Additionallytotheprovisionofprecisemapswithinthedemandedtimeframeof9to12hours(seetable)aFirstAvailableMap(FAM)isprovidedwithin3hoursfor delineation and gradingmaps representing an early information product with lowerthematicandpositionalaccuracy.



DeliverableID:D3.1


• ServiceLevel5targetstheprovisionofmapproductswithmaximumaccuracyandlevelofdetailwithinfiveworkingdays,wherebythistimeframecanbeadapteddependingonuserneeds.

MAPTYPE SL1 SL5

Referencemap

9h 5days

Delineationmap

12h 5days

Gradingmap 12h 5days

Table2-1:Targeteddeliverytimesforthethreemaptypesatthetwoservicelevels.

2.1.1.1 REFERENCEMAPSReferenceMaps serve thepurposeof providingbackground informationon the areahit by thedisaster.UsuallytheyarebasedonsatelliteorVeryHighResolution(VHR)ortho-imagerycapturedclose before the event. They contain available comprehensive information to support disastermanagement measures, including selected basic topographic features and exposed assets andfacilities.

Referencemapsnormallyincludethefollowing(topographic)features(Table2-2):

Featureclass Examples/Itemdescription

GeneralInformation areaofinterest,sensorfootprint,sources,noimagedata

Administrativeboundaries

relevantadministrativeboundarylevels

Settlements agriculture, commercial, educational, industrial, institutional,medical,military,recreational,religious,residential,transportation

Hydrology coastline,dam,lake,reservoir,river,stream,canal,waterway

Pointofinterest educational, industrial, institutional, medical, religious,transportation,cemetery

Physiography contourlineandelevation(m),heightabovesealevel(m)

Utilities powerplant,quarry,watertreatmentplant

Transportation motorway, primary road, secondary road, local road, tramway,subway/metro,railway,station,bridge,tunnel,aerodrome,runway,harbour

LandUse–Landcover bare soil, cropland, grassland, scrub, woodland, forest, wetland,firebreak

Table2-2:Mainitemsofreferencemaps.



DeliverableID:D3.1


DepictingtheexposurewithintheAOI,ReferenceMapsmayincludesupplementarysummarytablesprovidingoverviewinformationonexposedpopulation,settlements,transportation,utilitiesorlandusewithinthemap’sAOI(Figure2-2).

Figure2-2:Exampleofareferencemapdepictingthepre-eventsituationregardingseverefloodingsinJune2016inBavaria(Germany).Themapdisplaysbasictopographicfeatures.Itwasproducedusingpublicdatasetsandpre-eventimagery.

2.1.1.2 DELINEATIONMAPSBasedonpost-disastersatelliteimages,DelineationMapscontainanassessmentofthegeographicextentoftheevent.Ifrequestedtheevolutionofaneventisoutlined.Dependingonthetypeofevent, Delineation Maps may depict flooded or burnt areas, or they provide an outline ofearthquakeimpactareaorthelocationandextentoflandslides,forexample.

Delineationmapsdisplaycrisisinformationaswellasreferenceinformation.Henceasamainitem,additionally to the before mentioned set of topographic features included by reference maps,delineationmapscontainalayeroutliningtheaffectedarea(Table2-3).


CrisisInformation Geographicextentoftheaffectedarea;e.g.floodedarea,burntarea

Table2-3:Additionalavailablefeaturesofdelineationmaps.



DeliverableID:D3.1


DepictingthedisasterconsequenceswithintheAOI,delineationmapsmayincludesupplementarysummarytablescontaininginformationandassessmentsregardingexposedoraffectedpopulation,settlements,transportation,utilitiesorlandusewithinthemap’sAOI(Figure2-3).

Figure2-3:ExampleofaflooddelineationintheareaofCarlisle(UnitedKingdom)producedafterStormDesmondcausedsignificantsurfacefloodinginDecember2015.Themapshowstheextentofthefloodedareaattwotimes.Itwasobtainedfrompost-eventimageryusingasemi-automaticapproach.

2.1.1.3 GRADINGMAPSGradingMapsshowtheimpactofdamagescausedbyadisasterevent.Inparticulartheyprovideinformationontheextent,typeandmagnitudeofdamages.Theassessmentsarederivedfrompre-andpost-eventsatelliteimages.

Gradingmaps include reference information as well as crisis information specific to the event.Gradingassessmentlayermaydepictdisasterextentandimpactgraderegardingaffectedareasorthedamagetoassetsaswellasotherrelevantinformation.

Providing information on disaster consequences within the AOI, grading maps may includesupplementary summary tables containing data on affected population or assets classified perdamagegrade(Table2-4,Figure2-4).



DeliverableID:D3.1



Areagrading severityofimpactforaffectedareas;e.g.firegrading,floodgrading

Settlementsgrading Affectedassetsclassifiedaccordingtodamagegrade

Transportationgrading Affectedassetsclassifiedaccordingtodamagegrade

Utilitiesgrading Affectedassetsclassifiedaccordingtodamagegrade

Other relevant crisisinformation

Observation relevant in the emergency context; e.g. debris, roadblock,crater

Table2-4:Gradingmap-specificfeatures.

Figure2-4:ExampleofadamageassessmentfortheareaofAmatrice(Italy)aftertheearthquakeoccurredon24thAugust2016.Themapdisplaystheassessmentofdamagetobuildingsandtransportationinfrastructureaswellasfurther crisis information,which indicatesblocked roads, landslides,debris and tents. The crisis informationwasderivedfrompost-eventsatelliteimagebyvisualinterpretation.



DeliverableID:D3.1


2.1.2 RISKANDRECOVERYMAPPINGTheRiskandRecoveryMappingmodulesupportsdisasterriskmanagementmeasuresnotrelatedtoimmediatecrisisresponse.Withinaproductiontimeofweeksormonthsitprovidessophisticatedproducts on demand, aiming at the support of prevention, preparedness and reconstructionactivities.Inrespecttoawiderangeofemergencysituationstohandle,productsareprovidedondemandspecifictotheneedsrelatedtothetypeofeventandthephaseofcrisismanagementcycle.

Ingeneraltheinformationcontainedinmapproductscanbecategorisedintopographicfeatures,disasterriskinformationandtailoredinformationrelatedtotheeventandthecrisismanagementcyclephase.

Insupportofspecificcrisismanagementtasksrelatedtoemergencyphasesthreeproducttypesareoffered,addressing1)prevention,2)preparednessand3)recovery/reconstructionphase.

2.1.2.1 REFERENCEMAPSRelated to disaster risk reduction, reference maps contribute comprehensive and detailedinformationontheterritoryandexposedassets.Informationspecifictopreventionproductsassistintheinventoryofassetsatrisk,intheplanningofprotectivemeasuresandthedevelopmentofawareness-raising activities. Maps may contain for example information on existing mitigationmeasuresorlandusezoningplanstakingaccountofdisasterriskinformation.

2.1.2.2 PRE-DISASTERMAPSRelated to disaster preparedness, pre-disastermaps provide relevant information in support ofdisastermanagementmeasurestominimisedisasterimpact.Theproductsforexamplecanassistinthepreparationofresponseoperationsandrescuemeasures.Inthiscontextthemapsmayincludeinformation on hazard exposure, vulnerability/resilience, risk status for population and assets,evacuationplansandforecasts.

2.1.2.3 POST-DISASTERMAPS

Relatedtorecovery/reconstructionphase,post-disastermapscontaininformationrelevantbeyondimmediate response phase. The products support recovery measures aiming to re-establishconditionstopre-disasterstate.Forthatpurpose,themapsforexampleprovideassessmentsoflong-termimpactandrecoveryneedsaswellas informationonrecoveryplanning.Further,theymaycontaininformationregardinghazardexposure,vulnerabilityandresilience,particularlywithregard to new assets or they aim for a progress monitoring of reconstruction efforts andrehabilitation.



DeliverableID:D3.1


2.2 COPERNICUSEMSDATAPORTAL

AWeb-basedportalprovidesaccesstotheCopernicusEMSMappingproductsandtotheEMSEarlyWarningmodules EFAS and EFFIS (emergency.copernicus.eu). It acts as a central repository forrelevantinformationmaterialinparticularregardingtheEMSMappingcomponentandoffersthedownloadofServiceRequestForms,UserGuide,etc.

All Rapid Mapping and Risk & Recovery Mapping activations are featured on a map and in asearchablelist,whichcanbefilteredbasedoneventtype,eventdate,activationstatusandaffectedcountry.Theavailableproductsofanactivationcanagainbe filteredbymaptypeand location.Additionally metadata is provided for each activation, containing information on event time,activation time, event type, type and number of produced maps, activation status, affectedcountries,areadescriptor,nameofactivator,activationreasonandrequestedproducts.

For the EMSMapping Services GeoRSS feeds are available to receive regular content updatesrelatingtocurrentorpastactivations.

Furthermore,theportalfeaturestheCopernicusMapCoveragePlannertool,whichfacilitatesthedefinition of the geographical extent for the preparation of a following EMS -Mapping ServiceRequest.Additionally amap is provided that aggregates activationsof several other EmergencyMappingorganizationsbesidestheCopernicusMappingservices.

TheEMSportalprovidesgeospatialinformationin196activationssofar:

NumberofActivations

RapidMappingActivations 167

Risk and Recovery MappingActivations

29

total 196

Table2-5:Countofactivations.



DeliverableID:D3.1


Figure2-5:CopernicusEMSDataPortal;RapidMappingactivationprovidingmetadataandaccesstorasterandvectorproducts.



DeliverableID:D3.1


2.2.1 OVERVIEWONDATAPROVISIONTheproductsofferedbytheCopernicusEMSMappingServicesareprovidedasdigitalmapoutputsandasvectordatasetsontheEMSportal.

Ingeneral,allproductsimplementastandardizedstructureasdescribedbelow.Followingspecificrequestsorforusabilitypurposescase-by-caseadaptionsmaybeapplied.

Bydefault,thedigitalmapsareavailableindifferentdeliveryformatsandresolutions:

Printablemap FullcolourISOA1,orequivalent

Resolution:high=300dpi;medium=200dpi;low=100dpi

GeoPDF

Metadatafile

Georeferencedmap

FullcolourISOA1,orequivalent

Resolution:high=300dpi;medium=200dpi;low=100dpi

GeoTIFF,GeoreferencedJPEG(withworldfile)

Metadatafile

Table2-6:Specificationsofprovidedmapproducts.

MapproductsareusuallyprovidedinUTMcoordinatesystemusingWGS84datum.Mapscaleandthe covered area of themaps vary according to user requirements and the size of the area ofinterest.Downloadable rastermap files includemetadata informationonpublicationdate,mapscale and the product version. Map file names (e.g.EMSR194_05FARINDOLA_GRADING_OVERVIEW_v1_300dpi.jpg) include activation code, productcodeandname(referringtoaffectedlocation),producttype,productversionandresolution.

Additionally,allgeographicfeaturesthatarederivedfromtheanalysisandcontainedinthemaps,areprovidedas vector files. Theavailabilityof georeferenceddatasets in vector format is a keycomponentoftheCopernicusEMSservicedelivery,asitallowsusingthegeneratedinformationinsupportoffurthergeospatialanalysis.VectorfilesaredeliveredinESRIshapefileformatorinGoogleEarthKML(orKMZ)format,includingametadatafile.Foreachmapcomponentassociatedwithanactivationafullsetofvectorfilesisdelivered.Relevantattributesfordataclassificationaccordingtothelegenditemsofassociatedrastermapsareincluded.Bydefaultself-explanatorynamesanddescriptivecontentsareassignedtoattributes.



DeliverableID:D3.1


Availablefeatureclassesandcommonattributestructure:

Figure2-6:Mostlyavailablegeographicfeatures,labeledwithbasenames,andcommonattributes.

Detailed feature class names are applied as shown in the following example:EMSR194_05FARINDOLA_02GRADING_v1_7000_crisis_information_poly_grading.shp

Featureclassnamesgenerallyrefertothenamesusedforthelegenditems/headingsinthemapproducts.Dependingontheitem,geometrytypecanbepolygon,lineorpointandfilenamesmayindicatethegeometrytype.Incasealayerincludesgradinginformationanameaffixisaddedtothefilename.Furthermoreascaleparameterreferstothemapscale.Inherenttomapscaleisthelevelofdetail,affectingassetspolygons,especiallysettlementsdigitalisationgranularity.

Apartfromthat,thenomenclatureisconsistenttotherastermapproducts,includinginformationon activation code, product code and name (referring to affected location), product type andproductversion.

Allrelevant information,regardingtheCopernicusEMSServicewasgatheredinRD01,RD02andRD03.



DeliverableID:D3.1


2.3 COPERNICUSEMSINTEGRATION

Allthedatainserted,arecomingfromCopernicusEMSportal,whichareautomaticallyparsedbyaPythonscripttoretrievethelatestactivationsfromthewebsite.Inthefollowingparagraphs,theaccessibilityoptionstothesedataaredescribed.

BesidesthepossibilityofdownloadingdatamanuallyfromthewebsiteofCopernicus,therearenotmanyoptionstogetautomaticaccesstodisastermapsandmetadatathereof.FTP-ServersareonlyaccessibleforuserswhohaveactivatedtheEmergencyManagementService(EMS).Therefore,thisprojectmakesuseoftheGeoRSSfeedthatisprovidedonthesamewebsiteforallemergencyeventtypes(cf.http://feeds.feedburner.com/CopernicusEMSMappingRushModeActivations).

AGeoRSSfeedisanemergingstandardthatintegrateslocationdataintoaclassicalwebfeed.TheGeoRSSfeedofCopernicusEMSisupdatedaccordingtotheactivationtimeofaspecificeventthathasbeentriggeredbytheresponsibleadministrationorauthority.Usuallythereisatimedifferencebetweenthetimewhentheeventhasoccurred(EventtimeLOC)andtheactualactivationthatmaytakeplacea fewdaysafter theevent.TheGeoRSSalertsabout themost recentactivationsandprovidesanattributelisttoidentifythesingleevents:

• Titleofactivationincludinganemergencynumber

• Detailsandmaps

• Eventtype

• Eventtime(localtime,LOC)

• Activationtime(coordinateduniversaltime,UTC)

• Affectedcountries

• Areadescriptor

• Activationreason

TheabovelistedinformationcanbeusedinEMSdataintegrationmodulefortheautomatictransferofthegeodataintothebackendsystem.Furthermore,itrepresentsvaluableinformationabouttheshowngeodatafortheenduser.

Alsofortheupdateofthedatabasetables,containingallthepreviouslymentionsinformationabouttheGeoRSSfeedaswellasthemetadatainformationabouttheprovidedmaps,anautomatedservicewillbeusedtoactivelymonitortheGeoRSSfeedforupdates.Thisprovidesatimelyresponsebythedataintegrationmodulefornewactivationsandupdatingofthedatabasewhennewmapsaremadeavailable.



DeliverableID:D3.1


2.3.1 DESIGNOFEMSINTEGRATIONMODULE

The analysis of the data shows that disaster maps and forecast layers are vector based data.Thereforethecollectionofthesedatasetsinageospatialdatabaseisseenasviableandpracticalsolution for a data storage. A database also offers a basically unlimited and therefore scalablesolution for storing all geospatial data. Through the storage of layers in a spatial database, thearchivingofdataisenabled.Thus,historicflooddatacanbeaccessedatanytime, ifdemanded.Regardinggeodataprocessingcapabilitiesandspatialfunctions,thePostgreSQLdatabasewiththespatialextensionPostGISwasthemostreasonableandsuitabledatabasemanagementsystemforthedataintegrationmodule.EspeciallywiththespatialextensionPostGIS it ispossibletocreatehigh-performancesolutionsforgeodataprocessingandvisualization.PostgreSQLalsosupportstheconceptualworkflowofimportingtriggereddisastermapsbetterthanotherDBMSduetothenativehandlingofshapefilesandrasterfiles.

Thedevelopmentofalargelibraryofspatialfunctions,theenablingofspatialindices,thepossibilitytoautomateprocessingwithPythonscriptsandtheinteroperabilitywithGeoServerandotherOGCconform software makes PostgreSQL superior over other database system candidates for theintegrationandstorageofCopernicusEMSlayer.

Thedesignofthedatabaseislooselybasedonthepythonscript,whichrunsinthebackgroundfortheautomaticallyingestionofthedataandontheprovideddatastructureoftheCopernicusEMSwebsite.Therefore,threedifferenttableshadcreatedbutonlyonedoescontaingeospatialdata.Thefirsttablestoresallthemetadatainformationbelongingtoanactivation,suchasthetitle,dates,area,countryorthestatus.Aslongasthestatusisnotsetto“closed”,theactivationwillbecheckedineachrunofthePythonscript ifnewmapsweremadeavailable.ThesecondtablecontainsallmetadatainformationaboutaprovidedZIPfile,whichisdownloadedbythePythonscript.Here,informationaboutfilename,featurecount,geometrytype,publication,scaleandtypeofmapwillbestored.Inthelasttablealltheinformationabouttheattributesinashapefilearepresent.Sinceall provided datasets have more or less similar structure, only those attributes with the mostmeaningfulvaluesfortheend-usersarechosenfortheimportintothedatabase.Theillustrationbelowrepresenttheuseddatabasemodel(Figure2-7).



DeliverableID:D3.1


Figure2-7:Databasemodel.

Allthetableslinkedwithforeignkeyssothatcomplexqueriesoverallthethreetablesarepossible.Allattributes,whichcannotbefoundwithinthethreetables,havebeenneglectedforthisproject.

2.3.2 IMPLEMENTATIONOFEMSINTEGRATIONMODULEThissectionsummarisestheprocessingchainandimplementationofthesystemandstartswithanoverviewofthearchitecture:

Figure2-8:EMSIntegrationModuleprocessingchain.



DeliverableID:D3.1


TheprocessingchainbeginswithanewdisasteractivationpublishedontheCopernicusEMSviatheGeoRSS feeds.This triggerbeginsaseriesofprocessingstepsthatarenecessarytoproducethedatabasetableentry,whichthenaredeliveredtoIDIofthecloudinfrastructure.TheGeoRSSusedtotriggertheprocessingchaincarriestheinternetaddress(URL)oftheEMSactivation.

Using this information, theprogramcan thenaccess theEMSwebsiteandharvest the requiredinformationtocontinue.EMSactivationsproducemapproductsfocusedonthecrisisregion.Thecoordinatesusedforeachofthemapproductsarethegeospatialinformationrequiredtocontinuetheprocessingchain.TheCopernicusGeoRSS feed isproviding information innear-real time foreach occurred disaster event according to the activation time. The rush mode event mappingincludesworldwidedatawhichwillbeallincludedinourdatabase.AuniqueURLdefineseachevent,whereallrelevantmapsandmetadataarepackedinZIPfiles,whicharedownloadedautomaticallythrough HTML parsing and website control functions, e.g. confirming the disclaimer of liabilitybeforeeachfiledownload.

TheparsingfunctionsdefinetheexactpathtothestoredZIPpackagesviatheCopernicusGeoRSSfeedHTMLsourcecode.ThefilesareunpackedlocallyonthedataintegrationbackendliveserverandingestedinthePostgreSQLdatabase.Meaning,eacheventincludesseveralmappinganddetailversions. They are identified through unique namespaces: One shapefile per event and detailversion is loaded feature-wise into all three tablesof thePostgreSQL system,mentioned in theprevious section. The geometryof each feature is stored in a geometry columns in thePostGISformat“geometry”,whichenablesthepossibilitytoeasilyrunGISfunctionalitiesovertheinsertedfeatures.AsthenativecoordinatesystemofCopernicusgeodataisbasedonaUTMprojection,alllayersmustbetransformedtoEPSG3857(WGS84WebMercator),aprojectionsystemforwebandmobileapplications.

The storage, processing and projection of geodatawithin the spatial database is done throughPostGIS functions thathaveproven tobe very efficient for thepurposeof this data integrationmodule.Themainadvantagefortheingestionofthoselayersintoadatabaseisalsotheprovisionofattributedata,spatialindexingandmanyotherspatialanalysisfunctionsthatmightbeusedondemand.InordertosendthepreparedandreadytousedatafromthePostgreSQLdatabasetoI-REACTinfrastructure,shapefileswithauniquenamewillbecreatedforeachshapefileprovidedbyCopernicusEMS,whichhasbeensuccessfullyingested.Thisallowsthepossibilitytoaddonlythoseattributestoshapefiles,whicharenecessaryforthemappingofthedataintheI-REACTfront-end.This step is carriedoutwith the command line toolpgsql2shp,which is included in thePostGISextensionofthePostgreSQLdatabasesystem.Withthehelpofpre-definedSQLstatements,onlythe desired data can be retrieved from the database by adding the statement to the PostGIScommandlinetool.Aftertheshapefileswerelocallyproduced,theywillbecompressedandzippedinordertofacilitatethehandlingofthedatawithintheI-REACTback-endintheAzurecloud.ViaHTTPrequestdatacanbesendviatheweb.

TheIDIoffersanAPIinordertoauthenticateusers,querythemetadatatablewithintheI-REACTinfrastructureandinsertnewdatasetstoback-endofAzurecloudsystembyusingRESTfulservicesandHTTPrequesthandler.Foreachzippedshapefile,metadatainformationwillbequeriedfrom



DeliverableID:D3.1


thePostgreSQLdatabaseandstoredinaso-calledPythondictionaryvariable.HereinformationfromallthreetableswillbecollectedtosatisfytherequirementsoftheIDIAPIinputparameters.Thosedataarerelevanttocreateinalaterstageametadatadocumentabouteachlayerprovidedinthefront-end.AfteronedatasetanditscorrespondingmetadatahasbeensuccessfullysendtotheIDIviaaPythonrequest,thenextdatasetwillbeprepareduntilalldatahavebeensuccessfullysend,belongingtooneactivation.This import logic istriggeredeverytimeanewactivationwasmadeavailableonCopernicusEMSwebsite.



DeliverableID:D3.1


3 EUROPEANEARLYWARNINGSYSTEMSINTEGRATION

3.1 SERVICESOVERVIEWOFTHECOPERNICUSEWS

TheEuropeanearlywarningsystemsaremadeupoftwoCopernicusservices:theEuropeanFloodAwarenessSystem(EFAS)andtheEuropeanForestFireInformationSystem(EFFIS).Thesesystemsprovide early warning andmonitoring of flood and fire disasters respectively across Europe toauthorised users. They are both under the Copernicus emergency management service (EMS)(RD04).

3.1.1 THEEUROPEANFLOODAWARENESSSYSTEM(EFAS)TheEuropeanFloodAwarenessSystem,alsoknownastheEuropeanFloodAlertingSystem–EFAS–hasbeenindevelopmentsince2005afteraEuropeanCommissiondecisionwasmadebasedontheaftermathofthedevastatingElbeandDanubefloodsof2002.EFASwasalsothefirstCopernicusservicetobecomeoperationalwhenitbecamepartoftheEmergencyManagementServiceoftheCopernicusInitialOperationsandinsupportoftheEuropeanCivilProtection.Theservicehasbeenfullyoperationalsincethethirdquarterof2012.

Asanoperationalservice,EFASistaskedtoforecastandmonitorfloodsacrossEurope.TheprimaryclientsoftheservicearetheNationalHydrologicServicesaswellastheEuropeanResponseandCoordinationCentre(ERCC).EFASisabletoprovidethemwithprobabilisticfloodforecastswithanearlywarningwindowofupto10daysinadvance.Inordertoprovidethisoperationalservicetherearefourcentreschargedwithexecutingthedifferentpartsoftheservice:

1. TheEFASComputationalcentreiswherethefloodforecastsareproducedandtheEFAS-InformationSystemplatformishosted;

2. TheEFASDisseminationcentreiswheretheEFASoutputsareanalyseddailyandtheinformationdisseminatedtothepartnersandtheERCC;

3. TheEFASHydrologicaldatacollectioncentreiswhereriverdischargeandwaterleveldataacrossEuropeiscollected.Bothhistoricalandreal-timedata;

4. TheEFASMeteorologicaldatacollectioncentreiswherethemeteorologicaldataacrossEuropeiscollected.Again,bothhistoricandreal-timedata.

Operationally,eachof thecentreshasbeencontractedout todifferentconsortia.Detailsof thepartnersinvolvedcanbefoundonthewebsite.

Thepubliclyavailabledataareonlypastforecastsfromthesummerof2014becausethearchivetoolisonlyaprototype.Error!Referencesourcenotfound.showstheuserinterfaceofthepubliclyavailable service website [RD05]. The real-time flood forecasts are only made available to the



DeliverableID:D3.1


national/regional partner institutes. Furthermore, the actual EFAS forecast cannot be publishedopenlyduetoservicelicensingrestrictions.

Figure3-1:TheEFASforecastingwebsitethat ispubliclyaccessible.Notethatthepublicservicedoesnotprovidecurrentfloodforecasts.

3.1.2 THEEUROPEANFORESTFIREINFORMATIONSYSTEM(EFFIS)EFFIS-theEuropeanForestFireInformationSystemisinchargeofsupportingservicesrelatedtothe protection of forests against fires across Europe. It was established by the EC in 1998 incollaborationwiththenationalfireadministrationsandissupportedbyanetworkofexpertsfromEUcountriesaswellasneighbouringcountries.ThisExpertGrouponForestFiresisregisteredundertheSecretariatGeneraloftheEuropeanCommission.EFFISalsoprovidesservicestootherbranchesof the EC such as the ERCC and provides harmonised forest fire information to the EuropeanParliament. It became a component of the Copernicus EMS in 2015. Recently, EFFIS has beenextended toprovideglobalwildfire information through theGlobalWildfire InformationSystem(GWIS).

Fordownstreamwildfiredisasterservices,EFFISprovidesanumberofinterestingdatawithalicensethatgenerallyappearstobeopenbutnotstandard[RD06].Twoofthemostwidelyusedservicesarethefollowing:

• CurrentSituation– forecastsofup to6days inadvance frommeteorological firedangermapsforthecurrentEuropeanfireseasonaswellasupdatedmapsoffirehotspotsandfireperimeters;



DeliverableID:D3.1


• FireNews–dailyupdatesofnewsarticlesrelatedtowildlandfiresacrossEuropeselectedandgeo-locatedbytheEFFISteam.

Thecurrentsituationviewer(Error!Referencesourcenotfound.;RD06)providesthepossibilityforuserstovisualisesomeoftheinformationmadeavailablethroughEFFISincluding:

• FireDangerForecast;

• RapidDamageAssessmentand;

• AnalysisTools.

Figure3-2:TheEFFISCurrentSituationviewer.Thisviewerprovidescurrentseasonwildfiresituation informationacrossEurope.

However this is a visualisation tool and thedatadisplayed cannot bedirectly used as part of adownstream service. Customers of downstream services are interested in the following pre-firestateandfiredatabaseEFFISmodulesmentionedontheservicewebsite:

• FireDangerAssessment–basedon theCanadian Forest FireWeather Index (FWI)usingcurrentmeteorologicalforecasts.Wildfiredangerismappedintosixclassesfromverylowtoextreme;

• RapidDamageAssessment–includesactivefiredetectionfromsatelliteimageprocessingofthermalanomaliesandthemappingofburnedareas;

• EmissionsAssessmentandSmokeDispersion–notcurrentlypubliclyavailable;

• PotentialSoilLossAssessment–notcurrentlypubliclyavailable;

• VegetationRegeneration–notcurrentlypubliclyavailable;

• FireDatabase–detailedindividualwildfirerecordsprovidedbyEFFISnetworkcountries.



DeliverableID:D3.1


EFFISalsoprovidesavisualisationtoolforthefiredatabase(Error!Referencesourcenotfound.;RD07)whichatthetimeofwritingprovidedupdatesonlyupto2014.

ItshouldalsobenotedthattherearesomedataavailablethroughtheGWISthatarenotprovidedthroughtheEFFISviewer.Forexample,fireemissionsspeciesthatareprovidedbytheCopernicusAtmosphereMonitoringService(CAMS).

Figure3-3:TheEFFIShistoricalwildfireviewershowingresultsfor2014

AsshownontheEFFIS‘dataandservices’webpage[RD08],allthedataiscurrentlyonlyavailableasWMSwhichisnotidealfordownstreamserviceswantingtotakeadvantageofthedatatoproducenewservicesandnotonlyviewtheinformation.



DeliverableID:D3.1


3.2 COPERNICUSEFASDATAPORTAL

TheEFASdataportalcanpotentiallyprovideawiderangeofimportantinformationtodownstreamservicesinthecontextofflooddisasterearlywarning.BasedontheEFASarchiveprototype,thefollowinggeospatiallayersareavailable:

• Floodsummarylayers;

• Hydrologicallayers;

• Meteorologicallayers;

• InitialConditionslayers;

• Backgroundlayers;

• Flashfloodlayers.

ThemostimportantdatalayerstotheI-REACTprojectarethehydrologicallayers,flashfloodlayers,andpotentiallythefloodsummarylayers.

The hydrological layers provide the flood forecasts for Europe, which are essential to anydownstream early flood disaster warning services. EFAS has the best European wide floodforecastingsystemthatisabletoprovidefloodforecastswithleadtimesgreaterthananyoneelsein the EU. This capability would greatly enhance the I-REACT services portfolio and provide animportantbasisfornewflooddisasterrelatedservices.Unfortunately,atthistime,suchlayersarenotpubliclyavailableorprovidedasopendata.Thisisduetothelicensingissuesbetweenthelargenumberof EFASpartners thatmakes this servicepossible. It is alsodue to the fact that not allMemberStatesarecomfortablewithprovidingdisasterwarninginformationimmediatelyintothehandsofthepublicwithoutfirsthavingtheirauthorisedexpertsconfirmthefloodwarningandthenreportthewarningthroughtheestablishedchannelsfordisasterriskalerting.

Fromadownstreambusinessserviceperspective,thisisveryunfortunatebecauseitdoesnotallowfortheexploitationofpubliclyfundedservicesfornovelapplications.However, it istheprojectsunderstanding that the EC is aware of this and is working towards trying to find a mutuallyadvantageoussolutioninthenearfuture.

TheI-REACTprojectandthisparticulartaskcouldnotjustwaitandseewhethertheissuewouldberesolvedbecauseserviceswerepromisedandneedtobedelivered.Therefore,task3.2–EuropeanEarlyWarningSystemsIntegrationthatneededtocouplefloodforecastingfromEFASneededtofindadifferentsolution.



DeliverableID:D3.1


3.3 COPERNICUSEFASINTEGRATION

Asmentionedintheprevioussection,directintegrationoftheEFASfloodforecasts,specificallythehydrological layers was and currently is not possible through direct access using commonapplicationprogramming interfaces (API)and/oropenservices.For this reason, I-REACTworkedcloselyincollaborationwiththeJRCandthesub-contractorKajoS.R.L.toovercomethisissue.Atthebeginningoftheproject,anemployeeofEOXPLORE,thetaskleaderforT3.2wasinvitedasavisitingscientisttoworkonthepremisesoftheJRCinIsprawheretheEFASteamislocated.Overthe threemonthswhen the visiting scientist wasworkingwith the EFAS team, the goal of thecooperation was to identify applicable flood forecasting results and define the processingmethodologysothatitcanbemadeavailabletotheI-REACTpartners.

This section presents the details of the procedure that translates EFAS discharge forecasts intoclassifiedfloodhazardmapsthatareingestedandprocessedinsideI-REACT.Itshouldbenotedthatwhile the JRCandthe I-REACTconsortiumareconstantly looking forways toextendtheservicecooperationbeyondtheproject timeline, it ispossible that futureaccesswillbestoppedpost-I-REACTduetocurrentdatalicensingandsharingrestrictions.

The flood forecast information that is integrated into the I-REACT service is a procedure thattranslates EFAS discharge forecasts into classified flood hazardmaps. This procedure is runningconcurrentlyandrespectingthetimescheduleoftheoperationalEFASsuite.Therefore,processeddataaremadeavailabletwicedailybasedontheEFAS00:00and12:00forecasts.TheprobabilityoftheEFASECMWFensemble (ENS)based forecasts thatexceed the5-and20-year returnperiod(derived from EFAS long term climatology) threshold is taken as ameasure of the actual floodhazard.The5-and20-yearreturnperiodwasderivedfromEFASlong-termclimatologystudies.Inthis way, the range of ensemble forecasts is taken as a measure of the probability of floodoccurrence,whereasfloodforecastreturnperiodsallowfortheestimationofthemagnitudeofthepredictedfloodevents.

TheresultsoftheEFASfloodforecastsareoutputinrasterformat.Ateachgridcell,themedianoftheensembleforecastgivenbythe latestEFASprediction is first identified.Then, themaximumdischargeofthemedianoverthefullforecastingperiod,i.e.10days,isselected.Thisselectedvalueisusedtoderivethetimingofthefloodpeak.Thefloodpeakisconsideredasthelead-time,indays,whenmaximumdischargeistooccur.

In the proposed classification (seeError! Reference source not found.), the total probability ofexceedingcertainEFASthresholdsandthepeaklead-timesareusedtoclassifythepotentialfloodhazardateverygridcellinto3distinctclasses:low,medium,high.

Table3-1:Floodhazardclassificationmatrixbasedontheproceduredescribedinthetext.

Peaklead-time

Probability <24H 48-72H >72H

10-20 Medium Low No

20-50 High Medium Low

>50 High High Medium



DeliverableID:D3.1


Thefirst floodhazardscenariousesthe“raw”EFAS information.RawEFAS informationassumesthatnofloodprotectionmeasuresaretakenintoaccount.Thisscenarioprovidesthefloodshazardassessmentbasedonthenaturalhydrologicalresponseofthesystemand it isuptotheusertoevaluate the potential effects based on the known local flood protection infrastructure. Animportant limitation of this approach is the fact that there is a higher number of false alertsproduced.

In the second scenario, there is the inclusion of some information related to flood protectioninfrastructure. The data on flood protection was compiled based on risk-based estimations forEuropedevelopedbyJongmanetal.(2014).Thisinformationwasintegrated,whereavailable,withtheactuallevelofprotectionfoundthroughliteraturereviewand/orassessedbylocalauthorities.

Finally,thepredictedEFASstreamflowiscomparedtothe localfloodprotection level.Rivergridcells where the protection level is exceeded are considered to activate the impact assessmentprocedure.The floodprotection levelsareprovidedas the returnperiodof themaximumfloodeventthatcanberetainedbythelocaldefencemeasures,e.g.dykes.Thefloodhazardinformationisthenclassifiedasdescribedabove(Error!Referencesourcenotfound.).

Figure3-4:FloodprotectionlevelsacrossEuropebasedontheJongmanetal.(2014)methodology.

3.4 I-REACTPROCESSINGOFEFASDATA

ThegoaloftheI-REACTprojectisnotonlytointegrateCopernicusdataintothedownstreamservicebutalsoaddvaluetoitfortheusersanduseittodevelopnewproducts.ThissectiondescribesthemethodologythattransformstheEFASfloodhazarddatatotheproductthatisvisualizedinI-REACTbytheuser.



DeliverableID:D3.1


AnobjectiveoftheI-REACTprojecttaskT3.2–EuropeanEarlyWarningSystemsIntegration,istointegrateEFASdata.BeforedescribingthismethodologyandtoavoidconfusionitshouldbenotedthatwithintheI-REACTsystem,thewordtaskhasalsobeenadoptedtodescribeapieceofdataand/or information that isbeing ingested into the system.Whileamore in-depthdescription isfoundelsewhere (seeWorkPackage5documentation), thisnote is to clarify for the reader thedifference.AnI-REACTsystemtaskhasfourdigitstoidentifythedata.Forexample:task3201.ThisistheeasiestwaytodifferentiatebetweenprojectTasks(e.g.3.2)andI-REACTsystemtasks.

3.4.1 AFFECTED COMMUNITIES BASED ON THE EFAS 5-YEAR RETURN PERIODFORECAST

Thefirstdataprocessingworkflowtobedescribed is the ingestionofEFAS5-yearreturnperiodfloodhazardforecastdatatodelineatepotentiallyaffectedcommunities. IntheI-REACTsystem,this data task number is 3201 and involves the processing of the EFAS data to the NUTS(Nomenclature of territorial units for statistics) 3 administrative level [RD09].NUTS levelswerechosenbecausethishierarchicalsystemfordividinguptheEUterritoryhelpsharmonisethedataacrossEuropeandhavebeendelineatedforspecificdiagnoses.

Error!Referencesourcenot found.describestheprocessingchainthat isusedtogeneratetask3201.ThemajorityoftheprocessingtasksarebasedontheuseofOpenSourcetechnologiesbothtogatherandthenprocessthedata.

Asmentionedintheprevioussection,thefloodhazardclassificationdataisproducedatthesametimeastheEFAS00:00and12:00forecasts.Therefore,thetaskprocessingisscheduledtorunsoonafter the expected time that the data would be made available. Any kind of forecasting iscomputationally intensive. Furthermore, the EFAS flood forecasting is based on other weatherforecastoutputsandeventhoughtheyareoperational, issuescanoccurthateitherprolongtheprocessing or stop the forecast from being produced. Consequently, the task 3201 processingactuallychecks fornewfloodhazarddatathreetimesdaily incasethefloodforecastswerenotproducedontimeinordertocapturetheEFASdataassoonasitisavailable.ThisistoensurethatthemodelcanberunandproducethewantedoutputandfinallydeliveredtotheI-REACTIDIasquicklyaspossibleafternewfloodhazardforecastinformationisavailable.

At the moment, the scheduling of the process is not in-line with the current state-of-the-arttechnologiesthatarebeingimplementedintheI-REACTsystem.Ideally,onewouldliketheEFASdatatoautomaticallyletthetask3201processknowthatthedataisreadyforprocessing.Atthistime,thefloodhazardinputdataisprovidedviaFTPthatrequiresthetask3201tocheckwhetherthereisanewfilebasedontimestampandagreeduponfilenamingconvention.Thisissueiscausedby the fact that EFAS does not provide any public API’s to connect with the outputs that areproduced. However, this may change in the future and would certainly improve downstreamprocessing.Thepartnerswillupdatetheprocessifandwhenthereareserviceupgradeswithrespecttothisissue.



DeliverableID:D3.1


Thediagrambelow (Error!Reference sourcenot found.) describes theprocess toproduce task3201. The EFAS flood hazard forecast data is taken directly from a secure FTP1 (File TransferProtocol)server.AccesstotheFTPserverrequiresauthenticationbecauseoftheEFASdatalicensingand sharing restrictions. Since the availability of the data varies from day to day, i.e. not allscheduledEFASrunsaresuccessful,thetask3201processneedstokeeptrackofthelastsuccessfulprocess.BykeepingtrackofthesuccessfulEFASoutputs,thetask3201startsthecurrentprocessfromthistimeplus1day.Duetolateand/orincompletefloodhazardforecastruns,thiscouldbethemodelrundateminus1to2days.

Asshownintheprocessdiagram,thereisalocaldatabasetablethatkeepsthisinformationcurrent.The PostgreSQL2 database table is queried based on in-house developed Python3 programminglanguagecodetoqueryforthelastsuccessfulfloodhazardforecastrundate.

Thetask3201processthendownloadstheEFASarchivefilecontainingtherasterdatainGeoTiff4formatted files for the floodhazard forecastbasedon the5-year returnperiodand the20-yearreturnperiod.

Figure3-5:DescriptionoftheprocessingchainfortheI-REACTtask3201fortheproductionofthe5-yearreturnperiodfloodhazardforecastoutput.

OncetheEFASdataisstoredlocallyandextractedfromthearchive,thetask3201modelusesthePython library rasterstats, to extract the statistical values. Rasterstats is a Python module for

1https://en.wikipedia.org/wiki/File_Transfer_Protocol2https://en.wikipedia.org/wiki/PostgreSQL3https://en.wikipedia.org/wiki/Python_(programming_language)4https://en.wikipedia.org/wiki/GeoTIFF



DeliverableID:D3.1


summarizinggeospatialrasterdatasetsbasedonvectorgeometries5.Therasterstatslibraryisrunagainst theGeoTiff rasterandavector fileof theNUTS5administrativeboundaries.ThevaluesextractedfromtheGeoTiffpixelsarethenumberoffloodforecastmodelspredictingapotentialfloodforthatspecificarea.InordertopreparetheresultsforI-REACT,apercentageiscomputedbydividingthevaluefromthepixelbythetotalpossiblenumberofmodelrunswhichis51.ThisresultisthenstoredlocallyinthePostgreSQL/PostGIS6databasethatislinkedtotheNUTS5geospatialID.

APostGISviewthenmatchestheCommuneIDtotheIDofatablecontainingtheCommunedetailsincludingthegeometry.TheresultfromthisviewisthencalledfromthePythonModeltoloadintothespatialdatalibraryGeoPandas.GeoPandas7isanopensourceprojectwhosegoal istomakeworkingwithgeospatialdatainPythoneasier.

Using GeoPandas, the model checks each geometry and flood maximum percentage value asdescribedaboveandaddsstylerelatedcolumnsandvaluesforeachrow.ThisinformationistobeusedinthefinalpartoftheprocesswheretheGeoJSONfileisproduced.

Asshowninthetask3201processfigure(Error!Referencesourcenotfound.),theprocessthensavesthedataobjecttoavectorfileofthetypeESRIShapefile8(SHP).TheShapefilecontainsalltherequireddatabeforeitisfinallyconvertedtothewantedI-REACTstandardvectorfiletypeGeoJSON.TheGeoJSON9geospatialvectordatafileformatisanopenformatandisstandardsbased.ThisfiletypewaschosentobetheprimarygeospatialdataformatforsharingwithintheI-REACTprojectspecificallybecause it is standardsbased,openandeasily ingestedbymanydifferent tools.Theadditionalstep,i.e.convertingtheshapefileintoaGeoJSONfile,isrequiredinordertoallowforthemodeltoproducetheGeoJSONdatabasedonthelateststandardsincludingtherighthandruleforvectorgeometry.

OncetheGeoJSONfilehasbeenproduced,themodelbuildstherequiredIDImetadataobjectthatdescribesthegeneratedGeoJSONfile.ThecodethenusestheI-REACTIDIservicetouploadthedataproduced.Oncecomplete,themodelstoresalogoftheprocesstothelocaldatabase.

ThefollowingFigure3-6showsthefinalGeoJSONresultforasectionofNorthWesternPolandandNorthEasternGermany.TheshapesarebasedontheNUTS5leveladministrativeboundariesthatpotentiallycouldbeaffectedbyflooding.

5https://github.com/perrygeo/python-rasterstats6https://en.wikipedia.org/wiki/PostGIS7http://geopandas.org/8https://en.wikipedia.org/wiki/Shapefile9http://geojson.org/



DeliverableID:D3.1


Figure3-6:Final5-yearfloodhazardforecastoutputinGeoJSONfileformat.Thedarkerthebluecolour,thegreaterthefloodhazardforcast.

3.4.2 AFFECTED COMMUNITIES BASED ON THE EFAS 20-YEAR RETURN PERIODFORECAST

TheI-REACTtask3202isthesameasthetask3201fromaprocessingperspectivebuttheEFASinputisdifferentandthereforeprovidesadifferentendresult.Therefore,manyofthedetailsprovidedintheprocessingdescriptionfortask3201willnotberepeatedhere.

Theprocessfortask3202isrunagainstthe20-yearfloodhazardforecastproducedbyEFAS.Theprocessingdiagram is presented inError!Reference sourcenot found.where the communitiespotentiallyaffectedbyfloodingaredelineatedbasedonNUTS5.

Figure3-7:DescriptionoftheprocessingchainfortheI-REACTtask3202fortheproductionofthe20-yearreturnperiodfloodhazardforecastoutput.



DeliverableID:D3.1


Tohighlightsomeofthedifferencesinthedataandresultsofthetwotasks,thefollowingError!Referencesourcenotfound.presentstheinputdataforagivenlocationforboththe5-yearand20-yearreturnperiods.Notethattheexpectedbehavioristhatwhereverthereisa5-yearforecasta20-yearforecastshouldalsobepresent.However,thereverseisnottruebecauseundernormalconditions,alargerfloodevent(20-yearreturnperiod)shouldnotbeforecastwithouttheabilitytoforecastasmallerevent(5-yearreturnperiod).ThisisshowninError!Referencesourcenotfound.wherethe5-and20-yearreturnperiodforecastsareoverlapping.Indarkblueisshownthe20-yearwhereasthelightbluegridpointsarethe5-yearreturnperiod.

Thenextstepistotakethosegridpointsandmakethemmoreeasilytointerpretandvisualize.

Figure3-8:Comparinga5-year(leftside)and20-year(rightside)returnperiodfloodhazardforecast.Herethe'raw'dataispresentedatthepixellevel.

Figure 3-9: Five- and twenty-year return periods for the floodhazard forecast overlapping. Please refer toError!Referencesourcenotfound.fortheseparatefloodhazardforecasts.

Duetothefactthatdecisionswithrespecttohowtoreacttoaflooddisasterwarningisorchestratedbythenationaland/orlocalauthoritiesresponsible,therelevantregionisprovidedastheoutput



DeliverableID:D3.1


ratherthanapixel.Thisprocessingwasalsoappliedfortheother I-REACTservicesthatneedtoknowplacenamesand/orlocationstomonitorotherinformationsourcesonceawarninghasbeenposted.Forexample,havingplacenamesallowsthesocialmediamonitoringto lookoutforthenameswithrespecttofloodoccurrences.

InError!Referencesourcenotfound.,theNUTS5levelvectordataisplacedinthebackgroundtoshowhowthedifferentgridvaluesintersectwiththeadministrativeboundaries.ThesetwotypesofinformationaremergedtogetherinordertoproducethefinaloutputthatissenttotheI-REACTIDIforsharingwithotherpartnersandvisualization.

The finalGeoJSONoutput ispresentedwith the statistics foreachNUTS5 levelpolygon (Error!Referencesourcenotfound.).Thedifferencesintheblueshadingindicatesthenumberoffloodmodels indicatinga flood.Asdescribed in theprevioussection, therearea totalof51differentmodelsusedandtherefore,themoremodelsthatareforecastingaflood,thegreaterthepossibilityofanactualflooddisaster.

Figure3-10:5-and20-yearfloodhazardforecastwithNUTSlevel5datainthebackground.

Figure3-11:Thefinal20-yearfloodhazardforecastinGeoJSONformat.Theblueshadingprovidesinformationaboutthenumberofmodelsforecastingaflooddisaster.Darker=moremodelsforecastingflood.



DeliverableID:D3.1


It is up to the national, regional, and/or local authorities to interpret the flood hazard forecastwarnings.Thedarkertheblue,themoretheyshouldbepayingattentiontotheirlocalforecastsandputting intomotion their respective action plans. Each polygon that is provided to the I-REACTsystemalso provides the distribution of the floodhazard forecast values. This essentially is thenumberofmodelsoutof51possiblethatare indicatingriverfloodpotentialbasedonthemostrecentweatherforecasts.

3.5 COPERNICUSEFFISDATAPORTAL

The European Forest Fire Information Systemprovides standardisedWebMap Service10 (WMS)interface that is OGC compliant. From the EFFIS data and services webpage11, it is possible todownloaddata.TheError!Referencesourcenotfound.providesalistingofthedifferentdatasetsmadeavailablethroughWMS.

Table3-2:AvailableEFFISdatathroughWMS.

Dataset HotSpotsMODIS

HotSpotsVIIRS BurnedAreaMODIS

BurnedAreaVIIRS

Layer Last24hours Last7days Last30days Last90days Fireseason

TheWMSstandardprotocolservesgeo-referencedmapimages.Thismeansthatthesearesimplygeo-referenced pictures and therefore they do not store the real measures that the I-REACTdownstreamserviceisinterestedin.WMSisprimarilyusedforvisualisationpurposes.However,theI-REACTprojectwantstofurtherusetheavailableEFFISwildfireinformationtointegrateitintothedifferentgeo-processingchainsandbusiness logic.Therefore,WMSisnotthestandardprotocolthatisusefulwhendownstreamservicesneedaccesstotheactualmeasuresrepresentedbytheimages.Tobetterunderstandthisissue,pleasereferbacktoError!Referencesourcenotfound.wherethecurrentsituationviewerisshowingthefiredangerforecastbasedontheweatherindex(FWI).ThedifferentcoloursofthevisualisationrepresentactualvaluesoftheFWI.However,whenaccessingthisdataviatheWMS,oneonlyknowsthatitisredorgreen.Onecannotknowtheactualpixelvaluethatcausedthatareatobecolouredred.Adifferentprotocolwouldneedtobeavailableinordertogetaccesstotherawdatathatisneededforfurtherprocessingandnotjustvisualisation.

Thereisapossibilitytoacquirethewanteddataofflinethroughadataaccessrequest.EOXPLOREtookadvantageofthisinordertobetterunderstandthedataavailablewhentheJRCwelcomedan10https://en.wikipedia.org/wiki/Web_Map_Service11http://effis.jrc.ec.europa.eu/applications/data-and-services/



DeliverableID:D3.1


employeeasavisitingscientist.TogetherwiththeEFFISteam,wewereabletogetaccessandseethetypeofdatathatEFFIShasintheirwildfiredatabases.Unfortunately,thisisnotaviablesolutionfordownstreamservicesbecauseofthelagtimetomakewanteddataavailable.

TheEFFISteamiscontinuingtoworkondevelopingtheirservices.ThereisatestwebsitewheretheEFFISAPIthatpotentiallywillbemadeavailableinthefutureishosted[RD10].Basedonthedetailsthat can be viewed through this test website, the I-REACT partners are very confident thateventuallyafullyautomatedconnectionthroughastandardAPIsuchasREST12willbepossiblewithEFFIS.Furthermore, therearemanynewdataandservices listed fromwhich I-REACTcouldalsobenefit.ThisisexactlyhowdownstreamservicesshouldaccessCopernicusservices.

3.6 COPERNICUSEFFISDATAPORTAL

Unfortunately, thisalsomeant that theprojectpartnerscouldnot ‘waitandsee’whetherEFFISwould implement the necessary API to access the wanted European wildfire data from theCopernicusEFFISportal.Consequently,analternativesolutionneededtobefoundinordertobesurethattheI-REACTsystemwasabletoingesttheneededEuropeanwildfireinformation.

The solution in this casewas on the one hand to use the I-REACT partners’ access toweatherforecastdataandprocessingexpertisetore-producetheEuropeanFireWeatherIndexandontheotherhandtofindanotherproviderofthefirehotspotdata.ThenexttwosectionsprovidethedetailsofhowthehotspotdataandtheFWIareprocessedandprovidedto I-REACT.ThesearespecificallyrelatedtotheI-REACTtasks3203to3206.

3.6.1 THEFIREWEATHERINDEXOneof theways thatEFFISmonitors forwildfirepotentialacrossEurope is tocompute theFireWeather Index (FWI). This is aweather forecastbased indexdevelopedby theCanadian ForestServiceandtestedbytheEFFISteamtobeoneofthebest.ItrequiresasinputaweatherforecastwhichisprimarilyprovidedbytheECMWFalthoughEFFISalsousesothernationalweatherforecastmodels.ForI-REACT,ourpartnerFMIhasaccesstotheEuropeanweatherforecastfromtheECMWFbutalsoproducesafinerspatialresolutionweatherforecastcalledtheGrandLimitedAreaModelEnsemblePredictionSystemorGLAMEPSforshort.BothoftheseforecastsareusedtocomputetheFWI.Another I-REACTpartner,Meteosim isproducing theFWIbasedon the two forecasts. Thedetailsofhowthis isdone isprovidedthroughreporting fromthe I-REACTWorkPackage4andthereforewillnotbeduplicatedhere.

TheFWI13essentiallyprovidesameasureofwildfiredangerbasedonweatherforecasts.

12https://en.wikipedia.org/wiki/Representational_state_transfer13http://cwfis.cfs.nrcan.gc.ca/background/summary/fwi



DeliverableID:D3.1


3.6.2 AFFECTEDCOMMUNITIESBASEDONTHEFWIThefirstI-REACTtasktoprocesstheFWIdatais3203todelineatethecommunitiesthathaveahighfire danger potential. This task is based on the computation of the FWI based on the ECMWFforecasts.ThisissimilartotheprimaryFWIproductproducedbyEFFISandavailablethroughtheirviewer.

AswiththeprevioustasksusingEFASinformation,theseprocessingchainsalsomakeuseofopensourceprojectsincluding:

• PostgreSQL/PostGIS–forgeo-spatialdatabaseaccess;

• Python–includingmanydifferentdataprocessinglibraries;

• AndGDAL–theGeospatialDataAbstractionLibrary14.

Furthermore,theseprocessingchainstakeadvantageoftheServiceBusmessagingserviceandtheIDI of the I-REACT system as it was designed and developed specifically for this project. Error!Referencesourcenotfound.showstheprocessingsteps.

Figure3-12:TheprocessingchainforI-REACTtask3203basedontheECMWFFWIdata.

AsshowninthedatalisteningsectionofError!Referencesourcenotfound.,thereisadatalistenerthat has been implemented in Python. Within a continuous loop, this listener is listening formessagesfromtheI-REACTsystem,i.e.theServiceBus,foranyupdatesregardingtheI-REACTtask

14http://www.gdal.org/



DeliverableID:D3.1


4302.Notethatthistaskbeginswiththenumberfourwhichmeansitisbasedonworkpackage4results.Specifically,theI-REACTtask4302istheECMWFbasedFWIdataproducedbytheprojectpartnerMeteosim in the NetCDF15 format as a substitute for the EFFISmissing data. The datalistenerfortask3203issubscribedtospecificallyreceivemessagesaboutthetask4302.

Themessaging service,as implemented in the I-REACTsystem, is thecurrent state-of-the-art toimplementingwebbasedservices.Thisallowsaserviceto immediatelyreactwhenamessage isreceived.Unliketheoldersolutionswherebyaservicewouldneedtoconstantlycheckwhetherafilewasupdatedremotely.

AsshowninthedatalisteningsectionofError!Referencesourcenotfound.,thereisadatalistenerthat has been implemented in Python. Within a continuous loop, this listener is listening formessagesfromtheI-REACTsystem,i.e.theServiceBus,foranyupdatesregardingtheI-REACTtask4302.Notethatthistaskbeginswiththenumberfourwhichmeansitisbasedonworkpackage4results.Specifically,theI-REACTtask4302istheECMWFbasedFWIdataproducedbytheprojectpartnerMeteosim in the NetCDF16 format as a substitute for the EFFISmissing data. The datalistenerfortask3203issubscribedtospecificallyreceivemessagesaboutthetask4302.

Themessaging service,as implemented in the I-REACTsystem, is thecurrent state-of-the-art toimplementingwebbasedservices.Thisallowsaserviceto immediatelyreactwhenamessage isreceived.Unliketheoldersolutionswherebyaservicewouldneedtoconstantlycheckwhetherafilewasupdatedremotely.

Whenamessageisreceiveditalertsthetask3203tothefactthatthereisnewFWIdataavailableforprocessing.ThisessentiallytriggerstheprocessingchainbasedontheECMWFFWIdata.ThefirststepistoquerytheI-REACTIDIforthespecificdataserviceUniformResourceLocator17(URL)forthetaskthatsentthemessage,i.e.4302inthisparticularcase.ThequeryreturnsalistofthedataURLscreated.Thislistismadeupofthemostcurrentdataplusalldatacreatedinthelasttwodays since.With this list in-hand, themodel chooses from the list the latest data available anddownloadsittothelocalprocessingdirectory.

Once the data has been downloaded, the data processing can begin. First, the data must beextractedfromthedownloadedNetCDFfile.Specifically,themodelneedsthemaximumFWIvaluesfor the currentdateaswell as thenext twodays. This is possiblebecause theFWI isbasedonweatherforecastdata.TheresultisthecreationofthreedifferentFWIdatasetsinGeoTIFFformat.Thenextstepistocomparetherasterdata(FWI)totheNUTSlevel5vectordata.BothtypesofgeospatialdataareintheprojectionEPSG:4326whichistheWorldGeodeticSystem(WGS)1984standard.

TheFWIrasterdataisprocessedusingthePythonrasterstatslibraryinordertoextractthepixelvaluesandthenreturnthestatisticaldataforeachNUTS5polygon.TherastervaluesincludetheFWImaximum,minimumand90thpercentilewhicharethenstoredwiththeNUTS5levelpolygon

15https://en.wikipedia.org/wiki/NetCDF16https://en.wikipedia.org/wiki/NetCDF17https://en.wikipedia.org/wiki/URL



DeliverableID:D3.1


IDinthelocalPostGISdatabase.APostGISviewthenmatchestheNUTSlevel5IDtotheIDofatablecontainingthepolygondetailsincludingthegeometry.TheresultfromthisviewisthencalledfromthePythonmodeltoloadintothespatialdatalibraryGeoPandas.

UsingGeoPandas,themodelcheckseachgeometryandtheFWImaxvalueandaddsstylerelatedcolumnsandvaluesforeachrow.ThisinformationisusedinthefinalstepsoftheprocesstoproducethefinalGeoJSONoutputfile.

ThenextstepintheprocesssavesthedataobjectasanESRIshapefilecontainingalltherequiredinformationbeforeconvertingthisshapefileintoaGeoJSON.ThisadditionalstepisrequiredtoallowtheprocessingmodeltoproduceastandardisedGeoJSONfile,includingtheadditionoftheright-handruleforspecifyingpolygongeometry.OncetheGeoJSONfileisready,themodelbuildstheassociatedIDImetadataobjectdescribingthethreegeneratedGeoJSONfiles.InordertosharetheGeoJSONfilescorrecly,themetadataincludestheFWIleadtimeaswellastheorderinwhichtheyshouldbevisualizedfordisplaybytheI-REACTclient.Theprocesscontinuesbyuploadingthetask3203resulttotheI-REACTserviceviatheIDI.Oncetheprocessiscomplete,themodelstoresalogoftheprocesstothelocaldatabase.

ThefollowingError!Referencesourcenotfound.presentsanexampleofthetask3203GeoJSONoutput.ThetablepresentssomeofthevaluesassociatedwiththeNUTSlevel5polygonsincludingtheFWIstatistics.

Figure3-13:TheI-REACTtask3203outputbasedonECMWFFWIdata.ThetablepresentssomeofthedetailsstoredintheGeoJSONpolygons.



DeliverableID:D3.1


3.6.3 AFFECTEDCOMMUNITIESBASEDONTHEFWI–NUTSLEVEL3Similar to the previous section, it is also possible to produce the FWI but based on largeradministrativeboundaries.Inthiscase,theadministrativeboundariesarebasedonNUTSlevel3.ThisisusefulincaseswheresimilarFWIarecoveringlargeareasandmoregeneralnationalstatisticsarerequired.

ThisprocessingchainwasgiventheI-REACTtasknumber3206andisalsobasedontheECMWFweatherforecastFWI.Duetothefactthatthemajorityoftheprocessingissimilartothatofthetask 3203, the details will not be repeated in this section. Error! Reference source not found.presents thesameprocessingchainwith theoutputof task3206presented inError!Referencesourcenotfound..NotethemuchlargerregionshavingthesameFWIoutput.ThelocationisofsouthernFranceandnorth-easternSpain.ThetableinsidethefigurepresentsthedifferentstatisticsthatcanbequeriedthroughtheGeoJSONfileandthereforearealsoavailablethroughtheI-REACTuserinterface.

Figure3-14:TheI-REACTtask3206outputbasedonECMWFFWIdata.ThetablepresentssomeofthedetailsstoredintheGeoJSONpolygons.

3.6.4 FIREHOTSPOTDETECTIONACROSSEUROPEThe FWI provides ameasure of thewildfire danger however, this only provides information toexpertsandlocalauthoritiesaboutthepotentialofthewildfireshouldafirestart.Thefirehotspotsare locations where elevated temperatures were identified through satellite based EarthObservation (EO) techniques. They are also called ‘active fire data’ indicating the fact that apotentiallyactivefirehasbeenidentifiedinthesatelliteimagery.

ActivefiredataiscurrentlyacquiredviatwodifferentEOsensors:theModerateResolutionImagingSpectroradiometer(MODIS)andtheVisibleInfraredImagingRadiometerSuite(VIIRS).Inthefuture,itishopedthattheCopernicusSentinel-3sensorwillbeaddedtothelist.Thehotspotsoractivefiresarebasedonthermalprocessingoftheimagerytoidentify‘hot’pixels.Firedetectionsfrom



DeliverableID:D3.1


thetwosensorscandifferslightlyduetothespectralbandsthattheyusetoidentifyhotobjects.ThereisalsoadifferenceinthespatialresolutionbecauseMODISprovidesoutputat1kmspatialresolutionwhereastheVIIRSsensordataisabletoprovidegreaterresponseoverfiresofrelativelysmallareasduetoitsspatialresolutionof375m.Furthermore,theactivefiredataaredesignedtobeprovidedinnearreal-time.Whatthismeansisthattheteamsprocessinganddeliveringattempttoprocessandmakeavailablethehotspot informationasquicklyaspossible. Ingeneral, this iswithinthreehoursofthesatelliteacquiringtheimagery.

BothtypesoffirehotspotdataareusedwithintheTask3.2.WhileitisnotclearfromwhichservicetheEFFISteamdownloadstheirfirehotspotdata,itisknownthattheyarefromthesamesatellites.Forthisreason,thetwofirehotspotdataareintegratedintotheI-REACTsystem.

3.6.5 AFFECTEDCOMMUNITIESBASEDONTHEVIIRSFIREHOTSPOTDATAThe I-REACT task 3204 involves the processing of VIIRS hot spot point data and again the dataprocessingisasmuchaspossiblebasedonopensourcetechnologies.Suchtechnologiesareusedbothingatheringthedataaswellasprocessingit.

Atthemoment,VIIRSactivefiredataforEuropeisdownloadedfromanFTPserver.Unfortunately,there isnomessagingservicethatalertsthetask3204tonewdataandthereforethetaskmustcheckregularlyfornewinput.However,thismaychangeinthefutureifamoresuitableserviceisfound.Thefollowingdiagram(Error!Referencesourcenotfound.)presentstheprocessingstepsoftask3204.

Figure3-15:TheprocessingstepsoftheI-REACTtask3204toprocessVIIRSbasedactivefiredatatodelineateaffectedcommunities.



DeliverableID:D3.1


OncethehotspotdatahasbeendownloadedfromtheFTPserver,whichoccursoncedaily,theprocessingcanbegin.Aswithotheroperationaldata,thereisnoguaranteethatitwillbeavailablewhenitisexpected.Therefore,asshownintheprocessingdiagram,itisnecessarytocheckfornewdataagainstlocallogstobesureunnecessaryprocessingisdone.

TheVIIRSEuropeandailydataisdeliveredintheCSV18fileformatintheEPSG:4326projection.Theavailabilityofthedatavariesfromdaytodayandsotheprocessneedstokeeptrackofthelastsuccessfulprocessandstartsthecurrentprocessfromthistimeplus1day(thiscouldbethemodelrundateminus1to2days).Thisprocessrunsasaloopandthereforeiftherewasanissueand/oragapintheavailabilityofdata,theprocesswillattempttocatchupinprocessingalltheavailabledatatothecurrentdate.

Toachievethis,thePythoncodequeriesalocaldatabasetableinPostgreSQLforthelastsuccessfulrundate.ThefirststepoftheprocessistoimportandloadthehotspotdataintoaPostGISspatialtable,convertingtheCSVdataintopointgeometries.APostGISviewisthenusedtoqueryaspatialtableoftheNUTSlevel5boundariesacrosstheEUtofindthosewhichcontainahotspotandthenumberofhotspotswithinthepolygon.TheresultfromthisviewisthencalledfromthePythonmodeltoloadintothespatialdatalibraryGeoPandas.

UsingGeoPandas,themodelcheckseachgeometryandcountsthenumberofvaluesindicatinghotspots.Thentheprocessaddsstylerelatedcolumnsandvaluesforeachrow.ThisistobeusedinthefinalpartoftheprocesswheretheGeoJSONisproduced.TheprocessthensavesthedataobjecttoanESRIshapefilecontainingalltherequireddatabeforeconvertingthisshapefileintoaGeoJSONfiletype.ThisadditionalstepisrequiredtoallowforthemodeltoproducethewantedGeoJSONoutputfileinthelateststandardincludingtherighthandruleforgeometry.

With the final GeoJSON output file ready, the model builds the required IDI metadata objectdescribingthegeneratedGeoJSONfilethatwillbedisplayedbytheI-REACTclient.ThecodethenusestheI-REACTIDIservicetouploadthefirehotspotinformationlayer.Oncecompletethemodelstoresa logof theprocess to the localdatabase.A sampleGeoJSONoutputof the task3204 ispresentedbelow(Error!Referencesourcenotfound.).

AsshownintheError!Referencesourcenotfound.,eachpolygonprovidesthenumberofhotspotpixelsidentifiedwithintheregionaswellasotherinformationaboutthepolygonsuchaslocationandID.

ThesameVIIRSactivefiredataisusedfortheI-REACTtask3205.Thistaskshowsthelocationsofthepixelsthatwereidentifiedasfirehotspots.Thetask3205involvesaccessingtheVIIRShotspotpointdataandconvertingittotherequiredGeoJSONoutputformattouploadtotheIDIandthendisplayedviatheuserinterface.Thisprocesscontinuestobebasedonanumberofopensourcetechnologiesthatareusedtogatherandthenprocessthedata.

18https://en.wikipedia.org/wiki/Comma-separated_values



DeliverableID:D3.1


Figure3-16:TheVIIRSactivefireproductusedtoproducedelineationsoftheNUTS3levelregionswheretheywereidentified.Thetableshowsthateachofthepolygonsprovidesthenumberofhotspotpixelsfoundwithinthatregion(count).

ThediagraminError!Referencesourcenotfound.describestheprocessingsteps.Thefirstpartoftheprocessfollowsthesamestepastask3204becauseit isbasedonthesameVIIRSactivefirepointdatathatisavailableinthePostGIStable.APostGISviewisthenusedtoqueryaspatialtableofNUTSlevel5boundariesacrosstheEUtofindthosehotspotswhicharewithinthefullextentofthecommuneboundaries.TheresultingpointdatafromthisviewisthencalledfromthePythonmodeltoloadintothespatialdatalibraryGeoPandas.



DeliverableID:D3.1


Figure3-17:TheprocessingstepstoproducetheI-REACTtask3205GeoJSONoutput.

UsingGeoPandas,themodeladdsstylerelatedcolumnsandvaluesforeachrow.ThisistobeusedinthefinalpartoftheprocesswheretheGeoJSONoutputfileisproduced.Thetask3205processthensavesthedataobjectasanESRIshapefilecontainingalltherequireddatabeforeconvertingthisshapefileintoaGeoJSONfiletype.TheremainderoftheprocessingisthesameasthatfortheI-REACTtask3204.

Figure3-18:TheGeoJSONoutputoftheI-REACTtask3205showingtheVIIRSactivefiredatapointswithintheNUTSlevelpolygons.

TheexampleoutputshowninError!Referencesourcenotfound.showstheresultofthetask3205.TheVIIRSactivefirelocationsaremappedontopoftheresultsfromtask3204providingabettersenseofwherethehotspotsarefoundwithinthecontextofthecommunepolygon.



DeliverableID:D3.1


3.7 DISCUSSION

TheTask3.2–EuropeanEarlyWarningSystemsIntegrationwasfirstandforemosttodependonthe Copernicus EarlyWarning Services, EFFIS and EFAS for data and information on flood andwildfiremonitoringtobeintegratedaspartoftheI-REACTdownstreamservices.Unfortunately,upto the time of writing of this report, those services are not available and/or able to provideoperationally data for downstream services as expected. However, even though this particularprojectTask3.2wasunabletodirectlyusetheEWSservicesdirectly,thenecessaryandproposed‘downstream’ services based on the expected Copernicus EWS have been implemented andintegrated into the I-REACT system.Thiswasachieved through close collaborationwith the JRCincludingthehostingofaresearchscientistfromEOXPLOREforthreemonths.Furthermore,thiscouldnotbeachievedwiththehelpofotherI-REACTpartnersthathaveaccesstotherequiredinputdata (FMI=weather forecast)and theexpertise toproduce thewildfiremonitoring information(Meteosim=FWI).

TheresultsachievedhavelaidthegroundworkforotherpossibleCopernicusservicesthatmayinthefutureprovidewantedinformationfordownstreamdisasterservicesbutalsoisreadywhenthewantedCopernicusEWSdatabecomesavailable.ConnectingtouseEFFISandEFFASdata inthefutureshouldbequitesimple.

Animportantpointtorememberwithrespecttothetask3.2isthefactthatitispossibletodeliverstandard GeoJSON disaster related information from a variety of sources. This is an importantprojectachievementbecauseitdemonstratesthestrengthsbehindthetechnologicaldesignofI-REACT.Eventhoughdataiscomingfromdifferentplacesandindifferentformats,theprocessingtaskscandealproducethewantedstandarddataproductbasedonopensourcetechnologiestobeingestedintoI-REACTandprovidefireandflooddisasterinsights.



DeliverableID:D3.1


4 EXISTINGLOCALEMSINTEGRATION

4.1 OVERVIEWOFLOCALEMS

The integration of local Emergency Information Systems in the I-REACT framework focuses onobtaininginformationconcerningalertsandmeasurementsfrommonitoringsystems.Theriskswearefocusingon,forthisactivity,areflood,landslide,extremeweathereventsandfires;actually,forthesehazardswecandefinemoreeasilyaclearflowofinformationfromforecasttoemergencymanagement.

Alerts: Inthiscase,wemainlyextractandintegratethecontentofbulletinsthatpredictfiresand/ormeteorologicalandhydrogeologicalphenomena.

MonitoringSystems: Inthiscase,weextractandintegratebothrawmeasuresfromsensors,suchaswaterandraingauges,aswellasinformationaboutmeasuresexceedingwarningthresholds

To achieve this, we have designed and developed the LEMS (Local Emergency ManagementSystems),aspecificintegrationmodule,whichprovides:

§ connectiontoexternaldatasources;

§ appropriateprocessingofdataformat,fromatechnicalpointofview;

§ simplelogicalrules,usefultounderstandthecontent;

§ standardizationandtransfertotheI-REACTOR

AftertheiringestionintheI-REACTOR,thesedatabecomeprecious‘triggerevents’fortheI-REACTinfrastructure and can activate additional actions, on the basis of automatic flows, defined inadvanceandsuggestedbytheDecisionSupportSystemModule(DSS),ormanuallyscheduledbyemergencymanagers.

For example, the arrival of information about exceeding a warning threshold, in a hydrometermeasure,mayrequirea localmonitoringcampaign in theaffectedareabyUAVor technicians/expertswhocanreportthroughmobileapplications.

A floodalert codeona largearea, instead,may requireawidedisseminationof information toaffectedpopulationsand/ortheactivationofspecificinteractionswithsocialnetworks.

In this first phase we have selected and connected sources from countries where projectdemonstrations areplanned:UK, Finland, Italy, Spain and SavaRiverBasin.However, the LEMSmodulehasbeendesignedanddevelopedtoallow,inthefuture,furtherintegrationsofsimilardatasourcesfromotherEuropeancountries(Table4-1).



DeliverableID:D3.1


Thefollowingtablesummarizesinformationabouttheconnectedsources.

Table4-1:Connectedsources

Country Owner MainHazards Content Originalformat

Finland FMI Extremeweather/Flood/Fires

Warningbullettin Xmlcap

Italy ARPAPiedmont

Extremeweather/Flood

Meteo/idrogeologicalbullettin

Xmlcap

Italy ARPAPiedmont

Extremeweather Thunderstormbullettin Xmlcap

Italy ARPAPiedmont

Flood Measures from sensorsexceeding warningthreshold

Xmlcap

Italy AIPO Flood Raw measures fromsensors

Dbf

Spain CatalunyaCivilProtection

Extremeweather/Flood/Fires

Warningbullettin Webservice

UnitedKingdom

MetOffice Extremeweather/Flood

Meteo/idrogeologicalbullettin

Webservice

UnitedKingdom

EnvironmentalAgency

Flood LocalFloodwarnings Webservice

UnitedKingdom

EnvironmentalAgency

Flood Raw measures fromsensors

Webservice

Bosnia, Croatia,Montenegro,Slovenia,Serbia

Sava RiverBasinCommission

Flood Meteo/idrogeologicalbullettin

Dataset

NoteonXmlCapStandard

If alreadyavailable,wepreferred to integrateexternal sourcesexposed inXmlCap format. TheCommonAlertingProtocol(CAP) isastandardprotocolusedtofacilitateemergencyinformationsharinganddataexchangeacrosslocalandnationalorganizationsthatprovideemergencyresponseandmanagementservices[RD11].Itisareliablecandidatetobecomeaworldstandardandmanyorganizations have already adopted it. Its wide diffusionwill reduce the disadvantages of non-homogeneousinformationformats.



DeliverableID:D3.1


4.2 SURVEYOFEXISTINGEMERGENCYMANAGEMENTSYSTEMS

Asmentioned,theanalysisoflocalEMShasbeenfocusedoncountrieswherepilottestsofI-REACTsystemwill be conducted. The comparison of those various systems has led to generalize dataintegrationservices.

4.2.1 FINLANDTheFinnishMeteorologicalInstitute(FMI)isaresearchandserviceagencyundertheMinistryofTransport andCommunications. Themainobjectiveof theFinnishMeteorological Institute is toprovide theFinnishnationwith thebestpossible informationabout theatmosphereaboveandaroundFinland, forensuringpublic safety relating toatmospheric andairbornehazardsand forsatisfyingrequirementsforspecializedmeteorologicalproducts[RD12].

TheFinnishMeteorologicalInstitutemonitorstheweatherinFinlandandelsewhereintheworldround the clock. Depending on the situation, it issues warnings on dangerous or hazardousphenomenainFinland.

TheLEMSModule integratesanXmlCapstreampublishedbytheFMIthat includes informationcontainedintheirweather-warningbulletin(Figure4.1).

Figure4-1:WarningsissuedbytheFinnishMeteorologicalInstitute

Theseverityofthewarninglevelisshownonthemapbymeansofathree-colourcodesystem.

§ green-nomajordanger



DeliverableID:D3.1


§ yellow-dangerousweathermayoccur.Youareadvisedtotaketheweatherconditionsintoaccountwhenyouareexposedtoweather.Youshouldalsokeepaneyeontheweatherandavoidrisks.

§ orange - dangerousweather. Theweathermaycause injuriesandmaterialdamage.Youshouldavoidrisksthatmaybecausedbyweather.Youareadvisedtokeepaneyeontheweatheronaregularbasisandfollowtheinstructionsissuedbytheauthorities.

§ red-verydangerousweather.Injuriesandmaterialdamagecanbeexpectedoverawidearea.Youshouldkeepaconstanteyeontheweatherandtheawarenesslevel.Youshouldalso follow the instructions issued by the authorities and be prepared for exceptionalmeasures.Theredcolourappearsonthewarningmapveryrarely.

Warningsareissuedseparatelyforeachdayoftheweek.Forexample,onMonday,warningswillbeissuedforMonday,Tuesday,Wednesday,ThursdayandFriday.Theywillindicatetheworstsituationduringthatday,suchasthestrongestwind.Thewarningthresholdishigherinwarningsfor2–5days,andallwarningsarenotgivenforperiodslongerthan24hours.

TheFinnishMeteorologicalInstituteisaninformationproviderforMeteoalarm.eu19,thewebsiteinformingpublicandauthoritiesaboutsevereweatherconditionsin36Europeancountries.

WeatherparametersincludedinMeteoalarmdifferfromcountrytocountry.FinnishMeteorologicalInstitute(FMI)deliverssevereweatherinformationtoMeteoalarmaboutfollowingparameters:

§ Wind

§ Rain

§ Thunderstorms

§ Extremehightemperature(Heatwave)

§ Extremelowtemperature(Coldweather)

§ Snow/ice(Roadweather)

§ Forestandgrassfires

4.2.2 ITALYItaly

EmergencyManagementsSystemsinItalyareco-ordinatedbytheDepartmentofCivilProtectionwiththeFunctionalCentresNetwork,havingaCentreineachRegionandautonomousProvince.InthisdocumentwearefocusingontheexistingEMSintwoareas,PiemonteregionandtheareaofthePoriverstretchnearFerrara:thesetwoareaswillhosttheItalianpilotsitesforI-REACT.

Piemonte:Amongotheractivities,ARPA(theRegionalAgencyforEnvironmentalProtection)istheFunctionalCentreinPiemonteandoperatesahydro-meteorologicalmonitoringsystemthatuses

19http://meteoalarm.eu/



DeliverableID:D3.1


automatic tools to determine soil and atmospheric conditions. The interpretation of the dataacquired and the results of data processing by modelling make it possible to carry out thecontinuoussurveillanceoftheenvironmentandtheterritory,tomanageprediction,alertingandmonitoringsystemsforsituationsrequiringspecialattention,andtoupdateonacontinuousbasistheclimatologicaldocumentationforuseinplanningandprogrammingactivities.

TheLEMSModuleintegratesthreeexperimentalXmlCapstreamspublishedbyARPA:

1. Meteo/Hydrogeologicalwarningbulletin(Figure4-2).

TheAgencyissuesahydrogeologicalwarningbulletineveryday,butithasavalidityof36hours.

Figure4-2:Meteo/HydrogeologicalwarningbulletinissuedbyARPAPiemonte

Thebulletinprovides informationon11homogeneousareas. In the first section, foreacharea,criticalsituationsarehighlightedforweatherevents(avvisimeteo).Inthesecondsection,criticalsituationsrelatedtotheeffectsofweathereventsarehighlighted,inthesameareas.Inthiscase,thecriticallevelisexpressedbyfourstandardcolourcodes.

2. Thunderstormbulletin



DeliverableID:D3.1


This service, through meteoradar observations, generates alerts for the imminent arrival of athunderstorm and displays nowcast data on the position and amplitude of thunderstorms inprogress(Figure4-3).

Figure4-3:LivestormserviceissuedbyARPAPiemonte

3. Measuresfromsensorsexceedingwarningthreshold(Figure4-4).

Theserviceprovides,foreachofthewaterrainsensorsmanagedbyARPA,awarningconcerningtheovercomingofawarningthreshold.Inthiscase,besidestheindicationofthesensorinvolved,andthelevelofcriticality,thesurroundingareapotentiallyaffectedishighlighted.

Figure4-4:WarningsonsensorsissuedbyARPAPiemonte.



DeliverableID:D3.1


Poriverstretch(Figure4-5and4-6):thePoRiverInterregionalAgency(AIPO)acquiresmeasuresfromalargenetworkofwaterandrainlevelsensorsmanageddirectlybyitself,orbythe4regionsthebasinismadeupof:Piemonte,Lombardia,VenetoandEmiliaRomagna.InI-REACT,atthisstage,weintegrateroughmeasurementsfromsensors.Futuredevelopmentcouldincludetheintegrationofthelogicrequiredtocomparemeasureswithalertthresholdvalues,asinthecaseoftheARPAPiemonteservice.

Figure4-5:Monitoringhydrostations–RiverPo

Figure4-6:PoriverstretchbetweenFerraraandPanaro



DeliverableID:D3.1


4.2.3 SPAINForSpain(Catalunya)theLEMSModuleintegratesawarningbulletinpublishedtheMeteorologicalServiceofCatalunya(http://www.meteo.cat/wpweb/divulgacio/la-prediccio-meteorologica/avisos-smp/).

Thebulletinisissuedeachtimethereisthepossibilityofreachingaspecificthreshold(Table4-2).Thethresholdchangesdependingontheweatherevent(idmeteor).

Table4-2:Catalunya

Meteor Lowthreshold Thresholdhigh

Intensityofrain

Intensity>20mm/30minutes Intensity>40mm/30minutes

Accumulationofrain

Accumulated>100mm/24hours Accumulated>200mm/24hours

Snowaccumulatedin24hours

thickness≥0cm,altitudelessthan300meters thickness > 5 cm in height less than 300meters

thickness>2cminaltitudeover300metresupto600metres

thickness>15cminaltitudeover300metresupto600metres

thickness>5cminaltitudehigherthan600metresto800metres

thickness>20cminaltitudehigherthan600metresto800metres

thickness>10cminaltitudehigherthan800metresupto1000metres

thickness>30cminaltitudehigherthan800metresupto1000metres

thickness>20cminheightsabove1000metersupto1500meters

thickness > 50 cm in heights above 1000metersupto1500meters

Wind(Map)

Highest streak > 20 m/s to: Anoia, Alt Penedès,Bages, Baix Llobregat, Baix Penedès Barcelonès,Garraf, Gironès, Maresme, el Moianès, Vic, Plad'urgell, Segarra, Segrià, Selva, Tarragona, Urgell,VallèsOccidentalandVallèsOriental

Highest streak > 30 m/s to: Anoia, AltPenedès,Bages,BaixLlobregat,BaixPenedèsBarcelonès, Garraf, Gironès, Maresme, elMoianès, Vic, Pla d'urgell, Segarra, Segrià,Selva, Tarragona, Urgell, Vallès OccidentalandVallèsOriental

Higheststreak>25m/sin:AltCamp,AltUrgell,AltaRibagorça, Baix Camp, Baix Empordà, Berguedà,CerdanyaConcadeBarberà,Garrigues,laGarrotxa,Noguera,PallarsJussà,PallarsSobirà,Pladel'estany,Priorat,Riberad'ebre,elRipollès,El,TerraAltaandVald'aran

Higheststreak>35m/storestin:AltCamp,Alt Urgell, Alta Ribagorça, Baix Camp, BaixEmpordà, Berguedà, Cerdanya Conca deBarberà, Garrigues, la Garrotxa, Noguera,Pallars Jussà, Pallars Sobirà, Pla de l'estany,Priorat, Ribera d'ebre, el Ripollès, El, TerraAltaandVald'aran

Higheststreak>30m/sAltEmpordà,BaixEbreandMontsià

Highest streak > 40 m/s region of AltEmpordà,BaixEbreandMontsià

Stateofthesea

Waves>2.50metres(heavysea) Waves>4.00metres(marbrava)

ColdMinimumtemperatureextreme:temperaturelowerthanthepercentile2oftheminimumtemperaturedaily

Wave of cold: temperature below thepercentile 2 of the minimum temperaturedailyforthreeconsecutivedaysormore

HeatMaximum temperature extreme: a temperaturehigher than the percentile 98 of the temperaturemaximumdaily

Heatwave:temperatureabovethepercentile98 maximum temperature daily for threeconsecutivedaysormore



DeliverableID:D3.1


Thebulletin can contain 7 different possibilitiesweather event, that are: intensitat de pluja ( =meansrainfall),acumulaciódepluja(=accumulation),neu(=Snow),vent(=wind),estatdelamar(=stormsurges),fred(=coldwave)andcalor(=heatwave).Thesecondandthethirdcolumninthetablerepresentthetwothresholds,Llindarbaix=meansLowthresholdandLlindaralt=meansHighthreshold. When the meteorological service sends a bulletin, it is updated twice a day (called“evolucions”)untiltheendoftheepisode.

4.2.4 UNITEDKINGDOMFor UK the LEMSModule integrates three different web services, provided byMet Office andEnvironmentAgency:

Meteowarningbulletin(Figure4-7).

TheMetOfficeprovideswarnings forEnglandandWales,concerninghazardousweathereventswhich have the potential to cause damage, widespread disruption and/or danger to life(https://www.metoffice.gov.uk/public/weather/warnings).

Thisserviceincludeswarningson5daysaboutrain,snow,windfogandicerepresentedbyacolourdependingona combinationof both the likelihoodof theeventhappening and the impact theconditionsmayhave.

Thebasicmessageassociatedwitheachwarninglevelis:

High

• Youmayneedtotakeactionasweareexpecting...

• Therewillbe...

Medium• Weshouldbepreparedfor...• Thereislikelytobe...

Low• Beawareofthepotential/possibility...• Thereisthesmallchanceof...

VeryLow • Beawarethatthereisaverysmallriskof...



DeliverableID:D3.1


Figure4-7:MeteowarningbulletinissuedbyMetOffice

Floodwarnings(Figure4-8).

TheEnvironmentAgencyprovideslocalfloodriskwarningsandpublishesdetailedinformationonthepotentiallyaffectedareas(https://flood-warning-information.service.gov.uk/warnings).

Figure4-8:FloodriskwarningsissuedbyEnvironmentAgency.



DeliverableID:D3.1


Rawmeasuresfromsensors

Another service, publishedby EnvironmentAgency, provides data from themonitoring stationscheckingregularlytheriverandsealevels,helpingtounderstandthecurrentandfuturefloodrisk(https://flood-warning-information.service.gov.uk/river-and-sea-levels)

4.3 DEFINITIONANDDEVELOPMENTOFINTEGRATIONSERVICES

The LEMSmodule has been designed to extract information coming from the local EmergencyManagementSystems,andtointegrateitintoI-REACT.

Every local service is sending its own set of information, therefore a first effort was aimed atextractingcommonsetofcoredata,basicallywhat,whenandwhere.

Forthebulletinsandalerts:

- informationonthehazardtypeandseverity- informationonthetemporalextent(dateofissue,dateofexpiry)- informationonthelocation(polygonrepresentingtheaffectedarea)

Forthesensorsdata:

- informationonthemeasure:whatkindofsensor,valuemeasured- temporalinformation:dateandtimeofthemeasurement- informationonthelocation(coordinatesofthemeasuringstation)

TheintegrationschemabelowsummarizestheinteractionsbetweenthemodulesoftheI-REACTsystem(Figure4-9).

Figure4-9:LEMSintegrationmoduleandtheI-REACTsystem.



DeliverableID:D3.1


Forbulletinsandalerts,theLEMSmodulesendsthedatatothe I-REACTData Interfacemodule,followingastandardI-REACTtemplate.

TheLEMSmodule stores themetadataare stored in theData layer,andproducesoneormoreGeoJSONfilethatwillbeavailableviatheI-REACTfront-endormobileapp.Forthistaskadedicatedsetofdatabase tables isusedby theLEMStostoreboth retrieved information fromthesourceproviders,andbackgroundinformation(forexamplethegeographicaldatathatarenecessarytocreatetheGeoJSONfiles,Figure4-10)).

Figure4-10:LEMSdatabase

Table4-3:I-REACTDataInterfaceattributes

Requiredattributes Optionalattributes

identification_resourcetitle layerattributes

identification_resourceabstract identification_coupledresource

identification_resourcetype temporalreference_dateofcreation

identification_resourcelanguage qualityandvalidity_lineage

classification_topiccategory qualityandvalidity_spatialresolution_latitude

classification_spatialdataservicetype qualityandvalidity_spatialresolution_longitude

keyword_keywordvalue qualityandvalidity_spatialresolution_scale

keyword_originatingcontrolledvocabulary constraints_conditionsforaccessanduse

temporalreference_start constraints_limitationsonpublicaccess



DeliverableID:D3.1


Requiredattributes Optionalattributes

temporalreference_end metadataonmetadata_date

temporalreference_dateofpublication metadatafileuri

temporalreference_dateoflastrevision sourcenames

qualityandvalidity_spatialresolution_measureunit creationtime

conformity_specification lastmodificationtime

conformity_degree isdeleted

metadataonmetadata_language deletiontime

coordinatesystemreference_code geographicboundingboxes

coordinatesystemreference_codespace

ireacttask

acquisitiondate

Table4-4:GeoJSONattributes

Requiredattributes

service

hazardCode

hazardGLIDECode

hazardName

hazardLevel

hazardLevelDescription

areaName

dateStart

dateEnd

dateLastRevision

creationDate

organizationName

country

AnexampleforthehydrogeologicalbulletinissuedbyARPAissketchedinthefigurebelow.



DeliverableID:D3.1


Figure4-11:ARPAHydrogeologicalbulletininI-REACT.

Alertbulletinsareoftenissuedwithdifferentleadtimes.Forexample,theUKFloodRiskbulletincontains5-dayfloodriskforEnglandandWales.TheLEMSmoduleprocessesthisinformationdaily,producingonefilecontainingmetadatainformationandfivedifferentGeoJSONfiles,oneforeachdayofforecast,thatcanbeservedasmaps,etc.



DeliverableID:D3.1


Insomecases,forexamplewhenafloodalertisissuedforalocalizedarea,anditmightbeusefultonotifyfirstrespondersandcitizensactiveinthatarea,theLEMSmodulecansendthedatatotheI-REACTbackendmodule,andthedatawillbeusedintheI-REACTsystemtocreateareport.

An example is the UK service providing flood warnings for England (https://flood-warning-information.service.gov.uk/warnings).Thedataisstructuredasexplainedinthetablebelow,andtheintegrationtaskwilltakecareofmappingtheinputdatatothisstructure.

Table4.5:DatastructureofUKserviceprovidingfloodwarnings.

Requiredattributes

type

start

end

areaOfInterest

location

ruleId

level

hazard

receivers

description

targetAreaOfInterest

targetLocation

sourceOfInformation

communicationStatus



DeliverableID:D3.1


5 SENTINELDATAPROCESSINGINTEGRATION

TheCopernicusprogramisEurope'sEarthObservation(EO)programofferingaseriesofthematicservices.Copernicus'ownSentinelsatellitesprovideuniqueoperationalsensingcapabilitiesacrossthewholemeasurement spectrum. Thanks to their advanced sensing concepts andoutstandingspatio-temporalsamplingcharacteristics,theSentinelsatelliteswillcollectmoredatathananyEOprogrambefore.ThefirstoftheSentinelsatelliteseries,Sentinel-1(S-1)waslaunchedon3rdApril2014.TheS-1keyapplicationsoverlandincludesthemonitoringoftopographicmovements(landsubsidence,glacierflow,etc.)andhydrologicprocesses(floodmapping,soilmoisture,waterbodies,etc.). This section describes the utilization of space-borne SAR data for historical frequencycalculation and development of near real-time flood mapping service using Envisat AdvancedSyntheticApertureRadar(ASAR)andSentinel-1satelliteremotesensingdata.

5.1 SATELLITEDATAPROCUREMENT

Thedataacquiredbythesatellitesaredownlinkedtothecollaborativegroundsegment,fromwhereviaESAnetwork it isbeingdistributedtoothersources likeScientificHubandESAServerof the“ZentralanstaltfürMeteorologieundGeodynamik”(ZAMG).FromZAMG,dataispushedtonationalmirrorandthentotheEarthObservationDataCentre(EODC)datastorage.EODChasthecompletearchive of the Envisat ASAR mission, covering the timespan from 2005 to 2012, and acquiresregularly the Sentinel-1 (S-1) data. The S-1 data at the EODCwarehouse are currently availableapproximately2.5hoursaftertheinitialsignalprocessingbyESA(level-1product)and6.25hoursaftertheacquisition.TheS-1level-1dataarearchivedonfastdisksstorageandbackedupusingarobotictapelibraryonaregularbasis.Figure5-1-1showsthedataacquisitionchainfromsatellitetoEODCstorage.

Figure5-1:EODCdataacquisitionstrategy.SatellitedataaretransferredfromthesatellitetoCollaborativeGroundSegment,thentoarollingarchivesystematZAMGandstoredtoEODCinternalstoragesystem.



DeliverableID:D3.1


5.1.1 DATAANDSENSORSPECIFICATIONSThe first of the six Sentinel satellite series, Sentinel-1Awas launchedon3rdApril 2014. S-1 is aSyntheticApertureRadar(SAR)missionforoceanandlandmonitoringandrepresentsthecontinuitymissiontotheSARinstrumentsflownonboardofEuropeanRemoteSensing(ERS)satellitesandENVISAT.TheS-1missionisimplementedthroughaconstellationoftwosatellites(units1Aand1B).TheSentinel-1Bwaslaunchedon25thApril2016.Inthefollowingashortoverviewaboutthemainapplications,datapolicies,accessandsatellitespecificationsisgiven.(a)Applicationdomains:TheS-1keyapplicationoverlandincludesthemonitoringoftopographicmovements(surfacesubsidence,glacierflow,etc.)andhydrologicprocesses(floodmapping,soilmoisture,waterbodies, etc.). Furthermore, S-1 canplay an important role in sustainable forestmanagementwithclear-cutandpartial-cutdetection,foresttypeclassification,biomassestimation,disturbancedetectionandprecisionfarming.

(b)Data source:Copernicus is a European system formonitoring the Earth,which consists of acomplexsetofsystemscollectingdatafrommultiplesources:EOsatellitesandin-situsensors,suchasgroundstations,airborneandsea-bornesensors.S-1dataarealsodistributedbytheCopernicusprogramme.

(c) Data policy and access: The free, full and open data policy adopted for the CopernicusprogrammeforeseesaccessavailabletoallusersfortheSentineldataproducts.Registrationisopento all users via simple on-line self-registration accessible via the Sentinels Scientific Data Hub(https://scihub.copernicus.eu/). Following registration, the user can immediately downloadSentinelproductsgeneratedsystematicallyfromallacquireddata.(Note:MemberStatesrequiringdatafornationalinitiativesintheframeoftheSentinelsCollaborativeGroundSegmentneednotregisteronthisservice;theyareservedviathededicatedaccesspoint.)

OriginalS-1data,distributedbyCopernicusprogramme,isfree.Theprocesseddata/parameters,(SAR backscatter, temporal/monthly/seasonal composites) and other operational products andservices are distributed by Earth Observation Data Center (EODC), for more details visithttps://www.eodc.eu/.

(d) Sensor and data specifications: Table 5.1 gives the overview of Sentinel-1 system and itsspecifications,andtheTable5.2givesthedetailedexplanationofSentinel-1dataacquisition.

Table5.1:Sentinel1sensorspecifications.

Satellites/Sensor AdinterimOperator Lifetime Orbit Band Revisittime Datadelivery

Sentinel-1/A-B ESA 7years Sun-synchronous

C-band5.405GHz(wavelength

18cm)

12days(withthe

constellationof1Aand1B~6days)

Withitscontinuousandconflict-freeoperations,Sentinel-1willprovideahighlevelofservicereliabilitywithnearreal-timedeliveryofdatawithinanhourafterreceptionbythegroundstation,andwithdatadeliveryfromarchivewithin24hours.



DeliverableID:D3.1


Table5-2:Sentinel1:specificationsofdifferentdataacquisitionmodes.

5.1.2 DATASTORAGESYSTEMAttheEODCplatform,theS-1dataarestoredonbothdisksforfastdataaccess,andtapesforlong-term storage and backup. Currently, two Petabyte (PB) of disk and four PB of tape space areavailableandmore than2million individualSentineldata filesarearchivedon this system.Thestoragesystemiscurrentlygoingtobeexpandedonregularbasisaccordingtothepredefinedplans.

EODC'shigh-speeddiskstoragesystemisbasedonIBMElasticStorageServer(ESS).IBMGeneralParallelFileSystem(GPFS)clusteredfilesystemtechnologyisimplementedtoensuretheefficiencyandperformanceinfilestorage.Currently,theEODChigh-speeddiskstoragesystemiscapabletohosttwoPBofdataforhigh-speedprocessing.Alternatively,EODCuserscanalsouseanotherhigh-speed storage system which is operated by Vienna Scientific Cluster 3 (VSC-3), called VSC-3distributedvolume.ThisfilesystemiscurrentlyrunontheparallelclusteredfilesystemBeeGFS,whichconsistsof360spinningdisksconnectedthrough160Gb/secbandwidth[RD14].

Tosavethefast-accessspaceondisks,arollingarchivingplanissetuptocontinuouslystorethedatasetstotapesthatarerarelyusedbyEODCpartners.Afterarchiving,thefolderstructureshownonfront-endterminalwouldbethesame,butthephysicallocationofthedataisontapestorageinsteadofhigh-speeddisks.Thisstrategyhelpstosavethediskstorageforhigh-speeddataaccessneedswithoutlosingtheaccessibilityofarchiveddata.Furthermore,thedata,whicharephysicallyarchivedontapescanbere-calledtohigh-speeddisksinashortperiodoftime.

EODChas implementeda comprehensivebackupplan to continuouslybackup thedataon theirstoragesystem.Thisisaninfiniteincrementalrollingprocessedwhichprovidestheabilitytoensurethedataaccessanddataconsistencyinthecasesofdisasters(virusattacks,hardwareandsoftwarefailures).

Operationalmodes Polarization Spatialresolution(m) Swathwidth

StripMapMode

VV+VHorHH+HV

5x5 80km

InterferometricWideSwathMode 5x20 250km

Extra-wideSwathMode 25x100 400km

WaveMode VVorHH 5x20 20kmx20km

DataProcessing:

Level-0Compressedandunprocessedinstrumentsourcepackets

Level-1GroundRangeDetectedGeo-referencedProducts(forI-REACTprojectLevel-1datawillbeused)

Level-2Oceanproducts



DeliverableID:D3.1


5.1.3 DATACOVERAGEANDAVAILABILITYThefollowingSentinel-1CoverageMaps(seeFigure5-2and5-3)showthecurrentstatusofS-1(A&B)interferometricwideswath(IWGRD)dataasstoredontheEODCdatawarehouse.TheyareprovidedinV(ertical)/V,H(orizontal)/H,V/H,andH/Vpolarisation,aswellasinacombinedmap(i.e.compositeofV/V+H/Hpolarisations)andaredynamicallyre-createdfromourdatabaseeveryweek.ForfloodmappingSentinel-1IWGRDLevel-1productisbeingused.

Figure5-2:Sentinel-1AdatacoverageforIWGRDacquisitionmode.

Figure5-3:Sentinel-1AdatacoverageforIWGRDacquisitionmode.



DeliverableID:D3.1


5.1.4 HISTORICALREMOTESENSINGDATAEODC data storage archives the complete ENVISAT ASAR mission, covering the lifespan of themissionfrom2005to2012.Figure5-4showsthecoveragemapforWideSwathacquisitionmode.

Figure5-4:ENVISATASARcoveragemapforWideSwathmode

5.1.5 EODCPROCESSINGENVIRONMENTANDPLATFORMThe flood mapping algorithm is developed and implemented by TUWien running on a virtualmachineatScienceIntegrationanddevelopmentPlatform(SIDP)hostedbyEODC.Theprocessingchainincludespre-processingoftheSARdata,dataqualitycontrols,floodmapping,andnecessarypost-processingsteps.TheSIDPisalsoconnectedtotheViennaScientificClusterwithmorethan2000 computing nodes, which might be used for heavy processing tasks that need parallelprocessing.Thefinalproductswillbefloodandfloodfrequencymaps.

TheEODCisacollaborationbetweenpublic-privatescientificorganizations,researchcentresandcloud providers. Themainmission of EODC (or EODC platform, EODC cloud environment) is toprovideaccessandrequiredcomponentsforbigdataprocessingandanalysis.KeycomponentsofEODCinfrastructure,whichwentoperationalinspring2015,are:

• Acloud-basedvirtualresearchanddevelopmentenvironment.

• TheaccesstotheViennaScientificCluster3(VSC-3).

• Acomputationclusterformid-termdataprocessing.

• APetabyte-scaledatastorageandarchive.



DeliverableID:D3.1


• Anearreal-timeOperationsandRollingArchive(NORA),whichisahigh-availabilitystorageandprocessingclustertocomplementthedatasetsinnearrealtime.

Figure5-5providesanoverviewofEODCkeycomponents,whichareinoperationatthemoment(September2017).ThedetailsofeachcomponentandtheViennaScientificClusterarepresentedinthenextsections.

Figure5-5:EODCinfrastructurecomponents.

5.1.6 EODCDATASTORAGEANDSENTINEL-1DATAFLOWRawdatafromtheEODCdatawarehouse(Level-1product)arepre-processedtocalculateSyntheticAperture Radar (SAR) signal backscatter with different spatial resampling (resolution) in Equi7projection[moredetailsaboutEqui7projectioncanbefoundinRD15].Figure5-6showsthedetailedstepsinvolvedintheENVISATASAR/Sentinel-1datapre-processingchain,whichisimplementedintheSARGeophysicalParametersRetrievalToolbox(SGRT).



DeliverableID:D3.1


Figure5-6:ENVISATASAR/Sentinel-1datapre-processingworkflow.

5.2 SENTINEL-1WATERMAPPINGALGORITHMDESIGN

Afterthepre-processingstep,parametersretrievalmodules/workflowofSGRTisinvokedinordertocalculatedifferentparametersi.e.,slope,meanbackscatter.Parametersarethennormalizedtoreference incidenceangles fordataconsistency.After theparameterscalculation, thefinalstep,productgenerationworkflowofSGRTislaunched.Inthisstepautomaticthresholddetectionandnoiseremovalmodulesareusedforthegenerationoffloodandfloodfrequencymaps.Figure5-7showsthesystematicworkflowdesignforfloodandfloodfrequencymapgeneration.

Figure5-7:ENVISATASAR/Sentinel-1schemeforproductiongeneration.

ThegraphicaloverviewoftheSARdatapre-processingandproductgenerationschemeisshowninFigure5-8.



DeliverableID:D3.1


Figure5-8:AnexampleoverviewoftheSyntheticApertureRadar(SAR)datapre-processingandproductgeneration(NeusiedlLake:Austrian–Hungarianborder).

5.2.1 USERINTERFACEANDFLOODMONITORINGACTIVATIONMETHODService on request, this service is established to provide a rapid post disaster flood status andinundationextentmaps,whichcanbeusedforpost-disastermanagementandreliefactivities.Inthis service the user/customerwould contact TUVienna through I-REACT coordination (e.g. viaAzure Service Bus) to trigger the product generation. The requested product can be delivereddirectlytotheIDI(I-REACTDataInterface).Inthesectionbelowadetaileddescriptionofnear-realtimefloodmappingserviceisexplained.

5.2.2 SENTINEL-1 BASED NEAR-REAL TIME FLOOD MAPPING SERVICE LOGIC

IMPLEMENTATION

5.2.2.1DEVELOPMENTTOOLS,SUPPORTINGPRODUCTSANDQUALITYCONTROL

A. TUWienSARToolbox

The SAR Geophysical Retrieval Toolbox (SGRT) is a software package developed by the ViennaUniversityofTechnology(TUWien)forextractinggeophysicalparametersfromSyntheticApertureRadar (SAR) data. The version 2.0 of the SGRT,written in Python programming language, is anadaptationtoSentinel-1 (S1)of theearlierSGRT1.0developedforENVISATAdvancedSyntheticApertureRadar (ASAR)data, incorporatingoptimizations intended forhandling theconsiderablyhigherspatialresolutionandresultingexplosionindatavolumesforeseenofSentinel-1relativetoENVISATASAR[RD16].SGRTconsistsoffourtypesofprocessingchainswheredifferentnumberofworkflowsaredefinedundereachtype:



DeliverableID:D3.1


• Pre-processing: calibration, radiometric correction, georeferencing andterraincorrection,resamplingandtiling,qualitycontrol

• Analytics:timeseriesanalysistoextractmodelparameters

• Production:generateslevel-2andhigher-levelproductsfrompre-processeddatausingmodelparameters.

• Near-Real-Time (NRT): this component is designed to integrate differentworkflowsandSGRTfunctionalitiesinafullyautomaticprocessingchainforproductgeneration.

Furthermore,SGRTisequippedwithseveralimageandsignalprocessingcomponents.SomeoftheSGRTmodulesareusedinexternalutilitiesandplug-ins.e.g.thePythonbasedTimeSeriesAnalyseris an in-house tool, developed and integratedwith the open sourceQGIS software to visualizeSentinel-1 timeseries.SGRT isundercontinuousdevelopmentandwitheverynewrelease,newfunctionalitiesandworkflowsareintroduced.

B. SupportingProductsandModules

In order to develop an operational service a reliable quality control mechanism is essential tominimizetheartefactsinthefinalproduct.Inmountainousareas,topographicnoiseisverycommonwhichisduetosteepslopesandshadoweffectcausedbytheSARsidelookingacquisitiongeometry.Inordertoremovetopographicerrors,theHAND(heightabovethenearestdrainage)index[RD17]wasused.Figure5-9showsanexampleoffloodmapbeforeandafterapplyingtheHandIndexmask.

Figure5-9:ApplicationofHandIndexmasktoremovethetopographicnoise.

BordernoiseinS-1A/BandENVISATASARisquiteconsistentwhichisamajorsourceoferrorintime series analysis. In order to handle this error source an independent border noise removalmodule[RD18]wasdevelopedandintegratedintotheSGRT.Figure5-10showsanexampleforS1bordernoiseremovalmask(bordernoiseremovalmaskshowninmagentacolour).



DeliverableID:D3.1


Figure5-10:AnexampleS1bordernoiseremovalmask.

C. Dataacquisitionandmanagement

TheS-1dataattheEODCwarehousearecurrentlyavailableapproximately2.5hoursaftertheinitialsignalprocessingbyESA(level-1product)and6.25hoursaftertheacquisition.TheS-1level-1dataarearchivedonfastdiscsstorageandbackedupusingarobotictapelibraryonaregularbasis.InordertomanagetherawandprocessedS-1datafilesadedicatedmeta-database(EOMDB:EarthObservationMeta-DataBase)hasbeenestablished,whichallowstrackingofdataavailabilityandprocessingstatus.

5.2.2.2CONCEPTS,REQUIREMENTSANDINFRASTRUCTURETheSentinel-1basedfloodmappingandmonitoringserviceisanexternalmodule,whichhasbeendevelopedandmaintainedbyTUWienfortheI-REACTproject.ItcanbetriggereduponrequestviaAzureServiceBus–acloudmessagingservicebetweenapplicationsandservices.

A. Conceptandbigpicture

The I-REACT project has a multi-tier architecture and is named as I-REACTOR. Based on thepresentation,applicationprocessanddatamanagementtasks I-REACTORsfunctionsare logically



DeliverableID:D3.1


separatedintodifferentlooselycoupledmodulesthatcanbeeasilymodifiedwithoutaffectingtheothermodules.However,incaseofS-1floodmappingservicethepointofcontacttoI-REACTORisIDI(I-REACTDataInterface),wherefinalproductsarepushedontothefront-endforfurtherdataharmonization, processing and visualization. Figure 5-ä11 shows the different components of I-REACTORandtheirinternalassociationsanddependencies.

Figure5-11:AnoverviewofI-REACTORframework.

B. REQUIREMENTS

Apartfromthesatellitedataandservicemoduletherearesomekeyrequirementsthatarecrucialforoperationalservices:

• RegularupdateofEOMDB,toinclude/updatetherecentlyacquireddata.• Internetconnection:whichisrequiredtoconnecttotheAzureServiceBusandreceive

the message to trigger the flood monitoring processor. Internet connection is alsorequiredtoconnect-and-pushtheprocesseddatafiles(floodmaps)andmetadatatotheIDI(I-REACTDataInterface)databaseremotely.

C. InfrastructureandservicelogicimplementationschemeWithintheframeworkofI-REACTproject,theSentinel-1dataprocessingchainforfloodmappingservice is implemented within a virtual machine hosted by the Earth Observation Data Centre(EODC).TheprocessingchainanddataflowincludesfollowingstepsasillustratedinFigure5-12.

• Flood mapping processor: The flood mapping algorithm is developed by TUWien andimplemented within a virtual machine at the Science Integration and developmentPlatform(SIDP)hostedbyEODC.Theprocessingchainincludespre-processingoftheSAR



DeliverableID:D3.1


data,dataqualitycontrols,floodmapping,andnecessarypost-processingsteps.TheSIDPisconnecttotheViennaScientificClusterwithmorethan2000computingnodeswhichmightbeusedforheavyprocessingtasksthatneedparallelprocessing.Thefinalproductsarefloodandfloodfrequencymaps.

• Serviceonrequestandtriggeringmechanism:thisserviceisestablishedtoproviderapidpost disaster flooding status and inundation extentmaps,which can be used for post-disastermanagementandreliefactivities.Inthisservicetheuser/customerwillcontactTUWien through I-REACT coordination (e.g. via Azure Service Bus) to trigger the floodmapping/monitoringservice.TherequestedproductwillbedeliveredtotheIDI.Thefloodmapping/monitoringserviceisfullyautomaticandcanbetriggeredoveradefinedlocationwiththemessagereceivedviaAzureServiceBus.

• Watchdog: after getting the triggeringmessage–with a boundingboxof an areaofinterestandmonitoringperiod(startandenddate)–viaAzureServiceBus,theprogramwill automatically connect to the EOMDB and get the available raw files. Then theprocessing chain will be activated for available files. If there is/are no file(s) then awatchdogwillbeactivatedanditwillupdateEOMDBaftereverytwohoursandlookfornew files for the area of interest. Every time when it will find new file(s) a newindependentprocessingchaininstancewillbelaunched.Thisprocesswillcontinueuntiltheimageacquisitiondateislessthanorequaltotheenddatedefinedinthemessage.

Figure5-12:Anoverviewofinfrastructureandservicelogicfornear-realtimefloodmapping/monitoring.

D. PROTOTYPE

Thestabilityoftheprocessingchainandtheimplementedservicelogicwastestedbytriggeringtheservice for three different requests simultaneously. The processing chain ran successfully and



DeliverableID:D3.1


processed21Sentinel-1scenes.Thefinalproductof2.5GBwasuploadedtotheIDIdatabase.Duetothelimitednumberofcoresonthetestmachine,theprocessingchainranformorethan24hours.In caseofmultiple requests theprocessing timecanbeminimizedby increasing the computingpower.

Sentinel-1basedfloodandfloodfrequencymapswereproducedforallfivetestsite.Figure6showsanexampleofSentinel-1basedfloodfrequencyproductforatestsite in Italy(Emilia–Romagna,Figure5-13)andUnitedKingdom(Figure5-14).

Figure5-13:Anexampleofmulti-temporalfloodfrequencyproduct(Italy).

Figure5-14:Anexampleofmulti-temporalfloodfrequencyproduct(UnitedKingdom).



DeliverableID:D3.1


InadditiontothefinalproductoffloodorfloodfrequencymapsthatwillbedeliveredtotheIDI,differentSentinel-1basedintermediateproductswillalsobeproduced.Theintermediateproductsinclude both pre-processed data and various biophysical and statistical parameters. In order toquerythedata/productapre-definedsetofminimumrequiredmetadataisalsopushedtotheIDIasshowninFigure5-15.

Figure5-15:ExampleofSentinel-1productuploadedtoI-REACTdatabase.

Afterdataharmonizationfloodmapsbecomeavailableforvisualizationonthefront-end(I-REACTmainwebpage)oftheI-REACTOR.Thedisplayofthefloodfrequencyproductonthefront-endoftheI-REACTemergencyservicemanagementwebsiteisshowninFigure5-16.

Figure5-16:Floodmonitoringframework(top)andprototypeoffinalproductdisplayedonthefront-endoftheI-

REACTwebpage(bottom).



DeliverableID:D3.1


Insummary,thissectionconcludesthedemonstrationofaframeworkandfeasibilityofsettingupanoperational floodmappingandmonitoringservicebyexploitingabig-data infrastructureandhighperformancecomputingfacility.Thisserviceisoperational(intestphase)undertheframeworkof I-REACTproject, andnewdevelopment in termsofalgorithm improvementandperformanceenhancementwillbeintegratedintotheSGRT.



DeliverableID:D3.1

GrantAgreementNo:700256 CallID:H2020-DRS-1-2015H2020-DRS-1-2015 Page:83of84

6 CONCLUSIONS

Theworkdescribed in the frameofDeliverable3.1– “ReportonEMS, EuropeanEarlyWarningSystemsandSentineldataintegration”representstheimplementationofexistingdataandservicesof Copernicus Emergency Management Services and Early Warning Services, for data andinformation on flood and wildfire monitoring as part of the I-REACT downstream services.Furthermore,withinthisdeliverable,theintegrationofprocessedSentinel-1dataintermsofflooddelineation and frequency mapping is also a crucial task. Moreover, local EMS services areintegratedwiththefocusonobtaininginformationconcerningalertsandmeasurementsfromthesemonitoringsystems.

ThetechnicaldevelopmentandinnovationworkwithinWP3,includingfurthersystemprototypinganddataintegrationofexternalservicesanddatastreamsalreadyavailable,providesoperationaldatafordownstreamservicesasexpected.Although,issuescamealong,suchasthedirectuseofthe EWS, the expectedCopernicus EWS couldbe implemented and integrated into the I-REACTsystembycollaborationwiththeJointResearchCentreoftheEuropeanCommission(JRC)andthecooperation of other I-REACT partners, which have access to the required input data and theexpertisetoproducethewildfiremonitoringinformation.

The results achieved have laid the groundwork for the creation of an early warning system toachieveincreasedpreparednesstoemergencies.Furthermore,otherpossibleCopernicusservices,whichwillprovideinformationforthedownstreamoffurtherdisasterservicesinthefuture,couldbeimplementedquitewell.

Thedatastreamsandserviceshavebeenintegratedfollowingtheas-a-serviceapproach,sothatnewdataisingestedautomaticallyintoI-REACTwithouthavingtheusertoactivateadatastream.Hence, the overall objective of this deliverable, to integrate existing data sources and systemsrelatedtonaturalhazardsintoI-REACTandtofilltheoverallsystemwithusefulinformationsources,isregardedasachieved.



DeliverableID:D3.1

GrantAgreement:700256 CallID:H2020-DRS-1-2015H2020-DRS-1-2015 Page:84of84

ENDOFTHEDOCUMENT

IMPROVING RESILIENCE TO EMERGENCIES THROUGH … · Improving Resilience to Emergencies through...

Documents

Transcript of IMPROVING RESILIENCE TO EMERGENCIES THROUGH … · Improving Resilience to Emergencies through...