Cloud Architecture for Processing and Distribution of Satellites … · 2014-09-25 · This project...

This project has received funding from the European Union’s Seventh Framework Programme for research, technological development and

demonstration under grant agreement no 318389

Cloud Architecture for Processing and Distribution of Satellites Imagery

J. Becedas, R. Pérez, G. González, F. Pedrera, M. J. Latorre

Rubén Pérez Pascual

FOSS4G-Europe July 15th 2014

2

Table of Contents

�  Introduction � Cloud Architecture 4EO � Preliminary Results � Conclusions � Acknowledgements

3

1 Introduction

4

Introduction

� The Organization �  Technology branch of Elecnor Group �  About 500 employees �  Markets: Aeronautics, Aerospace, Defense, Transport, Energy and

Environment, ICT and Security. �  Deimos 2 satellite was launched last month �  Official Webpage: http://www.deimos-space.com/

5

Introduction

� Deimos 2 satellite: ◦ First high-resolution spanish satellite. ◦  In commissioning stage.

6

Introduction

� Deimos 2 satellite: ◦ First high-resolution spanish satellite. ◦  In commissioning stage.

7

Introduction

� Traditional Processing Earth Observation (EO) images

8

Introduction

�  Official Webpage: http://www.bonfire-project.eu/ �  Partner from Fed4FIRE (http://www.fed4fire.eu/) �  Multi-Cloud testbed for services experimentation ◦  EPCC cloud (https://www.epcc.ed.ac.uk/) : Offers compute resources with Virtual

Machines for processing. ◦  INRIA cloud (http://www.inria.fr/): Offers compute resources with Virtual Machines

for processing. ◦  IBBT for emulated networks and shared storages. (http://www.iminds.be/).

9

2 Cloud Architecture

10

Cloud Architecture 4EO

� Architecture constituted by two layers:

Layer 1

Layer 2

D2 GS

PDGSFOS

GS M&C

Instrument ParametersAcquisition Report

Catalogue ResponsesBrowse Images

Predicted/Restituted OrbitPayload Plan

Catalogue QueriesBrowse Queries

G/S

S/C Contractor(SI)

Cloud Coverage Data Provider

NORAD (SpaceTrack)

KSAT

Launcher Provider

Operators

Auxiliary Data ProvidersADFs

External CalibrationADFs

GS Infrastructure

ADFs

Injection Orbit Data

Cloud Coverage Info

External TLEsConjuntion Summary Messages

Pass Schedule RequestTwo-Line Elements

Alamrs

UsersProducts

IERSEarth Rotation Parameters

CelesTrackSpace Weather Info

TCGSta M&C Events

Pass Schedule ConfirmationPass Anomaly Report

Payload DataTMGSta M&C Events

Precise D2 Ephemeris

TM/TCPayload Data

M&C EventsConfiguration

LogsAuthentication


LogsAuthentication

S/C DatabaseOn-board SW Binary Files

M&C Events

ORCHESTRATOR

IaaS Processing Chain

Archive&Catalogue Processors Storage

A&C Datablock

Raster and Vector (R&V) Datastore

R&V Datastore

Tiles Cache Tiles Cache Datastore

EO Server

11

BonFIRE

EPCC INRIA ORCHESTRATOR

IaaS Processing Chain Archive&Catalogue Processors

Storage A&C

Datablock

hGp hGp Internet

Raster and Vector Datastore

R&V Datastore

be-‐ibbt

Tiles Cache

Tiles Cache Datastore

Layer 2: Image DistribuLon and VisualizaLon Layer 1: Image processing and archiving

hGp

EO Server

Cloud Architecture 4EO D2 GS

PDGSFOS

GS M&C

Instrument ParametersAcquisition Report

Catalogue ResponsesBrowse Images

Predicted/Restituted OrbitPayload Plan

Catalogue QueriesBrowse Queries

G/S

S/C Contractor(SI)

Cloud Coverage Data Provider

NORAD (SpaceTrack)

KSAT

Launcher Provider

Operators

Auxiliary Data ProvidersADFs

External CalibrationADFs

GS Infrastructure

ADFs

Injection Orbit Data

Cloud Coverage Info

External TLEsConjuntion Summary Messages

Pass Schedule RequestTwo-Line Elements

Alamrs

UsersProducts

IERSEarth Rotation Parameters

CelesTrackSpace Weather Info

TCGSta M&C Events

Pass Schedule ConfirmationPass Anomaly Report

Payload DataTMGSta M&C Events

Precise D2 Ephemeris

TM/TCPayload Data


LogsAuthentication


LogsAuthentication

S/C DatabaseOn-board SW Binary Files

M&C Events

12


� Layer 1 Components: ◦ Orchestrator: i t manages the ingestion,

processing, archiving and cataloguing. ◦ Processing Chain Cluster: It processes the raw

data that the Orchestrator ingests and obtains geo-located images. It uses the Processing Chain Shared Store. ◦ Archive & Catalogue: It stores and catalogues

the Processing Chain products. It uses the Archive & Catalogue Storage for storing the images.

13


� Layer 2 Components: ◦  Image distribution and visualization module �  EO Server �  Tiles Cache

14

BonFIRE



Storage A&C

Datablock

hGp hGp Internet


R&V Datastore

be-‐ibbt

Tiles Cache


L2: Image DistribuLon and VisualizaLon L1: Image processing and archiving

hGp

EO Server

Cloud Architecture 4EO: Orchestrator

15

Cloud Architecture 4EO: Layer 1

� Orchestrator ◦  It connects with the Ground

Stations by FTP protocol.

16



Stations by FTP protocol. ◦  It ingests the available Raw Data

from the Ground Stations

17




from the Ground Stations ◦  It creates a new Processing

Chain to process the ingested data.

18




from the Ground Stations ◦  It creates a new Processing

Chain to process the ingested data. ◦ When a Processing Chain ends

its execution, the Orchestrator sends the results to the Archive and Catalogue module.

19

BonFIRE


IaaS Processing Chain Archive&Catalogue

Processors Shared Storage

A&C Datablock

hGp hGp Internet


R&V Datastore

be-‐ibbt

Tiles Cache



hGp

EO Server


20


� Processing Chain Cluster ◦ Elasticity as a Service (IaaS) works in a cluster. ◦ When a new raw data is incoming, a new

Processing Chain is created by the IaaS manager for performing the data. ◦ When the Processing Chain has finished, the used

resources are released. ◦ Processors Shared Storage where the temporally

files are stored during the processing.

21


� Processing Chain: Stages ◦ L0 �  The acquired data is organized into

image sectors of predefined size and structure and it is converted into scenes.

22


� Processing Chain: Stages ◦ L1A �  L0 products are calibrated in units of

radiance.

23


� Processing Chain: Stages ◦ L1B �  L1A products are geolocated,

resampled and packed.

24


� Processing Chain: Stages ◦ L1C �  It performs the ortho-rectification of the

L1B products using ground control points.

25

BonFIRE



Storage A&C

Datablock

hGp hGp Internet


R&V Datastore

be-‐ibbt

Tiles Cache



hGp

EO Server


26


� Archive & Catalogue ◦  It is implemented by Geo-Server and CSW Geo-

Server’s plugin. ◦  It stores and catalogues the processed images. ◦  It provides the catalogue through a CSW interface.

27


� Tools

28

3 Preliminary Results

29

Preliminary Results

� Reduction in the delivery of the images to the end users. About 8 minutes a radiometric corrected and geolocated image.

� Reduction in the transfer of data. � Parallel image processing. �  Automatic archive and catalogue of satellite

imagery. �  Instantaneous access to satellite imagery through

the Internet.

30

4 Conclusions

31

Conclusions �  The architecture reduces the images delivery time

for end-users. � Using open-source platforms, an operational cloud

architecture can be implemented. �  The EO bussiness capabilities can be improved

using cloud computing. �  Fed4FIRE provides multiple test-beds and tools for

experimenters facilitating: ◦ The deployment, control and monitor of the experiment ◦ Easy access to the testbeds ◦ Valuable tools for experimentation

32

Acknowledgement �  This work was carried out with the support of the

Fed4FIRE-project (“Federation for FIRE"), an Integrated project receiving funding from the European Union’s Seventh Framework Programme for research, technological development and demonstration under grant agreement no 318389

�  It does not necessarily reflect the views of the European Commission. The European Commission is not liable for any use that may be made of the information contained herein.

33

Thanks for your attention

Contacts: [email protected] [email protected] [email protected] [email protected]

Cloud Architecture for Processing and Distribution of Satellites … · 2014-09-25 · This project...

Documents

Transcript of Cloud Architecture for Processing and Distribution of Satellites … · 2014-09-25 · This project...