DAS Industry Day - ESAemits.sso.esa.int/.../DAS_Industry_Day-final2.pdf · DAS INDUSTRY DAY – 1...

ESRIN – 1 July 2015

DAS Industry Day

Pascal Gilles (EOP-GT) DAS INDUSTRY DAY – 1 July 2015

Welcome – Introduction

DAS Industry Day – 1 July 2015

DAS Industry Day – 1 July 2015 - AGENDA

• 9:30 Welcome – Introduction: Pascal Gilles, H / EO Ground Segment Infrastructure and Operations Management Division

• 9:45 Ground Segment Context and Evolution: Pascal Gilles • 10:30 Common Services in GS Operations: Damiano Guerrucci,

H/Common Services Section • 10:45 DAS Scope of Data: Peggy Fischer, Mission Operations

Management Section • 11:00 Coffee break • 11:30 DAS Procurement Highlights: Michel Douzal, Common Services

Section • 12:15 DAS Procurement Features and Schedule: Luc Govaert, H/ EO

Ground Segment Procurement Service • 12:30 Q&A • 13:00 Adjourn


DAS Industry Day – 1 July 2015 - AGENDA

• 9:30 Welcome – Introduction: Pascal Gilles, H / EO Ground Segment Infrastructure and Operations Management Division

• 9:45 Ground Segment Context and Evolution: Pascal Gilles • 10:30 Common Services in GS Operations: Damiano Guerrucci,

H/Common Services Section • 10:45 DAS Scope of Data: Peggy Fischer, Mission Operations

Management Section • 11:00 COFFEE BREAK • 11:30 DAS Procurement Highlights: Michel Douzal, Common Services

Section • 12:15 DAS Procurement Features and Schedule: Luc Govaert, H/ EO

Ground Segment Procurement Service • 12:30 Q&A • 13:00 Adjourn

Pascal Gilles (EOP-GT) DAS INDUSTRY DAY – 1 July 2015

Ground Segment Context and Evolution


European EO Ground Segment – state of affairs in 2013

ESA • Budget composition and scales are changing: overall increase over years to come but decrease

in ESA member states budgets and increase in EU budgets linked to Copernicus. • Reduced ESA budgets impose rationalisation of efforts and functions. • Transition to service model on-going for some functions. • End of EOEP funded mission operation (but still to complete data transcription, ingestion,

systematic processing and access activity) and transition to LTDP. • Increased focus on “big data” and “Data Intensive Science” (e.g. Exploitation Platforms) and

new user collaboration schemes.

National Actors • Work under same budgetary constraints than ESA. Need to demonstrate return on national

missions, sometimes including dual-use ones. • Operate large parts of EO ground segment infrastructure in institutional domain.

Industry & Technology Trends

• Maturity of industry and technologies; increased appearance of “service offerings”. • Mature IT and communications sector looking for opportunities also in EO. • Aggressive stance of Google and appearance of initiatives from other, non-European ”Google

type” Cloud Service Providers – mainly focussed on data/products. • New media and channels for communication with user communities: Cloud, Social media, Link

EO data. • Cyber Security – protect data integrity and avoid data loss.

Presenter

Presentation Notes

Reduced EOP budget, other types of ESA missions, GSC budget with little industrial policy EO is affected by evolution of technology and user demand (bring processing to the data… architecture up to last year ESA current architecture evolution plans (exploitation platforms, PAC rationalisation etc questions for future (google etc?)


ESA EO GS Operations Concept Evolution: Core features of the target concept

• One core EO GS Reference Operations Concept for all ESA and TPM missions • Ensure migration from Envisat to EE era, i.e. big to small, static to flexible, user-

driven to systematic, NRT/Off-line to systematic/reprocessing, PB-EO paper to ITT • Modularity and flexibility: can be tailored as necessary to meet individual mission

needs • Compatible with historic (ERS/Envisat/TPMs), current (Sentinels, Earth Explorers,

TPMs) and future EO missions • Definition of the reference operations concept based on the following principles:

– Focus on services defined in SLAs and use of KPIs wherever appropriate – Focus ESA SW development effort to support innovation and cover strategic operations

functions where this is necessary – Avoid HW procurement (e.g. rely on IAAS type services) – Focus on standardization of interfaces – Focus on long term data and associated information preservation, curation and access – Focus on “data to users” and support to Scientific Exploitation of EO data

• Gradual evolution from current to reference scenario – Corresponding functional / organisational / procurement approach

Presenter

Presentation Notes

Tailored Use cases in Annex was stated on slide – could not find it!

Evolution Process and expected outcome

• Evolution Process – Apply to current ESA GS Architecture for ERS, ENVISAT, Explorers and TPMs – Foresee convergence with the current CSC architecture and contractual options

• Consultation Process with

– Current Industrial partners & New Types of Industries – GSCB & ESA Member States – EC (DG-Entr & DG-Connect, possibly DG-Research)

• Expected Outcome – Modern, efficient architecture for ESA and national ground segments – Transparent and consistent ESA/EC procurements in line with national investments and

approaches

Presenter

Presentation Notes


Functional architecture concept

• Five main areas – User area – Data access and services area – Data generation area – Data improvement area – Data preservation area

• Based on simple and straightforward interfaces between areas and

elements within areas

• No (or minimalistic) dependencies between areas or elements within areas

Presenter

Presentation Notes


Our Mission Statement

With the aim to develop a concept around the idea “Enable the users”, European Earth Observation Ground Segment operators attempted to address the following key challenges: Access conditions to Earth Observation data do not adequately support projected future

working practices within the geo-information user communities, Current operational architectures are not appropriate to support step changes in high volume

data access and processing, The number of Earth Observation satellites being launched continues to increase placing increased

pressure on data management infrastructure, and at the same time offering unprecedented opportunities to derive information (as long as data can be easily and widely used),

State-of-the-art high-volume data processing/fusion infrastructure is being put in place outside of Europe (e.g. US) and this is not necessarily focused on Earth Observation alone. This means that in these countries, Earth Observation does not bear the full costs of developments exploiting state of the art processing/ fusion/storage and Earth Observation developments are embedded in a wider innovation/information community. Europe is not present in this development whereas the US is.

EO ground segment development activities in Europe are not effectively connected to European level IT developments.

The Core Ground Segments need to be able to flexibly respond to evolving data demands from this environment.

The overall implementation approach should allow reducing the cost of operations.

Enable the user communities to create value through Earth Observation data exploitation

Our view of on-going discussion related to the evolution of the European Earth Observation Ground Segment

Common Subjects: - Technical architecture,

standards, I/F - Data & information policy - Industrial management &

partnership schemes - Funding schemes

Big Data Initiatives • driven by

dependency on US IT commercial providers

• EC led

ESA/MS GS-Evolution • driven by desire to enable

current and new users; part of ESA EO Strategies

• Preserving & fostering Europe’s EO Data and Knowledge

• ESA and National Programmes

Copernicus Space Component GS Evolution • driven by needs of Copernicus

services • Follows EC/ESA management process • EC/ESA Copernicus Agreement

Sentinels Collaborative GS Evolution • driven by ambition to

stimulate new services in MS beyond core services

• Cooperation Agreements ESA-MS/ESA

Presenter

Presentation Notes

----- Meeting Notes (17/2/15 10:39) ----- Delegations point out that we need to make sure that whatever concepts, architectures and interfaces we come up with find their place within the solutions/service definitions that the EC will come up with. A challenge consists in linking the GSCB process and results to the EC efforts. All agree that in the end these elements need to be aligned. Will need to find a way how we can achieve this. UK states they would want this alignement.

5 interconnected user groups

Users geosciences

Earth Science communities at large

International programmes (e.g. WCRP)

European research programmes (e.g. EPOS)

Multilateral Development Banks (e.g. World Bank)

Policy and societal drivers in areas such as global change and disaster risk management International Conventions (e.g. UNFCC)

Public needs at European, national, regional or local scales, including weather forecasts

Users public services

Users profit-making services

Value adding industry including SMEs and some represented by EARSC

Downstream industry sectors (e.g. Oil & Gas, Renewable energy, Insurance, re-insurance, Mining, Meteo, etc.)

Users R&D remote sensing

Experts for optical remote sensing

Experts for active microwave

remote sensing

Experts for passive microwave

remote sensing

Experts for altimetry remote sensing

Experts for atmospheric composition

remote sensing

Experts for gravimetry sensing

Experts for magnetism sensing

Experts for scatterometer remote

sensing Web / TV media

General public, education, media

Schools, teachers

Our data user communities: 5 interconnected groups

“traditional” data delivery

Network of community platforms

“EO-Innovation Europe”

“Big Data”

Heritage EO data

(e.g. Spot-1, Envisat)

Operational public EO data

(Sentinel, Meteo)

R&D EO data (e.g. Earth Explorers)

Commercial EO data

e.g. RapidEye,Deimos

The best worldwide EO data asset

Europe EO

Increasing data volume &

diversity

How:

Interoperable/interconnected platforms around a core element

Open to multi-source funding initiatives

Common governance rules

Objectives of the network concept:

Enabling large scale exploitation of EO data

Stimulating the innovation with EO data

Maximising impact of European EO assets and preserving European independence

Airborne & in-situ data

Vivid global user communities

Users R&D remote sensing Users profit-making

services

Users geosciences

Evolving user

expectations


5 interconnected user groups


… and the way they access and process data

Inner element

Enabling element

(common to all platforms)

Outreach element A Network of community platforms

The EO-Innovation Europe concept:

Enabling element, Inner element, Outreach element

Connect to other big data ecosystems

EO-Innovation Europe

Enabling element (common to all

platforms)

EO-Innovation Europe Enabling element

Technical aspects for interoperability, e.g.:

• Federated user interfaces subsystems (e.g. catalogues interlinked), interface & standards definition and agreement [technological EP]

• Federated user management (i.e. user registers only once and is recognized by all exploitation platforms)

• Development of common value-creation techniques (research in data analytics and information retrieval, information visualization, data mining, fusion of in-situ data with geoinformation, etc.) [technology r&d]

Open, flexible, interoperable

Economically attractive

The engine of EO-Innovation Europe

Economical aspects for interoperability, e.g.:

• Common EO data pool (joint negos like Copernicus Data Warehouse) and optimised joint observations strategy amongst missions

• Processing capability management (federation of resources and cloud services for processing)

• Management of user affiliation, user sponsorships and payments (i.e. user processing/data quota, etc.)

Programmatic aspects for interoperability, e.g.:

• Full synergy with ESA Exploitation Programmes (CCI, SEOM, STSE, VAE, DUE, MOM) and national programmes, e.g. re-use of toolboxes, new processing algos, etc..

• Relations with supporting activities funded by EC, in particular with science cloud computing

• Relations with similar activities for non-space data

Interoperable/interconnected platforms around an enabling element

EO-Innovation Europe Inner element

[ Stimulating element ]

• Mission Exploitation Platforms (e.g. Proba-V MEP, Swarm MEP, SMOS salinity MEP, Biomass MEP, etc.) animated by the DISCs (Data Innovation and Science Clusters) [former ESLs]:

DISCs for Earth Explorers (animating their respective MEP)

DISCs for traditional EO sensors: Altimetry data expertise Medium Res SWIR-TIR optical data expertise Medium Res VNIR optical data expertise High Res Optical data expertise Active microwave (SAR) data expertise Scatterometer data expertise Atmospheric composition UV-Vis data expertise Atmospheric composition profile data expertise etc. Expertise on EO sensors data

R&D communities

The EO data expertise of EO-Science Europe

Inner element

Users R&D remote sensing

Stimulating innovation with EO data

• Open data, Open API, Open Publications,

• Cloud-based data analytics, Open Source Toolboxes, libraries of workflows for EO processing,

• Virtual Living Labs, App camps, Hackhathons,

• Crowdsourcing & Citizens science activities,

• Advanced training of new class of data scientist,

• Scientific outreach on social media,

• E-learning, MOOC on EO for Climate,

Stimulating science with EO data


EO-Innovation Europe Outreach element

• Thematic Exploitation Platforms (Geohazards, Urban, Hydrology, Polar, Coastal, Forestry, etc.)

• Regional Exploitation Platforms (Baltic, Africa, etc…)

• Education Exploitation Platforms

• Link with non-EO data platforms, e.g. Geohazards TEP and European Plate Observing System (EPOS)

• Future support to exploitation, e.g. future EOEP VAE/SEOM/DUE/STSE, CCI future, and relevant national and EC activities

Connexion between EO communities and non-EO communities

EO-Innovation Europe for the world

Users public

services

Users geosciences General public,

education, media Outreach element

Users profit-making

services

Big Data

Connecting to the world


Users profit-making

services

Users geosciences


Outreach element

Network of community platforms


European Plate Observing System

(EPOS) Copernicus Land

Service

TEP Hydrology community platform

TEP Geohazards community platform

Education community platform

TEP Urban community platform

Regional/National community platforms

International development

community platform

ECV community platforms

Inner element

Science 2.0

R&D remote sensing communities

QA4EO

DISC

SMOS MEP

EarthCARE MEP

Archive EP

Biomass MEP

Enabling element

(common to all platforms)

Core element, Inner element, Outreach element


Long Term Data Preservation (LTDP) Element Activities

A. ESA & TPM EO Heritage EO Data Assets Preservation: preventing loss of heritage assets in data & associated knowledge.

o ESA EO Data & Knowledge Preservation System & Services set-up relying on state of art technology and in line with international & recognized digital data preservation standards; population, maintenance and evolution.

o Data and associated knowledge consolidation, reformatting, archiving and preservation according to international standards and best practices and through the use of preservation systems and services.

B. European EO Heritage Data Preservation: preventing loss of EO data judged of European long-term interest; international coordination & cooperation projects with MSs in the data preservation domain.

C. ESA Wide Coordination:

o Preservation systems and services sharing with D/SRE and D/HSO and definition and implementation of common policies and approaches to data & knowledge preservation at ESA level (D/EOP lead).

o D/SRE and D/HSO Specific Activities (D/SRE & D/HSO lead).

A. ESA & TPM EO Heritage Data Assets Access & Curation

• Products generation & access provision through GS systems, innovative services & tools relying on state of art technology (e.g. primitive features extraction, data & knowledge unified access, user workflow management).

• Data usability & quality improvement, algorithms evolution and reprocessing, data

transformation & adaptation for user enhanced access and information extraction, data fusion, data presentation and organization improvement, Sensor Data Records maturity level assessment & elevation, provision of “Data as a Service” as input to “EO Innovation Europe”.

“Mission to Heritage Data” Element Activities

Damiano Guerrucci (EOP-GTS) DAS INDUSTRY DAY – 1 July 2015

Payload Data Ground Segment Common Services


Overview

• Current Organisation Structure • Common Services Section Objectives • Common Services Portfolio • Status of Main Contracts • Monitoring and Report


Current Organisation Structure


Common Services Section Objectives

• Management of the Procurement and Operations of the Common Services managed by the Operations and Infrastructure Management Division, that currently consist of:

• User Services Help and Order Desk and Customer Management for ESA EO and TPM missions

• Information Services to Users • The CDS (Coordinated Data access System) for Copernicus missions • The overall PDGSs Configuration and Change Control • Data Dissemination services for ESA EO and TPM missions • Data Archiving services • Data Reprocessing services

• Make use of the Network and Infrastructure Services provided by the division (LANs and WAN)

• Monitoring the performances of the Common Services in accordance with their respective Procurement Contracts Service Level Agreements

• Provision of appropriate Reporting on Common Services to the Mission and PDGS Managers via the PDGS Operations Manager (including input to the QSRs)

• Review and update objectives and workplan accordingly


PDGS Services Reporting

Mission B

Mission C

Mission A

Mission X

Service A

Service B

Service C

Service Y

Report to Mission Managers

Mission Operation Manager

Ser

vice

Coo

rdin

ator

Services Reports


Common Services Portfolio

EO Data User Ground Stations/Remote Centres

Hosting HW (Cloud…)Network

Information Services

Data Discovery &

Access

Data Acquisition

Mission Planning

Processing & Re-processing

Data Circulation Data Archive

& Inventory

User Service & Order Handling

Data Management

SW MaintenanceService Desk

CollaborativeServices

Quality Control

NRT Processing

User Community

Platform


PDGS Common Services Data Flow


Data Archiving Service Objectives

• Implement the ESA Master Archive, which will represent the ESA reference

storage of Earth Observation data products

• Standardised interfaces and data formats

• Highly performing data ingestion and bulk extraction

• Reliable storage for long term data preservation

Peggy Fischer (EOP-GTE) DAS INDUSTRY DAY – 1 July 2015

DAS Scope of Data


DAS: Scope – Data (1/4)

The data scope includes: – Past, present and future ESA and ESA Third Party Missions

• Historic ESA missions (ERS, ENVISAT, GOCE); • Historic ESA Third Party missions (e.g. Landsat series, JERS, ALOS); • Active ESA Third Party Missions (e.g. Landsat-8, GOSAT); • Active ESA Earth Explorer Missions (SMOS, SWARM, Cryosat-2) • Future ESA Earth Explorer Missions (ADM-Aeolus mid 2017, Earth Care

2018)

– Instrument data: raw, level 0, L1A, L1B, L2 – Latest (re)processing versions + at least one previous processing

version/processing baseline (L1, L2) as per mission policy – Ancillary, auxiliary data – Data from airborne campaigns + In-situ data


DAS: Scope – Data (2/4)

The data will be provided: – As ESA consolidated master datasets (L0) if data consolidation activities

have been completed (majority of data) – As unconsolidated L0 datasets if further data consolidation is not

foreseen or will be performed at a later stage (outside DAS) – With associated metadata (as available) – In different formats as available (Envisat, EE, Net-CDF, EO-SIP

formatted, non EO-SIP formatted etc) – EO-SIP = Earth Observation Submission Information Package:

Harmonized packaging format for submission and internal exchange of ESA & TPM EO Data Records

– With Data related documentation (e.g. format description, EO-SIP specifications, completeness analysis reports, data release notes etc)


DAS: Scope – Initial data basket (3/4)

• An initial basket of datasets will be published with ITT:

– Envisat/ERS mission datasets: ~ 2.6 PB, ca. 250 instrument datasets + auxiliary data

– Historic TPM mission datasets: ~ 2.0 PB, ca. 80 instrument datasets + auxiliary data (e.g. Landsat 1-7 series, ALOS, MOS, JERS)

– Active EE missions (existing archive): ~ 0.4 PB initial – Active TPM missions (existing archive): ~1 PB initial (GOSAT, L8) – Active missions: ~ 7-8 TB weekly increase (newly acquired data)

• Total: ~ 6 PB initial data volume • Ca. 20 M EO data files initial


DAS: Scope – Initial data basket (4/4)

• Data for Phase-in period: – ~3-4 PB – All ERS/Envisat data – Existing archives of EE data – Selected TPM datasets (e.g. Landsat 1-7 series)

• Provision of remaining initial data basket gradually after phase-in period

during early operations phase • Data scope to evolve in time, expected growth ~ 5-10% p.a. • Data scope for DAS driven by ESA Mission Manager(s) and ESA PDGS Mission

Operations Manager(s)


Coffee break

Information Day on Data

Michel Douzal (EOP-GTS) DAS INDUSTRY DAY – 1 July 2015

DAS Procurement Highlights


Contents

• Context • Data Contents • Services – Functions • Performance Management • Data Interfaces and Data Flows • CFI’s • Transition: phase-in / phase-out • Service approach • Costing approach – Cost model • Timeline • Opportunity for Industry

DAS Procurement:

CONTEXT


DAS Context

• PACs re-arrangements • Phase-out project on-going (planned to finish by mid 2016) • Includes disposal of off-line legacy tapes • Data repatriation campaign started in 2012 and on-going

• Data consolidation and processing on-going by DSI service

• Level 0 master datasets for all ERS/Envisat instruments (inter alia)

• Replaced by Cold Backup Archive in ESRIN (LTDP) + Master Archive (DAS)

• DAS mandated functions:

• Core: Reliable storage, Data bulk extraction • Other functions (eg ingestion) as enablers of the core functions


PDGS EO Operational Concept

DAS Procurement:

DATA CONTENTS


DAS Scope

• Data scope: see dedicated presentation • Detailed view attached to the SoW • Data volume and composition are a realistic idea of data volume (order

of magnitude, not an accurate amount or composition), to be used as a reference for proposals evaluation and initial sizing

• Growth expected => scalability required + cost model approach

DAS Procurement:

SERVICES - FUNCTIONS


DAS Scope overview: Functions

• Core services: overarching objective of DAS • Data Archival • Data Bulk Extraction and Delivery

• Functional services: handling data to enable core services • Data Ingestion • Data Evolution • Data Alignment • Data Management

• Management and Support Services • Ticket Management, Reporting, Financial Management, Phase-in/out

• Out of scope: • Data transcription from historical media • Data consolidation / transformation / conversion


Data Archival

• Data integrity • Security & connectivity requirements • Resilience to disaster (no business continuity) – max 5 NWD loss of

data updates • Territory of ESA MS • Data to remain readable and aligned with the meta-data

• Continuous random checks on data samples

• Safeguard clauses • One full “central” copy of the data in a single location / dedicated

infrastructure • ESA reserves right to purchase central copy infrastructure from

Supplier at book value – at any time

• Bidder / supplier to propose convincing architecture – Including choice of technology


Data bulk-extraction and delivery

• Data identified through queries on the meta-data • Delivery on-media (> 1 TB) or on-line via FTP or similar (< 1 TB) • Data delivery traced, recorded, reported


Data Ingestion

• Requested by Mission Manager / Ops Manager as “Data Owner” • Data collected one-off or continuously (live missions)

• Data Sources: • Acquisition stations and processing facilities (live missions) => L0, Aux • ESA-contracted processing and exploitation services (eg DSI) =>

Consolidated data, higher levels • International mission operators eg USGS (TPMs) => Data as agreed • Other sources

• Pick-up point: case-by-case or specific front-end server TBD • Channel: media (one-off) / on-line (continuous)

• Population of the archive and Data Management System • Detection + recovery of anomalies (media / transcription) • Sample-based verification • Reporting


Data Evolution / Alignment

• Data Evolution • To reflect external events / data improvements • Update to the data (file addition / removal) and/or meta-data • Purging data on-request (safeguards apply)

• Data Alignment • Several archives in the conceptual model • Need to detect / resolve discrepancies • Data alignment / reconciliation physiological for live missions • Includes

• Comparing file listings, listing differences (based on “ls –l” + MD5) • Swapping corresponding files • Implementing (on-request) the alignment • Reporting

• Data Management System kept up-to-date at all times • No other data transformation (processing, conversion etc) in the

scope of DAS


Data Management

• Management of the data = value added between storage and archival • DAS service + Supplier = responsible for managing the data asset (not

for storing files) including: • Control the contents of the archive

=> data identification, versioning, configuration

• Track the operations performed by DAS itself => physical location, repatriation, evolution, alignment

• Meta-data captured at ingestion time • Related functions:

• Answer queries (interactive / Q&A) on the archive contents • Provide various reports • Export the meta-data to centralised system(s) • Track / undo changes to the data • Support data bulk-extraction and delivery


DAS Management

• Management and support processes • Management / life cycle of Service tickets • Technical and Support processes – scalability, availability • Performance Management • Communication Management – Service web site • Service Monitoring and Reporting • Interfaces / third parties (Service Interface Documents) • Tools

• Overall Service and Contract management:

• Meetings, deliverables, etc. • CFIs

• Financial Management • Cost modelling

DAS Procurement:

PERFORMANCE MANAGEMENT


Performance Management

• Performance requirements / KPIs • KPIs with / without penalties • Real-time monitoring of KPIs + end-of-period reporting • RCA for high-severity issues

– total or partial loss of critical functions: data management / delivery – risk to the integrity of the data

• Penalty scheme • First 4 months after SRR = SLA applicable, no penalties • Contract cancellation criteria: severe & prolonged under-performance

DAS Procurement:

DATA INTERFACES DATA FLOWS


DAS – Main data flows

Master archive

DAS

Data procurement Data acquisition Systematic processing Reprocessing Bulk-processing Conversion Cold backup Others

Stations

Sources

DSI

DISSHARM

External / centralised repository(ies) of meta-data

Functions Services

Others

Data dissemination Support to scientific exploitation Data quality Reprocessing Exchange with partners Others

Destinations

SPPA

DISSHARM

Functions Services

Others

TEP

DSI


Notes on DAS main data flows

• Some sources of data are also destinations • Data flows are on-media (NAS) or on-line (FTP) • Continual data flows (live missions) are on-line (FTP); one-off data flows

are on-line or on-media depending on volumes • Interfaces are the same regardless of source and destination, only the

details change: mail address for shipping the NAS / FTP address for on-line repository

• A single Front-End may be put in place to act as single physical source for all incoming data

• Continual ingestion (live missions) is similar to regularly repeated one-off ingestion - (documentation issued once + updated many times)


DAS – Interfaces

• Data input (ingestion) • One-off : on-media (NAS) or on-line (FTP) depending on volumes • Continuous: on-line (FTP)

• Data output (delivery) • On-media (NAS) or on-line (FTP)

• Meta-data export to centralised functions: • Export of structured files (CSV / XML) to be imported into central

data management repository(ies) outside the scope of DAS

• Data flow approach to be tuned according to: • Specific requirements of missions using DAS as archive component • Potential availability of a centralised Front-End server • Industrial proposals

• DAS to be part of the ESA network (see Security / Connectivity)


DAS Connection to EO Multi Mission WAN


DAS Connection to EO Multi Mission WAN

EO WAN Services:

• WAN Connectivity

• Front-end with FWs and LAN connectivity to interface Contractor’s DAS infrastructure

• Public Internet access

• Network auxiliary services (i.e.: DNS, NTP, Mail Relay and Proxy)

• Remote Access Services

DAS Procurement:

CFI’s


DAS INPUTS and CFI’s

• Minimised CFI’s / Maximised responsibility of the Contractor (see Service Approach)

• Initial data input: ~50% will be delivered to the Contractor within 3 weeks of KO (NAS) for start of ingestion – remainder within 3 months

• Initial Media left at the disposal of the Contractor –no guarantee • Connectivity to the ESA network: provided by ESA including

equipment and dedicated line • Service Management tool: TellUs (based on ServiceNow) for ticket

management

DAS Procurement:

TRANSITION Phase-in / Phase-out


DAS – Transition: phase-in / out

• Phase-in • Initial data mostly provided on NAS, ~50% shortly after KO • Initial data ingestion expected to be on critical path of phase-in

schedule • Convincing solutions expected for early start of ingestion

• Phase-out • Output data much larger still ! • NAS copy approach still practicable? • Option for ESA to purchase supplier’s storage infrastructure • Convincing solutions expected for efficient phase-out

• Disruption of archive phase-in/out mitigated by contract duration • Phase-in/out the most delicate operations in DAS, performed when the

contract is just starting or terminating => professionalism required

DAS Procurement:

SERVICE APPROACH


DAS – Service concept

• Service concept • Industry to propose the implementation / solution • Industry to take responsibility for delivering the services • No solution / technology imposed by ESA • Architectural constraints as boundaries • CFI’s minimised – driven by overall efficiency / security • ESA not supplying the supplier • Prime Contractor managing / representing the whole Consortium and

taking overall responsibility

• Infrastructure owned / provided and hosted by the Supplier

– ESA reserves the right to purchase

DAS Procurement:

COSTING APPROACH COST MODEL


Costing approach – Cost Modelling

• Cost model is a powerful instrument • Simplifies cost determination, handles fluctuating / increasing scope of

data • Based on objective criteria known before work is committed, minimises

negotiations • Simplifies contractual and financial management • Costs are predictable, transparent, linked to the level of activity • Continued benefits of initial competition

• Technique successfully applied in previous environments / contracts • Covers all costs of all services – except Phase-in / Phase-out • Trade-off accuracy vs complexity (law of large numbers)


Cost Model

• 3 phases – Proposal and evaluation phase:

• FFP per dataset

– Negotiation phase: based on the FFP, agreement on the definition of the cost model including

• Units of cost • Charge to ESA for every unit • Cost qualifiers • Resulting formula to calculate service costs over contract duration

– Operations phase: • Application of the formula to all datasets • Tuning if needed at yearly service review


Example cost model: proposal & evaluation phase

• Initial basket attached to SoW contains the currently expected datasets to be archived

• Financial proposal contains FFP quotes for each dataset, taking into account the features of the data

• Naïve example:

Dataset Size Nb files Format Other features

Cost of 1 year archival including all services

Dataset 1 100 Gb 100 Envisat … 200 Eur

Dataset 2 200 Gb 50 EO-SIP … 90 Eur

Dataset 3 50 Gb 100 ERS … 110 Eur

Total 350 Gb 250 - … 400 Eur


Example cost model: negotiation phase

For Data in the initial basket, based on the negotiated FFP:

1.Agree on basic Unit of Cost eg 1 File to be archived 2.Agree on method to count the units eg “ls –l” on data input 3.Define qualifiers (cost impacting features) eg avg size of each file, format 4.Associate coefficients to qualifiers eg big -> 1.3, medium -> 1.0 , small -> 0.6 EO-SIP -> 0.8, other -> 1.1

5. Derive formula to count the units of cost in the Service, eg

Nb of DAS Units = Nb basic Units * size coeff * format coeff 6. Apply formula to initial basket eg 110+52+66 = 228 7. Derive the cost per Das Unit eg 400 Eur / 228 = 1.75 Eur

8. Formula for the cost of archiving any dataset for 1 year:

Cost of archiving a dataset for 1 year = Nb DAS Units of the dataset * 1.75 Eur

Basic units FFP cost Service units Cost per service unit Nb Files Avg size Size coeff Data format Format coeff in proposal per dataset

Dataset 1 100 100 Gb 1.0 Envisat 1.1 200 110 Dataset 2 50 200 Gb 1.3 EO-SIP 0.8 90 52 Dataset 3 100 50 Gb 0.6 ERS 1.1 110 66 Total 250 400 228 1.75


Example cost model: contractual phase

Upon request for archiving any Dataset: 1. Apply method to count the Files -> “ls –l” on the media delivered 2. Determine the value of the qualifiers -> size range, format 3. Look up value of coefficients for qualifiers -> 1.3, medium -> 1.0 , small -> 0.6 EO-SIP -> 0.8, other -> 1.1 4. Calculate the Nb of DAS Units of the Dataset -> Nb Files * size coeff * format coeff 5. Apply cost formula -> Yearly Cost = Nb DAS Units * 1.10 Eur

Basic units FFP cost Service units Cost per service unit

Service Cost per dataset Nb Files Avg size Size coeff Data format Format coeff in proposal per dataset

Dataset 1 100 100 Gb 1.0 Envisat 1.1 200 110 193 Dataset 2 50 200 Gb 1.3 EO-SIP 0.8 90 52 91 Dataset 3 100 50 Gb 0.6 ERS 1.1 110 66 116 Total 250 400 228 1.75 400


Cost Model main features

• Procurement Proposal approved by IPC (*) over 2+3 years (beyond that amount, new IPC approval is needed)

• Initial basket is for quoting and sizing purposes • No relationship between IPC approved amount and initial basket • Datasets assigned case by case – 5 million Euro min.

guaranteed over 2+3 year duration • Adjustments of unit costs linked to time (covering years 2 to 5

of the contract) and volume (covering expansions / contractions)

• Model applies at start of operations - Phase-in is charged as FFP (*) budget availability depending on CMIN

DAS Procurement:

TIMELINE


Timeline for DAS ITT

• DAS Industry day 1 July 2015 • Issue of DAS ITT 10-15 July 2015 • Closing date for proposals 18 Sept 2015 • TEB Nov 2015 • Negotiation Feb 2016 • KO Mar 2016 • Phase-in / SRR (6 months) Mar - Sept 2016 • Initial Operations (4 months) Oct 2016 – Jan 2017 • Routine Operations 2017

DAS Procurement:

OPPORTUNITY FOR INDUSTRY


DAS Industrial opportunity

• Why bid? • Competitive approach • Involvement in the -still early- phases of the GS operating model • Supplier has high level of responsibility including major technological choices • Service approach with mix of skills • 2+3 - year contract

• Parameters for a successful bid: • Technical expertise with the data

Format interpretation / meta-data handling Ability to dialogue with Service stakeholders on the archive contents

• IT expertise Data integrity Archiving solutions, phase-in/out

• Management strength Management of data-related processes Management of sub-co’s, ability to work as one Successful experience of overall Service Manager

• Can-do attitude and Customer orientation

Luc Govaert (IPL-POE) DAS INDUSTRY DAY – 1 July 2015

DAS Procurement Features and Schedule


Procurement Features and Schedule

ITT, Contract and Funding – Preference Measure

The procurement is funded under various existing (EOEP-4, Earthnet and LTDP) and future programmes. From 2018 onward, DAS will be funded under the continuation of the current Earthnet and LTDP Programmes, subject to their approval, but already in 2016 there will be a breakpoint. ESA may have to adjust the contractual setup in case of insufficient subscription to EOEP-5 of Participating States in which Consortium Members have their registered office. The ITT will be open to ALL Member States (Earthnet and LTDP are Mandatory Programmes). Industrial Policy Preference Measure: Preference will be given to tenders containing a significant involvement – in terms of quantity and quality – of economic operators registered in the countries listed below, and which will have obtained an overall weighted mark of at least 60, provided their proposal would have been assessed as offering an acceptable balance between technical quality (including risks) and price. Considering the imbalance and trends in the mandatory programmes of the last years, the countries benefitting from this preference measure, for the ITT are: Austria, Belgium, Czech Republic, Denmark, Finland, Greece, Ireland, Luxembourg, Norway, Poland, Portugal, Romania, Spain, Sweden, Switzerland and the United Kingdom.



Who should bid?

Consortium composition: Understanding of Earth Observation data, knowledge of archiving techniques and processes, and related knowledge necessary to ingest, manage and extract the data according to requirements. Efficient utilisation of Information Technology to deploy and maintain a cost-effective infrastructure. Understanding of security and reliability constraints such to comply consistently with the ESA Earth Observation security policy. The Prime Contractor will need to demonstrate a strong ability to manage service processes and coordinate the contributions of the various subcontractors in order to fulfil its overall responsibility. New subcontractors will be allowed during contract execution should specific, e.g., instrument competences be required as a function of new projects/activities injected in the scope of work. Prime and subcontractor shall not be tied in by exclusivity agreements, i.e., one can participate in one or more tenders. Compliance: Compliance to ALL requirements. Deviations to be presented as OPTIONS.



What is expected in the proposal in terms of financial information 1. There will be a list of initial data (initial basket).

2. Each dataset shall be quoted at FFP conditions.

3. The FFP will be the basis for the evaluation of the proposals received.

4. The Bidder (if selected) has to be aware that during the negotiation his financial proposal will be the basis to derive the Cost Model, including the applicable UoCs/FUPs. These UoCs/FUPs - contractual tariffs - will be used during the contract execution for all data.



Contract Characteristics

Initial duration of 2 years + possible three-year extension subject to funding and Contractor performance. Service contract where the Contractor executes ESA-defined processes for the activities in scope under agreed levels of quality and performance (Service Level Agreement). 5MEuro guaranteed over the 5 years. Contractor will set up and use his own infrastructure and deploy all necessary means (except defined CFIs) to perform the service and achieve the results: Full end-to-end responsibility/liability in terms of result (sanctioned by SLA, which can impact Contractor’s remuneration). The service infrastructure will not be owned by ESA, but contractual provisions will allow ESA to purchase any appropriate part of it from the Contractor at remaining Net Book Value (NBV) – to be reported on each quarter – at ESA’s discretion. In order to warrant fair competition, in case the Contract would not be extended beyond the initial two-year phase, ESA will reimburse the investment cost incurred by the Contractor in the initial two-year phase at a price that reflects the remaining the Net Book Value (NBV). Net Book Value is the original acquisition cost less accumulated depreciation or amortization. It is the value at which the assets are valued in the balance sheet of the company as on a given date. Additional tasks and data may be added throughout the contract duration.



The ITT will be subject to the following set of Procurement Rules and Procedures:

1. Procurement conducted by ESA.

2. ESA procurement rules and procedures: Procurement Regulations, with the Right to Review (PART VI Of the Procurement Regulations), and the Tender Evaluation Manual as published on EMITS.

3. General Clauses and Conditions for ESA Contracts, ESA/REG/002,

rev. 1.



Primes and subcontractors must register with ESA as an ESA tenderer:

Industry Portal: http://www.esa.int/esaMI/Industry/index.html Intended ITT will soon be published on EMITS:

http://emits.esa.int/emits/owa/emits.main ITT will be published on EMITS:

in English. Schedule: • ITT out: mid-July • Proposal submission: mid-September • Contract Proposal to IPC • Negotiation: February 2016 • KO: March 2016

http://www.esa.int/esaMI/Industry/index.html









http://emits.esa.int/emits/owa/emits.main











Evaluated by ESA Tender Evaluation Board

– Evaluation criteria.

– Weighting factors will be published.

– The TEB will make a final recommendation for awarding the contract to the tenderer who offer the most economic and effective employment of the Agency’s resources. This shall be the best combination of the total weighted mark and price.

DAS Industry Day - ESAemits.sso.esa.int/.../DAS_Industry_Day-final2.pdf · DAS INDUSTRY DAY – 1...

Documents

Transcript of DAS Industry Day - ESAemits.sso.esa.int/.../DAS_Industry_Day-final2.pdf · DAS INDUSTRY DAY – 1...