Towards a demonstrator for autonomous object detection on board Gaia From stars to silicon... and...

Towards a demonstrator for autonomousobject detection on board Gaia

From stars to silicon... and back

Shan Mignot

Outline

I. Gaia

user requirements

II. On-board processing

constraints and technologies

III. Image analysis

methods and implications

algorithmic framework

IV. Elements on the demonstrator

some solutions to some problems

V. Synthesis

scientific and technical assessment

PhD defense – January 10th 2008 – Shan Mignot 2/49

I. Gaia

Timeline

ESA cornerstone mission (“cosmic vision” program)

proposal: L. Lindegren, M.A.C Perryman, and S. Loiseau (1995)Global Astrometric Interferometer for

Astrophysics (GAIA)

phase A: mission & system definition (2001-2006)

“Payload Data Handling” working group

“Payload Data Handling Electronics” technical definition activity

phase B2: definition (2006-2007)

support to Astrium (PDH-S contract)

phases C & D: development phases (2007-2011)

launch in December 2011

phases E & F: operational phase (2011-2016 +1)

final products in 2020

Gaia


Scientific return

structure & evolution of the Galaxy

solar system (satellites, minor planets etc.)

large catalogue of objects (stars, quasars, extra-solar planets etc.)

fundamental physics (general relativity, gravitational waves etc.)

Dataset

phase-space map of the Galaxy (astrometry)

positions: right ascension, declination & parallax

velocities: proper motion & radial velocity

astrophysical database

multi-band & multi-epoch photometry

intermediate resolution spectroscopy

→ very precise → very numerous → highly homogeneous

Gaia

Science case


HiPParCoS´s legacy

Global astrometry

scanning principle: spin & precession motions

→ on-the-fly data acquisition

→ cross-matching between transits

2 lines of sight: 2 telescopes (combined focal planes)

Gaia


Beyond HiPParCoS

Magnitude-limited survey

no catalogue → on-board object detection

~50 TB of data → reliability

14 magnitudes → flux ratio up to 1/400 000

stellar densities → max/mean > 120

Detectors

106 CCDs

→ full read-out: 403 MB/s (total)

→ digital image processing: detect

Orbit around L2 (1.5 106 km)

dynamical & thermal stability

→ attitude & orbit control

limited bandwidth to earth & visibility

→ autonomous data management: characterise

Gaia


Payload IG

aia


Payload IIG

aia

dete

ctio

n

con

firm

ati

on

observation

propagation

dete

ctio

n


Time-delay integration (TDI)G

aia

4 phase charge transfer

column per column read-out(every 0.9828 ms)

pixel packets sent toVideo Processing Unit

(SpaceWire serial interface)

4.4s


Stars ?G

aia

G = 2

G = 6

G = 10

G = 20G = 15


Needs

Individual objects

galactic physics → all densities

→ high background

extra-galactic physics → unresolved galaxies

→ resolved stars→ high densities

stellar physics → star types & colour

→ multiple stars→ variable stars

rare events (supernovae, micro-lensing) → reliability

solar system (minor planets, satellites) → resolved objects

→ moving objects

Collectively

statistical quality of the catalogue → selection function

global iterative solution → predictability

reference frame → coherence

Gaia


double stars

density

Some cases of interestG

aia

prompt particle events

extra-galactic

complex skies


RequirementsG

aia

Selection function

performance → 98% detection probability→ sensitivity to

object types→ increase signal to noise ratio

→ estimate the total noise (sky background)

homogeneity → calibrate CCD data & correct defects→ stationary (along

scan) → uniform (across scan)

graceful degradation → priority-driven detection→ signal

undetected objects

Processing & resources

→ enforce limiting magnitudes

→ avoid false detections

→ filter prompt particle events

→ adapt imaging properties (sampling)


II. On-board processing

Dependability & reliability

part selection & testing

operational modes / reconfiguration

redundancies

Survival & operation

mechanical → withstand launch

temperature → range (-55oC to 125 oC)→ cycling

→ dissipation

vacuum → outgassing (contamination)

electromagnetic: → power supply→

compatibility→ electrostatic discharges

→ radiation

Electronics in spaceO

n-b

oard

pro

cess

ing


Radiative environmentO

n-b

oard

pro

cess

ing

Photons

Sun & Earth (albedo & black body)

→ thermal stability: sun shield, no eclipse, constant orientation

→ attitude: radiation pressure

Solar wind plasma

interplanetary orbit

protons & α particle (< 1 keV)

→ electrostatic discharges

→ erosion of covering

High energy particles

solar protons (1 keV to GeV)

cosmic rays (1 keV to GeV)

secondary particles


num

ber

ener

gy

Radiation effectsO

n-b

oard

pro

cess

ing

Processes

electron / positron pairs

ionisation

displacement damage

electrostatic discharges

Impact

single event upset

material degradation

latch-up

Cause Anomalyfrequency (%)

RadiationPlasmasThermalDebrisOther

512710 7 5


MicroelectronicsO

n-b

oard

pro

cess

ing

Radiation

tolerance → fineness of engraving→ frequency of

operation

hardness → triple module redundancy→ error detection and

correction

Performance

space → availability delay (adapted from commercial)→ degraded density & speed

commercial parts → procurement problem→ design

complexity (TMR & EDAC)Processor Data Address Performances MissionERC32LeonIIRCA 1802HM6100RAD 600RAD 750IMA31750

3232 812323216

32321612323216

20 MIPS @ 25 MHz25 MIPS @ 25 MHzno minimum frequencyno minimum frequency35 MIPS @ 33 MHz< 300 MIPS @ 166 MHz1 MIPS at 8 MHz

Cryosat, Pleiades

Voyager (3), Viking, GalileoHipparcosSpirit, OpportunityMars reconnaissance orbiterProtéus


Mixed architectureO

n-b

oard

pro

cess

ing

PDHE TDA

multiplicity of tasks → context switches

data intensive → cache misses & slow memory bus

interface with CCDs → 50% of TDI

mixed architecture recommended :

→ CPU (software) + FPGA / ASIC (dedicated hardware)

FPGA / ASIC

“custom processor”

→ library of elementary logical blocks

→ interconnexions

development (codesign)

→ bit-level

→ power

→ timing

performances

→ efficient for control

→ inefficient for arithmetics


III. Image analysis

RationaleIm

ag

e a

naly

sis

Principle

locate the objects of interest

characterise them

Generic approach

locate all objects with the same logic

→ simplify verification / validation

→ avoid problems at the interface

→ save resources (single process)

Hardware / software partition

detection is compression: exhaustive pixel list → abstract description

real-time constraints & complexity

→ pixels are regular & simple: hardware

→ objects are random & complex: software


Transforms

global view

concentrate the information: 2 steps in one

→ requires a priori information vs. variability of imaging:object types (extended, colour, brightness) heterogeneities (smearing,

aberrations)

→ complex analysis of transform space

linear transforms are convolutions

→ systematic

→ arithmetically intensive: adders & multipliers

→ in AC only (data access vs. TDI read-out)

Approaches IIm

ag

e a

naly

sis


Approaches IIIm

ag

e a

naly

sis

Local analysis

detect & characterise within a neighbourhood of predefined shape

→ limited & systematic data accesses (fixed pattern)

limited information in the neighborhood:

discriminate: artefacts related to noise & PSFparticles

identify: saturated stars (degeneracy)extended

objects (different patterns)compound objects (multiple stars & density)

→ need for a selective yet flexible generic approach:

difficult type I (false positive) & type II (false negative) trade-off

SWA working window


Approaches IIIIm

ag

e a

naly

sis

Segmentation

region-growing method

attribute each pixel to background or objects

→ based on total noise estimate→ discard background ( ≥ 90% pixels)

identify objects in binary mask

→ compatible with raster order (TDI read-out)

characterisation

→ flexible geometry: rich content

→ independent for each domain

→ priority-driven

hardware / software partition

real-time data flow computing platform

pixel-based hard 3.8 MB/s systematic & control hardware

object-based soft < 10 % adaptive & arithmetics software


Functional architectureIm

ag

e a

naly

sis

raw data thresholdedbackground map

deblended connectedcomponents

measurements

pixel level

object level


SamplingIm

ag

e a

naly

sis

Needs

ensure completeness

→ increase signal to noise ratio

save resources

→ decrease resolution

Method

hardware pixel binning (at CCD level): 2x2-pixel “samples”

Consequences

data flow reduction: 3.8 MB/s → 0.95 MB/s

real-time: 2 TDIs per column

→ same hardware/software for the two detection CCDs

precision: OK for science, 4x more objects for attitude control


Pre-calibrationIm

ag

e a

naly

sis

Needs

selection function (intra & inter CCDs)

→ stationary detection probabilities

→ accurate measurements

control false detections

graceful degradation in time

Functional

calibration: pixel response & dark & offset

cosmetic defects: “black” & “white” pixels

Method

linear transform (generalises flat-field & dark)

fixed-point arithmetics

replacement mechanism

VIMOS CCD


Background IIm

ag

e a

naly

sis

Needs

estimate the total noise (including the sky background)

→ stationary detection probabilities

→ accurate measurements


Functional

latency & resolution trade-off

adapted to pixel statistics

robust to stellar content

systematic calculation

Method

regional estimates: hyperpixels

histograms: 4 ADU bins

interpolated mode: precise & robust

2D bilinear interpolation

fixed point arithmetics


Background IIIm

ag

e a

naly

sis

hyperpixel

mode values

interpolation


Background IIIIm

ag

e a

naly

sis


Pixel selectionIm

ag

e a

naly

sis

Needs

save resources

→ discard background pixels


→ robustness to noise

→ filter faint stars

Functional

signal to noise threshold

Method

signal: subtract background (bkgd)

noise: Poisson noise (pix) & read-out noise (σRON

)

fixed-point arithmetics

pix−bkgd

pixσ RON2? SNR pix−bkgd 2? SNR 2× pix σ RON

2


Simple object model IIm

ag

e a

naly

sis

Needs

transition from hardware to software

priority-driven characterisation

Functional

form object data units

insert in priority-ordered interface with software

Method

connected-component labelling

→ compatible with raster-order: label / merge / relabel

simple descriptors (flux, background flux, number of pixels)


Simple object model IIIm

ag

e a

naly

sis

object 2

object 1

Label Merge Relabel Extract


CharacterisationIm

ag

e a

naly

sis

Needs

filter unwanted object

refine object model

measure objects

Functional

save resources: software-optimised

filter prompt particle events: cascade of descriptors

enforce limiting magnitudes

identify components in compound objects

Method

adaptive sequence (decision tree)

object-wise SNR test, energy density, number of interior pixels etc.

watershed-based component segmentation

compute flux & barycentre


IV. Elements on the demonstrator

Software modelD

em

on

stra

tor

Uses

R&D → performance evaluation

software part → characterisation engine

reference model → fixed-point arithmetics→ data accesses

→ PDHE TDA (ANSI C)→ hardware developments


ArchitectureD

em

on

stra

tor

Interfaces

input: CCDs via serial SpaceWire link

→ video reception buffer: PC with IO boardno output

→ intermediate datastorage

→ 2 SRAMs

Simplified

FPGA board developments

→ no real-time processordesign simplifications

→ no connected component labelling→ no management of software interface→ free pins

inspectable design

→ output data stored in PC


PlatformD

em

on

stra

tor

Part

Actel: ProASIC3E instead of RTAX-S

→ reprogrammable: flash-based instead of antifuse→ slower (interconnections)→ less dense

Starter Kit

ProASIC3E 600

SRAMs

ISSI ISI61LV51216

static

asynchronous

16-bit data

19-bit address

PC interface

handshake

16-bit data


ProcessingD

em

on

stra

tor

Sequential

Parallel

Pipeline

1 10 1 11 0

01

0100


DesignD

em

on

stra

tor

Pipeline

pre-calibration → background → pixel selection

each is a pipeline

Clocks

DCLK: data clock → pipeline control (~1 MHz)

SCLK: SRAM clock → sequential optimisations (~32 MHz)

CLK: main clock → SRAM interface (125 MHz)

Design for test

processing core & debug core

conditional instantiation: inspectable, piece-wise verification


Pixel selectionD

em

on

stra

tor

Tasks

background interpolation → on demand

signal to noise threshold

output stream of selected pixels

PipelineDCLK

mode addresses (4)AL & AC coordinates

interpolation coefficientsread modes (4)

contributions (4)

sum

signal

signal2

threshold

testoutput

pix−bkgd 2? SNR 2× pix σ RON2


V. Synthesis

DemonstratorS

yn

thesi

s

VHDL

ESA standard (except for testing: verification & validation)

Simulation

validated pre-synthesis & post-synthesis

Synthesis

14722 cells > ProASIC3E 600 → ProASIC3E 1500 (100% margin)

slow routing

target: RTAX-S 1000 (ITAR)


Science aimsS

yn

thesi

s

Performances

→ realistic scenes to verify processing

completeness

location

magnitude

false detections


Density(stars/deg2 )

Test case False detection rate(per 10000 pixels)

3 310 000 609 000 195 000 22 250 3 500

Baade l54b0 l74b0 l74b15 l74b74

0.889 0.092 0.019 0.002 0.005

GaiaS

yn

thesi

s

OBDH working group

Payload Data Handling Support

phase A: PDHE

→ algorithms→ port to real-time software platform→ analysis

phase B: support to prime

→ algorithmic→ specification reviews→ validation

ALGOL ACI

contribution to highly constrained embedded computing

Publications

23 technical notes: detection & other on-board processing

1 conference proceedings


PerspectivesS

yn

thesi

s

Demonstrator

existing → place & route

→ validate interfaces→ verification

→ validation campaign

extension → second generation

→ complete architecture→ software interface

→ real-time software engine

Collaborations

experts in electronics

experts in embedded applications

experts in spacePhD defense – January 10th 2008 – Shan Mignot 47/49

If it looks easy, you do not understand it.M.A.C. Perryman, 2001.

Acknowledgements

Pyxis

F. Chéreau, C. Macabiau, J. Chaussard, F. Arenou

based on APM, Sextractor, the watershed transform

Simulations

F. Chéreau, F. Arenou, C. Babusiaux

Demonstrator

P. Laporte, F. Rigaud

ALGOL ACI

PRiSM, LaMI

Ack

now

led

gem

en

ts


Image credits

EADS Astrium SAS

Data Processing and Analysis Consortium (DPAC)

Gaia Image and Basic Instrument Simulator (GIBIS)

European Space Agency (ESA)

National Aeronautics & Space Administration (NASA)

European Southern Observatory (ESO)

A. Short

E. Oseret

F. Rigaud

internet: N. Giffin, A. Bloom, answers.com, lipcoop.com

Imag

e c

red

its


Towards a demonstrator for autonomous object detection on board Gaia From stars to silicon... and...

Documents

Transcript of Towards a demonstrator for autonomous object detection on board Gaia From stars to silicon... and...