OmegaCAM: The 16k x 16k Survey Camera for the VST Calibration, Data Analysis Strategy and Software...
-
Upload
penelope-mason -
Category
Documents
-
view
225 -
download
0
Transcript of OmegaCAM: The 16k x 16k Survey Camera for the VST Calibration, Data Analysis Strategy and Software...
![Page 1: OmegaCAM: The 16k x 16k Survey Camera for the VST Calibration, Data Analysis Strategy and Software Calibration, Data Analysis Strategy and Software Erik.](https://reader034.fdocuments.in/reader034/viewer/2022051316/56649e895503460f94b8d887/html5/thumbnails/1.jpg)
OmegaCAM: The 16k x 16k Survey Camera for the VST
OmegaCAM: The 16k x 16k Survey Camera for the VST
Calibration, Data Analysis Strategy and Software
Calibration, Data Analysis Strategy and Software
Erik R. DeulKonrad KuijkenEdwin A. Valentijn
Erik R. DeulKonrad KuijkenEdwin A. Valentijn
![Page 2: OmegaCAM: The 16k x 16k Survey Camera for the VST Calibration, Data Analysis Strategy and Software Calibration, Data Analysis Strategy and Software Erik.](https://reader034.fdocuments.in/reader034/viewer/2022051316/56649e895503460f94b8d887/html5/thumbnails/2.jpg)
28/08/2002 SPIE Conference [4836] Survey and Other Telescopes Technologies and Discoveries
2
People involvedPeople involved
• The Netherlands Kapteyn Institute: J-W. Pel, K. Begeman, D.R. Boxhoorn, E. Valentijn, K.
KuijkenSterrewacht Leiden: R. Rengelink, E.R. Deul
• Germany Universitäts-Sternwarte München: R. Bender, L. Greggio, R. Häfner, U. Hopp,
H. Kravkar, W. Mitsch, B. Muschielok, M. Neeser, R. SagliaUniversitäts-Sternwarte Göttingen: R. Harke, H. Nicklas, W. Wellem Sternwarte der Universität Bonn: K. Reif
• ItalyAstronomical Observatory of Capodimonte - Napoli: E. CasconeOsservatorio Astronomico di Padova: A. Baruffolo, E. Cappellaro, E. V. Held,
H. Nazaryan, G. Piotto, H. Navarsadyan, L. Rizzi• ESO
D. Baade, A. Balestra, J-L. Beckers, C. Cavadore, C. Cumani, F. Christen, S. D'Odorico, S. Deiries, N. Devillard, C. Geimer, N. Haddad, G. Hess, J. Hess, O. Iwert, H. Kotzlowski, J-L Lizon, A. Longinotti, W. Nees, A. Renzini, J. Reyes Moreno, G. Sikkema, M. Tarenghi
• The Netherlands Kapteyn Institute: J-W. Pel, K. Begeman, D.R. Boxhoorn, E. Valentijn, K.
KuijkenSterrewacht Leiden: R. Rengelink, E.R. Deul
• Germany Universitäts-Sternwarte München: R. Bender, L. Greggio, R. Häfner, U. Hopp,
H. Kravkar, W. Mitsch, B. Muschielok, M. Neeser, R. SagliaUniversitäts-Sternwarte Göttingen: R. Harke, H. Nicklas, W. Wellem Sternwarte der Universität Bonn: K. Reif
• ItalyAstronomical Observatory of Capodimonte - Napoli: E. CasconeOsservatorio Astronomico di Padova: A. Baruffolo, E. Cappellaro, E. V. Held,
H. Nazaryan, G. Piotto, H. Navarsadyan, L. Rizzi• ESO
D. Baade, A. Balestra, J-L. Beckers, C. Cavadore, C. Cumani, F. Christen, S. D'Odorico, S. Deiries, N. Devillard, C. Geimer, N. Haddad, G. Hess, J. Hess, O. Iwert, H. Kotzlowski, J-L Lizon, A. Longinotti, W. Nees, A. Renzini, J. Reyes Moreno, G. Sikkema, M. Tarenghi
![Page 3: OmegaCAM: The 16k x 16k Survey Camera for the VST Calibration, Data Analysis Strategy and Software Calibration, Data Analysis Strategy and Software Erik.](https://reader034.fdocuments.in/reader034/viewer/2022051316/56649e895503460f94b8d887/html5/thumbnails/3.jpg)
28/08/2002 SPIE Conference [4836] Survey and Other Telescopes Technologies and Discoveries
3
DetectorsDetectors
• Science array 1 x 1 degree, 32 CCDs– 15 m pixels – 0.21 arcsec/pixel– Marconi (former EEV) 2k x 4k– 16k x 16k pixels
• Auxiliary CCD’s – 4 CCDs– For guiding– Image analysis
• Science array 1 x 1 degree, 32 CCDs– 15 m pixels – 0.21 arcsec/pixel– Marconi (former EEV) 2k x 4k– 16k x 16k pixels
• Auxiliary CCD’s – 4 CCDs– For guiding– Image analysis
![Page 4: OmegaCAM: The 16k x 16k Survey Camera for the VST Calibration, Data Analysis Strategy and Software Calibration, Data Analysis Strategy and Software Erik.](https://reader034.fdocuments.in/reader034/viewer/2022051316/56649e895503460f94b8d887/html5/thumbnails/4.jpg)
28/08/2002 SPIE Conference [4836] Survey and Other Telescopes Technologies and Discoveries
4
FiltersFilters
• Primary set– Sloan u’, g’, r’, i’, z’– Johnson B, V– Narrow-band e.g. H up to 8000 km/s– Composite u’,B,V,i’ in four quadrants
• User filter
• Primary set– Sloan u’, g’, r’, i’, z’– Johnson B, V– Narrow-band e.g. H up to 8000 km/s– Composite u’,B,V,i’ in four quadrants
• User filter
More details see Harald Nicklas [4836-34]
More details see Harald Nicklas [4836-34]
![Page 5: OmegaCAM: The 16k x 16k Survey Camera for the VST Calibration, Data Analysis Strategy and Software Calibration, Data Analysis Strategy and Software Erik.](https://reader034.fdocuments.in/reader034/viewer/2022051316/56649e895503460f94b8d887/html5/thumbnails/5.jpg)
28/08/2002 5
VST constructionsee [4836-09]
Mancini
Details instrument control see [4848-10] Baruffolo
VST constructionsee [4836-09]
Mancini
Details instrument control see [4848-10] Baruffolo
![Page 6: OmegaCAM: The 16k x 16k Survey Camera for the VST Calibration, Data Analysis Strategy and Software Calibration, Data Analysis Strategy and Software Erik.](https://reader034.fdocuments.in/reader034/viewer/2022051316/56649e895503460f94b8d887/html5/thumbnails/6.jpg)
28/08/2002 SPIE Conference [4836] Survey and Other Telescopes Technologies and Discoveries
6
Wide Field Imaging ScienceWide Field Imaging Science
• Provide targets for VLT• 2/3 of time through ESO’s OPC• Individual programs
– Supernovae, Lensing, Kuiper belt objects, Gamma ray, bursts, Microlensing, Brown dwarfs, High proper motion objects, Galactic halo objects, Quasars, AGNs
• Sky Surveys• Long term archival research (10 yr mission)
• Science Cases– Finding exceptional single, rare objects– Statistics on large samples of objects
• Provide targets for VLT• 2/3 of time through ESO’s OPC• Individual programs
– Supernovae, Lensing, Kuiper belt objects, Gamma ray, bursts, Microlensing, Brown dwarfs, High proper motion objects, Galactic halo objects, Quasars, AGNs
• Sky Surveys• Long term archival research (10 yr mission)
• Science Cases– Finding exceptional single, rare objects– Statistics on large samples of objects
![Page 7: OmegaCAM: The 16k x 16k Survey Camera for the VST Calibration, Data Analysis Strategy and Software Calibration, Data Analysis Strategy and Software Erik.](https://reader034.fdocuments.in/reader034/viewer/2022051316/56649e895503460f94b8d887/html5/thumbnails/7.jpg)
28/08/2002 SPIE Conference [4836] Survey and Other Telescopes Technologies and Discoveries
7
Large Data VolumeLarge Data Volume
• Wide-field imaging instruments, vast amounts of data– E.g.: VST = Southern sky (30 min exp, 300 nights/y) in
3 years. Large amount of data! 100 Tbyte
• Wide-field imaging instruments, vast amounts of data– E.g.: VST = Southern sky (30 min exp, 300 nights/y) in
3 years. Large amount of data! 100 Tbyte
• Science can only be archive-based• Science can only be archive-based
• Handling of the data is non-trivial– Pipeline data reduction– Calibration and re-calibration– Image comparisons and combinations– Working with source lists– Visualization
• Handling of the data is non-trivial– Pipeline data reduction– Calibration and re-calibration– Image comparisons and combinations– Working with source lists– Visualization
ESOcompliantESOcompliant}}
![Page 8: OmegaCAM: The 16k x 16k Survey Camera for the VST Calibration, Data Analysis Strategy and Software Calibration, Data Analysis Strategy and Software Erik.](https://reader034.fdocuments.in/reader034/viewer/2022051316/56649e895503460f94b8d887/html5/thumbnails/8.jpg)
28/08/2002 SPIE Conference [4836] Survey and Other Telescopes Technologies and Discoveries
8
Concepts for solutionConcepts for solution
• Environment that provides systematic and controlled– Access to all raw and calibration data– Execution and modification reduction/calibration pipelines– Execution of source extraction algorithms– Archiving reduced data and source lists, or regenerates these
dynamically– Can be federated to link different data centers
• Environment that provides systematic and controlled– Access to all raw and calibration data– Execution and modification reduction/calibration pipelines– Execution of source extraction algorithms– Archiving reduced data and source lists, or regenerates these
dynamically– Can be federated to link different data centers
• Dynamical archive continuously grows, can be used for – small or large science projects– generating and checking calibration data– exchanging methods, scripts and configuration
• Dynamical archive continuously grows, can be used for – small or large science projects– generating and checking calibration data– exchanging methods, scripts and configuration
• Key functionality– Link back from source data to the original raw pixel data and
calibration files
• Key functionality– Link back from source data to the original raw pixel data and
calibration files
![Page 9: OmegaCAM: The 16k x 16k Survey Camera for the VST Calibration, Data Analysis Strategy and Software Calibration, Data Analysis Strategy and Software Erik.](https://reader034.fdocuments.in/reader034/viewer/2022051316/56649e895503460f94b8d887/html5/thumbnails/9.jpg)
28/08/2002 SPIE Conference [4836] Survey and Other Telescopes Technologies and Discoveries
9
How to use thisHow to use this
• Deep multi-color fields– No need to take all data in one campaign– Combine data of particular quality, assess results– Select sources, visualize interesting ones, …
• 1-in-1,000,000 events spurious or not?
• Deep multi-color fields– No need to take all data in one campaign– Combine data of particular quality, assess results– Select sources, visualize interesting ones, …
• 1-in-1,000,000 events spurious or not?
• Large homogeneous surveys– E.g. weak lensing maps, cluster searches, star counts
• Large homogeneous surveys– E.g. weak lensing maps, cluster searches, star counts
• Variability (source list - or pixel based) – Proper motions (asteroids, nearby stars)– Flux variations
• Variability (source list - or pixel based) – Proper motions (asteroids, nearby stars)– Flux variations
• Monitor instrument (calibration files)• Monitor instrument (calibration files)
• Planning observations– View quality of existing data– Build on what already exists, add more filters, more
exposure time, better seeing, …
• Planning observations– View quality of existing data– Build on what already exists, add more filters, more
exposure time, better seeing, …
![Page 10: OmegaCAM: The 16k x 16k Survey Camera for the VST Calibration, Data Analysis Strategy and Software Calibration, Data Analysis Strategy and Software Erik.](https://reader034.fdocuments.in/reader034/viewer/2022051316/56649e895503460f94b8d887/html5/thumbnails/10.jpg)
28/08/2002 SPIE Conference [4836] Survey and Other Telescopes Technologies and Discoveries
10
SolutionSolution
• Procedurizing– Data taking at telescope for both science and
calibration data– Full integration with data reduction– Design – Data model (classes) defined for data reduction and
calibration– View pipeline as an administrative problem
• Procedurizing– Data taking at telescope for both science and
calibration data– Full integration with data reduction– Design – Data model (classes) defined for data reduction and
calibration– View pipeline as an administrative problem
![Page 11: OmegaCAM: The 16k x 16k Survey Camera for the VST Calibration, Data Analysis Strategy and Software Calibration, Data Analysis Strategy and Software Erik.](https://reader034.fdocuments.in/reader034/viewer/2022051316/56649e895503460f94b8d887/html5/thumbnails/11.jpg)
28/08/2002 SPIE Conference [4836] Survey and Other Telescopes Technologies and Discoveries
11
Observing ModesObserving Modes
• Dither matching max. gap between arrays ~400 pixels– N pointings (N=5 is standard) – nearly cover all gaps in focal plane and maximizes sky coverage– the context map will be very complex – couple the photometry among individual CCDs.
• Dither matching max. gap between arrays ~400 pixels– N pointings (N=5 is standard) – nearly cover all gaps in focal plane and maximizes sky coverage– the context map will be very complex – couple the photometry among individual CCDs.
• Jitter matching the smallest gaps in CCDs ~5 pixels– optimizes for maximum homogeneity of the context map – observations for which the wide CCD gaps are not critical– all data from single sky pixel originates from single chip
• Jitter matching the smallest gaps in CCDs ~5 pixels– optimizes for maximum homogeneity of the context map – observations for which the wide CCD gaps are not critical– all data from single sky pixel originates from single chip
• Stare reobserving fixed pointing positions multiple times– main workhorse monitoring instrument and optical
transients.
• Stare reobserving fixed pointing positions multiple times– main workhorse monitoring instrument and optical
transients.
• SSO observing Solar System objects– non-siderial tracking and the auto guiding switched off.
• SSO observing Solar System objects– non-siderial tracking and the auto guiding switched off.
![Page 12: OmegaCAM: The 16k x 16k Survey Camera for the VST Calibration, Data Analysis Strategy and Software Calibration, Data Analysis Strategy and Software Erik.](https://reader034.fdocuments.in/reader034/viewer/2022051316/56649e895503460f94b8d887/html5/thumbnails/12.jpg)
28/08/2002 SPIE Conference [4836] Survey and Other Telescopes Technologies and Discoveries
12
Observing StrategiesObserving Strategies
• Standard– Single observations (one observing block)
• Deep– Long, multiple integrations– Selected atmospheric conditions– Several nights
• Frequent– Monitors same field– Timescales from minutes to months (overriding)
• Mosaïc– Maps areas of sky > 1o
• Standard– Single observations (one observing block)
• Deep– Long, multiple integrations– Selected atmospheric conditions– Several nights
• Frequent– Monitors same field– Timescales from minutes to months (overriding)
• Mosaïc– Maps areas of sky > 1o
![Page 13: OmegaCAM: The 16k x 16k Survey Camera for the VST Calibration, Data Analysis Strategy and Software Calibration, Data Analysis Strategy and Software Erik.](https://reader034.fdocuments.in/reader034/viewer/2022051316/56649e895503460f94b8d887/html5/thumbnails/13.jpg)
28/08/2002 SPIE Conference [4836] Survey and Other Telescopes Technologies and Discoveries
13
Calibration proceduresCalibration procedures
Sanity checksSanity checks
Quality controlQuality controlCalibration proceduresCalibration procedures
Image pipelineImage pipeline
Source pipelineSource pipeline
![Page 14: OmegaCAM: The 16k x 16k Survey Camera for the VST Calibration, Data Analysis Strategy and Software Calibration, Data Analysis Strategy and Software Erik.](https://reader034.fdocuments.in/reader034/viewer/2022051316/56649e895503460f94b8d887/html5/thumbnails/14.jpg)
28/08/2002 SPIE Conference [4836] Survey and Other Telescopes Technologies and Discoveries
14
Science ObservationsScience Observations
Photometric pipelinePhotometric pipeline
Bias pipeline
Flatfield pipeline
Image pipeline
Source pipeline
![Page 15: OmegaCAM: The 16k x 16k Survey Camera for the VST Calibration, Data Analysis Strategy and Software Calibration, Data Analysis Strategy and Software Erik.](https://reader034.fdocuments.in/reader034/viewer/2022051316/56649e895503460f94b8d887/html5/thumbnails/15.jpg)
28/08/2002 SPIE Conference [4836] Survey and Other Telescopes Technologies and Discoveries
15
Monitoring Photometric CalibrationMonitoring Photometric Calibration
![Page 16: OmegaCAM: The 16k x 16k Survey Camera for the VST Calibration, Data Analysis Strategy and Software Calibration, Data Analysis Strategy and Software Erik.](https://reader034.fdocuments.in/reader034/viewer/2022051316/56649e895503460f94b8d887/html5/thumbnails/16.jpg)
28/08/2002 SPIE Conference [4836] Survey and Other Telescopes Technologies and Discoveries
16
Share the loadShare the load
• Processing– Hardware
• Beowulf processors – 32 (most cases)• Multi Terabyte disks (10 – 100)
– Data reduction• Derive calibration• Run image pipeline (1 Mpx/s)
• Processing– Hardware
• Beowulf processors – 32 (most cases)• Multi Terabyte disks (10 – 100)
– Data reduction• Derive calibration• Run image pipeline (1 Mpx/s)
• Archiving– Storage
• Images (100’s Tbyte), Calibration files (10 Tbyte)• Source parameters (1-10 Tbyte)
– Federate (network speed)• 5 Mb/s (24 hours/day) full replication • 200 Mb/s no replication, on-the-fly retrieval
![Page 17: OmegaCAM: The 16k x 16k Survey Camera for the VST Calibration, Data Analysis Strategy and Software Calibration, Data Analysis Strategy and Software Erik.](https://reader034.fdocuments.in/reader034/viewer/2022051316/56649e895503460f94b8d887/html5/thumbnails/17.jpg)
28/08/2002 SPIE Conference [4836] Survey and Other Telescopes Technologies and Discoveries
17
Contents of federationContents of federation
• Raw data– Observed images– Ancillary information
• Calibration results– Calibration files time stamped
• Reduced images– Single observation– Coadded images
• Software– Methods (pipelines) for processing calibration– Configuration files
• Source lists – catalogues– Extracted source information– Associated among different data objects
• Raw data– Observed images– Ancillary information
• Calibration results– Calibration files time stamped
• Reduced images– Single observation– Coadded images
• Software– Methods (pipelines) for processing calibration– Configuration files
• Source lists – catalogues– Extracted source information– Associated among different data objects
![Page 18: OmegaCAM: The 16k x 16k Survey Camera for the VST Calibration, Data Analysis Strategy and Software Calibration, Data Analysis Strategy and Software Erik.](https://reader034.fdocuments.in/reader034/viewer/2022051316/56649e895503460f94b8d887/html5/thumbnails/18.jpg)
28/08/2002 SPIE Conference [4836] Survey and Other Telescopes Technologies and Discoveries
18
Concepts of federationConcepts of federation
• Federation maintained by a single database• Full history tracking
– of all input that went into result – providing on-the fly reprocessing
• Dynamical archive - Context as object attributes– Project: Calibration, Science, Survey, Personal– Owner: Pipeline, Developer, User– Strategy: Standard, Deep, Freq (monitoring), Mosaïc– Mode: Stare, Jitter, Dither, SSO– Time: Time stamping
• Software standards– Classes/data model/procedures– 00 – inheritance/ persistency– Python scripts/ c-libraries
• Federation maintained by a single database• Full history tracking
– of all input that went into result – providing on-the fly reprocessing
• Dynamical archive - Context as object attributes– Project: Calibration, Science, Survey, Personal– Owner: Pipeline, Developer, User– Strategy: Standard, Deep, Freq (monitoring), Mosaïc– Mode: Stare, Jitter, Dither, SSO– Time: Time stamping
• Software standards– Classes/data model/procedures– 00 – inheritance/ persistency– Python scripts/ c-libraries
![Page 19: OmegaCAM: The 16k x 16k Survey Camera for the VST Calibration, Data Analysis Strategy and Software Calibration, Data Analysis Strategy and Software Erik.](https://reader034.fdocuments.in/reader034/viewer/2022051316/56649e895503460f94b8d887/html5/thumbnails/19.jpg)
28/08/2002 SPIE Conference [4836] Survey and Other Telescopes Technologies and Discoveries
19
ScheduleSchedule
• Hardware– Dome/Telescope erected at location– Camera on telescope Q1 2004– First run: Jan 2004– Second run: Mar 2004
• Software– Design – review Q2 2002- Done– Basic operations – Q4 2003– Evaluate and prepare for mass production 2004– Qualify and populate 2005– Deliver survey system – satellites
• Hardware– Dome/Telescope erected at location– Camera on telescope Q1 2004– First run: Jan 2004– Second run: Mar 2004
• Software– Design – review Q2 2002- Done– Basic operations – Q4 2003– Evaluate and prepare for mass production 2004– Qualify and populate 2005– Deliver survey system – satellites