Distributed data analysis and survey production networks Liverpool, 14 June 2008
description
Transcript of Distributed data analysis and survey production networks Liverpool, 14 June 2008
![Page 1: Distributed data analysis and survey production networks Liverpool, 14 June 2008](https://reader035.fdocuments.in/reader035/viewer/2022081603/56813b27550346895da3e7d9/html5/thumbnails/1.jpg)
AstroNET / AstroWISE
Distributed data analysis and survey
production networks Liverpool, 14 June 2008
Distributed data analysis and survey
production networks Liverpool, 14 June 2008
Edwin A. ValentijnOmegaCEN
Kapteyn Institute University Groningen
Edwin A. ValentijnOmegaCEN
Kapteyn Institute University Groningen
![Page 2: Distributed data analysis and survey production networks Liverpool, 14 June 2008](https://reader035.fdocuments.in/reader035/viewer/2022081603/56813b27550346895da3e7d9/html5/thumbnails/2.jpg)
What is What is
• an e-science infrastructure with fully distributed resources,
• which allows teams distributed over Europe • to jointly collaborate on the data production,
calibration, QC, analysis of wide field imagers, • in the optical, IR and radio.• Current: [email protected], INT, Subaru, ACS@ST • Near future: Panstarrs, VST, VISTA, Lofar• Far future: EUCLID
• an e-science infrastructure with fully distributed resources,
• which allows teams distributed over Europe • to jointly collaborate on the data production,
calibration, QC, analysis of wide field imagers, • in the optical, IR and radio.• Current: [email protected], INT, Subaru, ACS@ST • Near future: Panstarrs, VST, VISTA, Lofar• Far future: EUCLID
AstroNET / AstroWISE
![Page 3: Distributed data analysis and survey production networks Liverpool, 14 June 2008](https://reader035.fdocuments.in/reader035/viewer/2022081603/56813b27550346895da3e7d9/html5/thumbnails/3.jpg)
–OmegaCAM 24% (~2500 GB)
–VIRCAM 72%–(~7500 GB)
–All Current Paranal–Instruments 4% (433.2 GB)
Paranal Monthly Data Rates 2007 statistics
–Mark Neeser –Mark Neeser
AstroNET / AstroWISE
![Page 4: Distributed data analysis and survey production networks Liverpool, 14 June 2008](https://reader035.fdocuments.in/reader035/viewer/2022081603/56813b27550346895da3e7d9/html5/thumbnails/4.jpg)
UKIDSS workshop 2007
VIKING
VISTA
0.6 sq.deg. InfraRed camera
16 2kx2k detectors
0.35” pixels
KIDS
OmegaCAM @VST
1500 sq.deg. u g r i
2 Million CCD readouts
32 2kx4kdetectors
0.21” pixels
3 – 10 year project
Example Public Survey KIDS – VIKING
~250 nights
~500 nights
![Page 5: Distributed data analysis and survey production networks Liverpool, 14 June 2008](https://reader035.fdocuments.in/reader035/viewer/2022081603/56813b27550346895da3e7d9/html5/thumbnails/5.jpg)
AstroNET / AstroWISE
![Page 6: Distributed data analysis and survey production networks Liverpool, 14 June 2008](https://reader035.fdocuments.in/reader035/viewer/2022081603/56813b27550346895da3e7d9/html5/thumbnails/6.jpg)
![Page 7: Distributed data analysis and survey production networks Liverpool, 14 June 2008](https://reader035.fdocuments.in/reader035/viewer/2022081603/56813b27550346895da3e7d9/html5/thumbnails/7.jpg)
Astro-WISEGrid Computing
Astro-WISEGrid Computing
AstroNET / AstroWISE
![Page 8: Distributed data analysis and survey production networks Liverpool, 14 June 2008](https://reader035.fdocuments.in/reader035/viewer/2022081603/56813b27550346895da3e7d9/html5/thumbnails/8.jpg)
AstroNET / AstroWISE
AstroWISE-Up AstroWISE-Up
![Page 9: Distributed data analysis and survey production networks Liverpool, 14 June 2008](https://reader035.fdocuments.in/reader035/viewer/2022081603/56813b27550346895da3e7d9/html5/thumbnails/9.jpg)
AstroNET / AstroWISE
LOFAR @ 20-80 MHz 120-240 MHz
LOFAR @ 20-80 MHz 120-240 MHz
- EoR
- transients
- Survey
- astroparticles
- EoR
- transients
- Survey
- astroparticles
![Page 10: Distributed data analysis and survey production networks Liverpool, 14 June 2008](https://reader035.fdocuments.in/reader035/viewer/2022081603/56813b27550346895da3e7d9/html5/thumbnails/10.jpg)
AstroNET / AstroWISE
E-Lofar: from experiment to observatoryE-Lofar: from experiment to observatory
![Page 11: Distributed data analysis and survey production networks Liverpool, 14 June 2008](https://reader035.fdocuments.in/reader035/viewer/2022081603/56813b27550346895da3e7d9/html5/thumbnails/11.jpg)
E-LOFAR Participants
E-LOFAR Participants
• GLOW:-- Effelsberg station delivered-- Garching, Potsdam, Tautenburg stations ordered--Jülich station federal funding proposal submitted
• LOFAR-UK:– Chilbolton, Cambridge, Jodrell, Edinburgh– 1 station funded, site to be chosen--3 more stations
• Sweden:– Onsala station funded
• + Poland, Italy, Austria, Bulgaria, Ireland, France
• GLOW:-- Effelsberg station delivered-- Garching, Potsdam, Tautenburg stations ordered--Jülich station federal funding proposal submitted
• LOFAR-UK:– Chilbolton, Cambridge, Jodrell, Edinburgh– 1 station funded, site to be chosen--3 more stations
• Sweden:– Onsala station funded
• + Poland, Italy, Austria, Bulgaria, Ireland, France
AstroNET / AstroWISE
![Page 12: Distributed data analysis and survey production networks Liverpool, 14 June 2008](https://reader035.fdocuments.in/reader035/viewer/2022081603/56813b27550346895da3e7d9/html5/thumbnails/12.jpg)
AstroNET / AstroWISE
Lofar StorageLofar Storage
![Page 13: Distributed data analysis and survey production networks Liverpool, 14 June 2008](https://reader035.fdocuments.in/reader035/viewer/2022081603/56813b27550346895da3e7d9/html5/thumbnails/13.jpg)
AstroNET / AstroWISE
Lofar distributed analysis Lofar distributed analysis
![Page 14: Distributed data analysis and survey production networks Liverpool, 14 June 2008](https://reader035.fdocuments.in/reader035/viewer/2022081603/56813b27550346895da3e7d9/html5/thumbnails/14.jpg)
Astro-WISE: ServicesAstro-WISE: Services
AstroNET / AstroWISE
![Page 15: Distributed data analysis and survey production networks Liverpool, 14 June 2008](https://reader035.fdocuments.in/reader035/viewer/2022081603/56813b27550346895da3e7d9/html5/thumbnails/15.jpg)
Extreme data lineage Extreme data lineage
![Page 16: Distributed data analysis and survey production networks Liverpool, 14 June 2008](https://reader035.fdocuments.in/reader035/viewer/2022081603/56813b27550346895da3e7d9/html5/thumbnails/16.jpg)
AstroNET / AstroWISE
AstroWISEObservatories AstroGrid Euro-VO
EGEE - GRID
AstroWISEObservatories AstroGrid Euro-VO
EGEE - GRID
Distributed data analysis and
survey production networks
Distributed data analysis and
survey production networks