GaiaCal2014: Creating and Calibrating LSST Data Product

1 GAIACAL 2014 | RINGBERG, GERMANY | JULY 9, 2014 Name of Mee)ng • Loca)on • Date -‐ Change in Slide Master

Crea%ng and Calibra%ng the Large Synop%c Survey Telescope’s Data Products

Mario Juric LSST Data Management Project Scien5st

GAIACAL2014 July 9th, 2014

Robyn Allsman, Yusra AlSayyad, Tim Axelrod, Jacek Becla, Andrew Becker, Steve Bickerton, Jim Bosch, Bill Chickering, Andy Connolly, Greg Daues, Gregory Dubois-‐Fellsman, Mike Freemon, Andy Hanushevsky, Fabrice Jammes, Lynne Jones, Jeff Kantor,

Kian-‐Tat Lim, Dus5n Lang, Ron Lambert, Robert Lupton (the Good), Simon Krughoff, Serge Monkewitz, Jon Myers, Russell Owen, Steve Pietrowicz, Ray Plante, Paul Price, Andrei Salnikov, Dick Shaw, Schuyler Van Dyk, Daniel Wang featuring Chris Stubbs and the LSST Project Team

2 GAIACAL 2014 | RINGBERG, GERMANY | JULY 9, 2014

Overview

−  LSST overview and status

−  LSST data products: what will be measured and how

−  Instrumental and Astrophysical Calibra)on

−  Dethroning “the catalog”

−  Does soccer need a mercy rule?

3 GaiaCal 2014 • Ringberg, Germany • July 9, 2014

LSST: A Deep, Wide, Fast, Optical Sky Survey

8.4m telescope 18000+ deg2 10mas astrom. r<24.5 (<27.5@10yr)

ugrizy 0.5-‐1% photometry

3.2Gpix camera 30sec exp/4sec rd 15TB/night 37 B objects

Imaging the visible sky, once every ~3 days, for 10 years (825 revisits)


A Dedicated Survey Telescope

−  A wide (half the sky), deep (24.5/27.5 mag), fast (image the sky once every 3 days) survey telescope. Beginning in 2022, it will repeatedly image the sky for 10 years.

−  The LSST is an integrated survey system. The Observatory, Telescope, Camera and Data Management system are all built to support the LSST survey. There’s no PI mode, proposals, or )me.

−  The ul%mate deliverable of LSST is not the telescope, nor the instruments; it is the fully reduced data. •  All science will be come from survey catalogs and images

Telescope è Images è Catalogs


Open Data, Open Source: A Community Resource

−  LSST data, including images and catalogs, will be available with no proprietary period to the astronomical community of the United States, Chile, and Interna%onal Partners

−  Alerts to variable sources (“transient alerts”) will be available world-‐wide within 60 seconds, using standard protocols

−  LSST data processing stack will be free soKware (licensed under the GPL, v3-‐or-‐later)

−  All science will be done by the community (not the Project!), using LSST’s data products


LSST Survey Themes

Time Domain Science

Census of the Solar System

Mapping the Milky Way

Understanding the Nature of Dark MaVer and Dark Energy

and everything in between …


Current Status

−  December 6th, 2013: Passed the NSF Final Design Review; declared ready for Construc<on.

−  January 17th, 2014: FY2014 budget signed, with NSF appropria<on allowing for LSST start.

−  May 8th, 2014: NSB authorizes NSF Director to start the project.

−  Expec5ng the signing of coopera5ve agreement and start of construc5on this month.


Combined Primary/Ter%ary Mirror Thin Meniscus Secondary

−  Primary-‐Ter)ary was cast in the spring of 2008. −  Fabrica)on underway at the Steward Observatory

Mirror Lab -‐ comple)on by the end of 2014.

−  Secondary substrate fabricated by Corning in 2009. −  Currently in storage wai)ng for construc)on.


LSST Camera

Parameter Value

Diameter 1.65 m

Length 3.7 m

Weight 3000 kg

F.P. Diam 634 mm

1.65 m 5’-5”

–  3.2 Gigapixels –  0.2 arcsec pixels –  9.6 square degree FOV –  2 second readout –  6 filters


Bandpasses: u,g,r,i,z,y

R ~ 0.2 spectrograph


LSST Observatory (cca. late ~2018)


HQ Site Science Opera)ons Observatory Management Educa)on and Public Outreach

Archive Site Archive Center

Alert Produc)on Data Release Produc)on

Calibra)on Products Produc)on EPO Infrastructure

Long-‐term Storage (copy 2) Data Access Center

Data Access and User Services

Summit and Base Sites Telescope and Camera

Data Acquisi)on Crosstalk Correc)on

Long-‐term storage (copy 1) Chilean Data Access Center

Dedicated Long Haul Networks

Two redundant 40 Gbit links from La

Serena to Champaign, IL (exis)ng fiber)


LSST Data Products: Images and Catalogs


Why We Create Catalogs?

Model ß inference – Data

And metadata!

Model ß inference – Catalog ß Data Processing – Data

Project Scien<sts Scien<sts

Scien)sts Project Project Project Scien)sts

Computa)onally (and cogni)vely) expensive, science-‐case speciific

Computa)onally cheaper, Easier to understand, Science-‐case speciific

•  Computa)onally expensive, general •  Reprojec)on; may or may not involve

compression •  Almost always introduces some

informa)on loss •  Data Processing == Instrumental

Calibra)on + Measurement


Guiding Principles for LSST Products

−  There are virtually infinite op)ons on what quan))es one can measure on images

−  But if catalog genera)on is understood as a (generalized) cost reduc<on tool, the guiding principles become easier to define

−  Defining principles for the LSST data products: 1.   Maximize science enabled by the catalogs

- Working with images takes )me and resources; a large frac)on of LSST science cases should be enabled by just the catalog.

2.   Minimize informa%on loss -  Provide (as much as possible) es)mates of likelihood surfaces, not just single point es)mators

3.   Provide and document the transforma%on (the soKware)


LSST From the User’s Perspec%ve

−  A stream of ~10 million )me-‐domain events per night, detected and transmiped to event distribu)on networks within 60 seconds of observa)on.

−  A catalog of orbits for ~6 million bodies in the Solar System.

−  A catalog of ~37 billion objects (20B galaxies, 17B stars), ~7 trillion single-‐epoch detec)ons (“sources”), and ~30 trillion forced sources, produced annually, accessible through online databases.

−  Deep co-‐added images.

−  Services and compu)ng resources at the Data Access Centers to enable user-‐specified custom processing and analysis.

−  Soqware and APIs enabling development of analysis codes.

Level 3 Level 1

Level 2


LSST Catalog Contents (Level 2)

−  Object characteriza%on (models): •  Moving Point Source model •  Double Sérsic model (bulge+disk)

-  Maximum likelihood peak -  Samples of the posterior (hundreds)

−  Object characteriza%on (non-‐parametric): •  Centroid: (α, δ), per band •  Adap)ve moments and ellip)city

measures (per band) •  Aperture fluxes and Petrosian and Kron

fluxes and radii (per band) −  Colors:

•  Seeing-‐independent measure of object color

−  Variability sta%s%cs: •  Period, low-‐order light-‐curve moments,

etc.

LSST Science Book, Fig. 9.3


Never measure off the coadds!

−  Impossible to combine mul)-‐epoch data taken in different seeings/exposure )mes without informa)on loss •  Subop)mal S/N

−  Warping and resampling correlates pixel values and noise; correla)on matrices are (prac)cally) impossible to carry forward. •  Source of systema)c error

−  Detector effects have to be taken out at the pixel level •  Further correlates the noise

−  The effec)ve bandpass changes from exposure to exposure. •  Coadding different sorts of apples…

−  Stars move!


Op%mal Mul%-‐Epoch Measurement

Exposure 1

Exposure 2

Exposure 3

Coadd

Galaxy / Star Models

Fiyng Warp,

Convolve

Coadd Measurement

Exposure 1

Exposure 2

Exposure 3

Galaxy / Star

Models

Transformed Model 1

Transformed Model 2

Transformed Model 2

Warp, Convolve

Fiyng

Mul)Fit (Simultaneous Mul)-‐Epoch Fiyng)

Hard, but we only have to do it once.

Easy; rela)vely few data points.

Easier (depends on model), but we have to do it every itera5on!

Same number of parameters, but with orders of magnitude

more data points.


Sampling and retaining the posterior (likelihood)

Perform importance sampling from a proposal distribu<on determined on the coadd. Plan to characterize (and keep!) the full posterior for each object. (Unexplored) possibili5es for compression.


Op%mal Mul%epoch Source Measurements

Op)mal measurement of proper)es of objects imaged in mul)ple epoch. Leq: extrac)on of a moving point source (Lang 2009).

Individual exposures: objects are undetected or marginally detected

Moving point-‐source and galaxy models are indis)nguishable on the coadd


Where Does Calibra%on Come In?

−  With mul)-‐epoch data, (rela)ve) instrument calibra)on becomes inextricably connected with the measurement process.

−  Instrumental: •  Photometric

-  Provide above-‐the-‐atmosphere flux es)mate through a standard passband (rela)ve and absolute).

•  Astrometric -  Provide the posi)on of each object with respect to an external (Gaia) reference

frame.

•  Shapes -  Provide an unbiased es)mate of some measure of the PSF-‐deconvolved shape of

each object.

−  Astrophysical: •  No unique answer, as it depends on (subjec)ve) externally imposed priors. It is

not a data product of the LSST project. •  Short answer (in the Galac)c context): Gaia!


LSST Photometric Calibra%on (In Brief)

−  Goal: es)mate (PSF) flux above the atmosphere through a standard bandpass that does not change.

−  Approach: •  Directly measure the system bandpass (monochroma)c flats) •  Measure and model the atmospheric bandpass

-  A calibra)on telescope will take spectra of standard stars as the observing unfolds. These will be used to fit a nightly, slowly changing, atmosphere model (MODTRAN).

-  Addi)onal measurements of precipitable water vapor will be collected with a GPS system and a co-‐boresighted microwave radiometer. Needed to accurately calibrate the y band.

-  Idea being explored: It is possible that the atmospheric bandpasses could be derived from the imaging data alone.

•  Run self-‐calibra)on (Ubercal) to determine the rela)ve zero-‐points. •  Tie to an external system (DA WD standards or Gaia).


Astrometry: Tree Rings

Above: PRNU (Photo-‐response non-‐uniformity) of an LSST sensor segment. One sees tree rings, sensi)vity varia)ons at at a ~percent level. Due to varying dopant density in silicon boules, which creates parasi)c lateral E fields. The gotcha: these DO NOT behave as QE varia)ons. If you try to flat field this, you make the problem WORSE by ~2x.


Astrometry/Photometry: The “Brighter-‐Fafer PSF” Effect

Most of today’s devices (DES, HSC, LSST, GPC1) are thick. The photon conver)ng at the top has a long way to go to reach the bopom −  Tree rings (and related effects) −  “Brighter-‐fafer” effect

As the poten)al wells fill up with electrons, the bias voltage drops making it easier for electrons to be diverted to neighboring pixels. Correlates the values of neighboring pixels; results in an intensity-‐dependent PSF.

HSC

LSST


Astrophysical Calibra%on: Gaia, the “Great Calibrator”

−  Will provide a network of astrometric standards covering the en)re sky −  Similarly, will provide an internally consistent photometric catalog to aid

calibra)on

−  Scien)fically, it will determine to unprecedented precision a number of astrophysical rela)ons that will directly enable LSST science: •  Directly calibrate color-‐luminosity (photometric parallax) rela)ons for MS and

other stars •  Calibrate period-‐luminosity rela)ons for a wide range of variables

−  Enable the LSST to extend Galac)c census/maps 4-‐7 magnitudes deeper

Eyer, Ivezic & Monet Sec<on 6.12, LSST Science Book hVp://ls.st/sb


Chapter 6: Stellar Populations

Figure 6.26: A comparison of photometric, proper motion and parallax errors for SDSS, Gaia and LSST, as a functionof apparent magnitude r, for a G2V star (we assumed r = G, where G is the Gaia’s broad-band magnitude). Inthe top panel, the curve marked “SDSS” corresponds to a single SDSS observation. The red curves correspond toGaia; the long-dashed curve shows a single transit accuracy, and the dot-dashed curve the end of mission accuracy(assuming 70 transits). The blue curves correspond to LSST; the solid curve shows a single visit accuracy, andthe short-dashed curve shows accuracy for co-added data (assuming 230 visits in the r band). The curve marked“SDSS-POSS” in the middle panel shows accuracy delivered by the proper motion catalog of Munn et al. (2004).In the middle and bottom panels, the long-dashed curves correspond to Gaia, and the solid curves to LSST. Notethat LSST will smoothly extend Gaia’s error vs. magnitude curves four magnitudes fainter. The assumptions usedin these computations are described in the text.

194



194



194

6.12 A Comparison of Gaia and LSST Surveys

Table 6.6: Adopted Gaia and LSST Performance

Quantity Gaia LSST

Sky Coverage whole sky half sky

Mean number of epochs 70 over 5 yrs 1000 over 10 yrs

Mean number of observations 320a over 5 yrs 1000b over 10 yrs

Wavelength Coverage 320–1050 nm ugrizy

Depth per visit (5�, r band) 20 24.5; 27.5c

Bright limit (r band) 6 16-17

Point Spread Function (arcsec) 0.14⇥0.4 0.70 FWHM

Pixel count (Gigapix) 1.0 3.2

Syst. Photometric Err. (mag) 0.001, 0.0005d 0.005, 0.003e

Syst. Parallax Err. (mas) 0.007f 0.40f

Syst. Prop. Mot. Err. (mas/yr) 0.004 0.14a One transit includes the G-band photometry (data collected over 9 CCDs), BP and RP spec-trophotometry, and measurements by the SkyMapper and RVS instruments.b Summed over all six bands (taken at di↵erent times).c For co-added data, assuming 230 visits.d Single transit and the end-of-mission values for the G band (from SkyMapper; integrated BPand RP photometry will be more than about 3 times less precise).e For single visit and co-added observations, respectively.f Astrometric errors depend on source color. The listed values correspond to a G2V star.

195

Gaia and LSST: The Science LSST Science Book: hVp://ls.st/sb

Gaia: hVp://sci.esa.int/gaia


Figure 3.13, LSST Science Book hVp://ls.st/sb

The volume number density (stars/kpc3/mag, log scale

according to legend) of ∼2.8 million SDSS stars with

14 < r < 21.5 and b > 70◦, as a func)on of their distance

modulus (distances range from 100 pc to 25 kpc) and their g − i

color.

The sample is dominated by color-‐selected main sequence

stars.

31 GaiaCal 2014 | Ringberg, Germany | July 9, 2014

LSST: Mapping the Milky Way Halo

RR Lyrae limit

MS stars limit

Dwarf Galaxies


LSST: Extended Mapping of the Milky Way

−  LSST will enable extended mapping of the Milky Way because of a unique combina)on of capabili)es: •  The existence of the u band, allowing the measurement of stellar metallici<es of

near turn-‐off stars and its mapping throughout the observed disk and halo volume.

•  The near-‐IR y band, allowing the mapping of stellar number densi<es and proper mo<ons even in regions of high ex<nc<on.

•  Well sampled <me domain informa<on, allowing for the unambiguous iden<fica<on and characteriza<on of variable stars (e.g., RR Lyrae), facilita<ng their use as density and kinema<c tracers to large distances.

•  Proper mo<on measurements for stars 4 magnitudes fainter than will be obtained by Gaia (see LSST Science Book; § 3.6).

•  The depth and wide-‐area nature of the survey, which combined with the characteris<cs listed above, permits a uniquely uniform, comprehensive, and global view of all luminous Galac<c components.

Juric & Bullock Sec<on 7.2 LSST Science Book hVp://ls.st/sb


Discussion Point: Can We Skip the Catalogs?

−  As our measurements become more and more systema)cs limited, what occurs in the “Data Processing” box above becomes incredibly important.

−  Some)mes, an assump)on or an algorithmic choice that’s been made there may introduce a systema)c that drowns out the signal (or eliminates it). •  Different deblending algorithm (or no deblending) •  Extremely crowded field photometry (e.g., globular clusters) •  Searching for SNe light echos •  Characteriza)on of diffuse structures (e.g., ISM)

−  For op)mal inference, one would always construct a measurement that directly forward-‐models the aspect of the imaging data they’re interested in, and not the catalog. Or derive a more appropriate catalog. Can we do this?





−  Some reasons we don’t do this: 1.  Computa)onally (and I/O !!) intensive 2.  Conceptually difficult

-  Exper)ze in sta)s)cs, applied math, and soqware engineering is oqen not there -  Catalogs are too oqen taken as “$DEITY given”, fundamental, result of a survey

−  Things are changing •  Big data problems are becoming increasingly computa)onally tractable

-  Most of the cost of LSST DM is not in the hardware, it’s in the people wri)ng the soqware. LSST compu)ng hardware in ~2020 == ~$3-‐5M/yr (just ~200 TFLOPS!).

•  The basic, modular, soqware components are being made available by big surveys, lowering the barrier to entry. Average astronomer in the 2020s will grow up with an expecta)on of being well versed in Stats, SE, Appl. Math.





−  LSST “Level 3” concept is the first step in that direc)on: Enabling the community to create new products using LSST’s soqware, services, or compu)ng resources. This means: •  Providing the soKware primi%ves to construct custom

measurement/inference codes •  Enabling the users to run those codes at the LSST data center,

leveraging the investment in I/O (piggyback onto LSST’s data trains).

−  Looking ahead: Right now, we see the data releases as the key product of a survey. By the end of LSST, I wouldn’t be surprised if we saw the soKware as the key product, with hundreds specialized (and likely ephemeral) catalogs being generated by it.

−  The “data releases” will just be some of those catalogs, designed to be more broadly useful than others, and retained for a longer period of )me.

−  LSST soKware soKware and hardware is being engineered to make this possible.


Summary

−  LSST will provide photometric (ugrizy) and shape characteriza)on for ~40B objects (to r=27.5) over ~half the sky (southern hemisphere), with ~800-‐1000 epochs per object (to r=24.5).

−  LSST products will consist of images, catalogs, and the soqware used to produce them. We try to minimize informa)on loss by retaining the informa)on about the likelihoods (extended sources only (for now)). We will report above-‐the-‐atmosphere fluxes calibrated to a standard band.

−  Astrophysical calibra)on of LSST is not a formal part of the project; the community (Science Collabora)ons) is beginning to think about it. In the Galac)c context, Gaia will provide crucial calibra)ons for LSST.

−  Going forward, we expect that enabling user-‐generated soqware (“Level 3”) will be increasingly important.

GaiaCal2014: Creating and Calibrating LSST Data Product

Science

Transcript of GaiaCal2014: Creating and Calibrating LSST Data Product