Integrated Data Cycle Systems Harvey E. Rhody Chester F. Carlson Center for Imaging Science.
-
Upload
marsha-kennedy -
Category
Documents
-
view
219 -
download
2
Transcript of Integrated Data Cycle Systems Harvey E. Rhody Chester F. Carlson Center for Imaging Science.
![Page 1: Integrated Data Cycle Systems Harvey E. Rhody Chester F. Carlson Center for Imaging Science.](https://reader033.fdocuments.in/reader033/viewer/2022051516/56649ef15503460f94c02830/html5/thumbnails/1.jpg)
Integrated Data Cycle Systems
Harvey E. Rhody
Chester F. Carlson Center for Imaging Science
![Page 2: Integrated Data Cycle Systems Harvey E. Rhody Chester F. Carlson Center for Imaging Science.](https://reader033.fdocuments.in/reader033/viewer/2022051516/56649ef15503460f94c02830/html5/thumbnails/2.jpg)
The Problem
An expensive facility with many customers
The customers have different needs
Expert knowledge is needed to operate the facility
Operate the facility at low cost with good service
![Page 3: Integrated Data Cycle Systems Harvey E. Rhody Chester F. Carlson Center for Imaging Science.](https://reader033.fdocuments.in/reader033/viewer/2022051516/56649ef15503460f94c02830/html5/thumbnails/3.jpg)
The Need
Increased Productivity
Fast Response
Continuous Improvement
![Page 4: Integrated Data Cycle Systems Harvey E. Rhody Chester F. Carlson Center for Imaging Science.](https://reader033.fdocuments.in/reader033/viewer/2022051516/56649ef15503460f94c02830/html5/thumbnails/4.jpg)
Required Functions
Job Specification
Job Planning
Job Execution
Delivery of Results
Archive
![Page 5: Integrated Data Cycle Systems Harvey E. Rhody Chester F. Carlson Center for Imaging Science.](https://reader033.fdocuments.in/reader033/viewer/2022051516/56649ef15503460f94c02830/html5/thumbnails/5.jpg)
Kinds of Knowledge
• Technical knowledge
• Systems knowledge
• Project management knowledge
• Business knowledge
All are necessary for a successful high-value project
![Page 6: Integrated Data Cycle Systems Harvey E. Rhody Chester F. Carlson Center for Imaging Science.](https://reader033.fdocuments.in/reader033/viewer/2022051516/56649ef15503460f94c02830/html5/thumbnails/6.jpg)
Astronomical Observatories
Complex and Expensive System
Remote Location
Shared Resource
Leading-edge Technology
Long System Life
![Page 7: Integrated Data Cycle Systems Harvey E. Rhody Chester F. Carlson Center for Imaging Science.](https://reader033.fdocuments.in/reader033/viewer/2022051516/56649ef15503460f94c02830/html5/thumbnails/7.jpg)
Spirex Telescope with Abu Camera in Antarctica
![Page 8: Integrated Data Cycle Systems Harvey E. Rhody Chester F. Carlson Center for Imaging Science.](https://reader033.fdocuments.in/reader033/viewer/2022051516/56649ef15503460f94c02830/html5/thumbnails/8.jpg)
![Page 9: Integrated Data Cycle Systems Harvey E. Rhody Chester F. Carlson Center for Imaging Science.](https://reader033.fdocuments.in/reader033/viewer/2022051516/56649ef15503460f94c02830/html5/thumbnails/9.jpg)
Winterover on arrival at the pole….
![Page 10: Integrated Data Cycle Systems Harvey E. Rhody Chester F. Carlson Center for Imaging Science.](https://reader033.fdocuments.in/reader033/viewer/2022051516/56649ef15503460f94c02830/html5/thumbnails/10.jpg)
Six Months Later….
![Page 11: Integrated Data Cycle Systems Harvey E. Rhody Chester F. Carlson Center for Imaging Science.](https://reader033.fdocuments.in/reader033/viewer/2022051516/56649ef15503460f94c02830/html5/thumbnails/11.jpg)
Abu Pipeline
Astronomer
ScienceCenter
AnalysisCenter
DataArchive
ObservationCenter
NOAO/RIT
SouthPole
White Sands/RIT
RIT
![Page 12: Integrated Data Cycle Systems Harvey E. Rhody Chester F. Carlson Center for Imaging Science.](https://reader033.fdocuments.in/reader033/viewer/2022051516/56649ef15503460f94c02830/html5/thumbnails/12.jpg)
IDL, C,...XMLFITS, TIFF, ...
Pipeline
Database
ImageData
CAMLDocuments
Algorithms
CAMLController
Java, ORB
OperatorInterface
NetworkInterface
![Page 13: Integrated Data Cycle Systems Harvey E. Rhody Chester F. Carlson Center for Imaging Science.](https://reader033.fdocuments.in/reader033/viewer/2022051516/56649ef15503460f94c02830/html5/thumbnails/13.jpg)
Large Magellanic Cloud
30 Doradus nebula
![Page 14: Integrated Data Cycle Systems Harvey E. Rhody Chester F. Carlson Center for Imaging Science.](https://reader033.fdocuments.in/reader033/viewer/2022051516/56649ef15503460f94c02830/html5/thumbnails/14.jpg)
Star formation
NGC 6334
Abu/Spirex 1998
![Page 15: Integrated Data Cycle Systems Harvey E. Rhody Chester F. Carlson Center for Imaging Science.](https://reader033.fdocuments.in/reader033/viewer/2022051516/56649ef15503460f94c02830/html5/thumbnails/15.jpg)
![Page 16: Integrated Data Cycle Systems Harvey E. Rhody Chester F. Carlson Center for Imaging Science.](https://reader033.fdocuments.in/reader033/viewer/2022051516/56649ef15503460f94c02830/html5/thumbnails/16.jpg)
![Page 17: Integrated Data Cycle Systems Harvey E. Rhody Chester F. Carlson Center for Imaging Science.](https://reader033.fdocuments.in/reader033/viewer/2022051516/56649ef15503460f94c02830/html5/thumbnails/17.jpg)
Productivity Challenge
960 observing hours per year
160 observing flights per year
Variety of instruments
Large community of observers
Conclusion: The traditional “PI” approach is unworkable. A new system for efficiency and relentless observation is needed.
![Page 18: Integrated Data Cycle Systems Harvey E. Rhody Chester F. Carlson Center for Imaging Science.](https://reader033.fdocuments.in/reader033/viewer/2022051516/56649ef15503460f94c02830/html5/thumbnails/18.jpg)
The DCS Mission
The DCS shall provide the infrastructure to support the operation of facility instruments on SOFIA using best current knowledge and tools, and supporting continuous improvement in an efficient, extensible, and modular architecture.
![Page 19: Integrated Data Cycle Systems Harvey E. Rhody Chester F. Carlson Center for Imaging Science.](https://reader033.fdocuments.in/reader033/viewer/2022051516/56649ef15503460f94c02830/html5/thumbnails/19.jpg)
SOFIA Facility Instruments
Instrument Institution Characteristics
AIRES NASA-Ames Echelle Spectrometer17 to 210 micronsR=10,000
HAWC U. ChicagoRITNASA-Goddard
Far Infrared BolometerCamera40-300 microns
FORCAST Cornell Mid IR Camera5-40 microns
FLITECAM UCLA Near IR Test Camera1-5 microns
![Page 20: Integrated Data Cycle Systems Harvey E. Rhody Chester F. Carlson Center for Imaging Science.](https://reader033.fdocuments.in/reader033/viewer/2022051516/56649ef15503460f94c02830/html5/thumbnails/20.jpg)
SOFIA Team Players and Strengths
IPAC
IRAS, ISO, 2Mass, WIRE, SIRTF
Archiving, Observation planning
ARC
Smart scheduling activities for instruments and telescopes for HST, Mars Rover, Deep Space 1
GSFC
HST and NGST development
HAWC and SAFIRE instrument architecture
Object Oriented development
RIT
South Pole Data Pipelining
HAWC development
UCLA
Keck
FLITECAM
![Page 21: Integrated Data Cycle Systems Harvey E. Rhody Chester F. Carlson Center for Imaging Science.](https://reader033.fdocuments.in/reader033/viewer/2022051516/56649ef15503460f94c02830/html5/thumbnails/21.jpg)
SOFIA Data CycleD ata
M is s io nP lan
O b s erver'sP ro p o s al &Exp erim ent
S im ulatio n
Q uic kLo o k
A nalyzedD ata
A rc hive
O b s erver'sP lan
PR
DataCo l l e c t i o n
P l an
P ub lic atio n
A nalys isP lan
![Page 22: Integrated Data Cycle Systems Harvey E. Rhody Chester F. Carlson Center for Imaging Science.](https://reader033.fdocuments.in/reader033/viewer/2022051516/56649ef15503460f94c02830/html5/thumbnails/22.jpg)
Science Center Activities
Community Development and Support
Maintain Science Data Archives
General Investigator Support
Maintain Center Information Infrastructure System Documentation , Observatory Performance Information, Observation Constraints
Calibration Information, Atmospheric Models
Maintain FI Tools Facility Software Management, Proposal Development Tools, Proposal Evaluation Tools
Observation Planning Tools , Observing Tools, Flight Planning Tools
Data Reduction Tools -- Pipeline analysis modules
Coordinate with Observatory Operations
![Page 23: Integrated Data Cycle Systems Harvey E. Rhody Chester F. Carlson Center for Imaging Science.](https://reader033.fdocuments.in/reader033/viewer/2022051516/56649ef15503460f94c02830/html5/thumbnails/23.jpg)
User Needs
• Archival Research • Get Information on Observatory Capabilities • Get Information on Instrument Capabilities • Build and submit a Proposal • Build an Observation Plan • Build a Data Reduction Pipeline • Access Observation Results • Interpret Experimental Results • Publish
![Page 24: Integrated Data Cycle Systems Harvey E. Rhody Chester F. Carlson Center for Imaging Science.](https://reader033.fdocuments.in/reader033/viewer/2022051516/56649ef15503460f94c02830/html5/thumbnails/24.jpg)
User Access Requirements
User access to all user functions shall be supported via the Web
1.Access to archived science products 2.Access to supported external archives 3.Access to observatory capabilities information 4.Access to instrument capabilities information 5.Proposal development, submission and tracking 6.Observation timing estimation 7.Atmospheric modeling resources 8.Access to observation data 9.Access to flight and housekeeping data 10.Access to flight logs 11.Data processing 12.Published document archiving
![Page 25: Integrated Data Cycle Systems Harvey E. Rhody Chester F. Carlson Center for Imaging Science.](https://reader033.fdocuments.in/reader033/viewer/2022051516/56649ef15503460f94c02830/html5/thumbnails/25.jpg)
System Requirements
User Access
Administrator Access
Data Storage and Retrieval
Data Processing
Planning and Scheduling
Documentation Support
![Page 26: Integrated Data Cycle Systems Harvey E. Rhody Chester F. Carlson Center for Imaging Science.](https://reader033.fdocuments.in/reader033/viewer/2022051516/56649ef15503460f94c02830/html5/thumbnails/26.jpg)
Data Reduction Resources
FSI & MCS
Interfaces
Task Library
DCS Data Storage
User Interaction
Layer
prefs, context
requests, results
run an experiment
raw and processed
data
AOTs, exp info, raw data, obs logs..
various searches
run a pipelineSSMOC
Internet
![Page 27: Integrated Data Cycle Systems Harvey E. Rhody Chester F. Carlson Center for Imaging Science.](https://reader033.fdocuments.in/reader033/viewer/2022051516/56649ef15503460f94c02830/html5/thumbnails/27.jpg)
DCS Data Storage
DCS DataStorageInterface
DB LocatorAnd Selector
User DB
Accounts Groups
Privileges Contexts
Preferences
Experiment DB
Templates Planned Accepted
Completed Reviews
Comments
Data DB
Raw Instr DataOBS Logs
Processed Data
Instr DB
GSFC’s IMLFSI Chain ListsHistory of Instr
Use
Provides common DCS interface to all storage,
translates DCS data access to standard DB
interface methods Convenient switch for criteria such as: load
balancing, fault tolerance
![Page 28: Integrated Data Cycle Systems Harvey E. Rhody Chester F. Carlson Center for Imaging Science.](https://reader033.fdocuments.in/reader033/viewer/2022051516/56649ef15503460f94c02830/html5/thumbnails/28.jpg)
The RIT Astronomy Pipeline Team
Harvey Rhody, Scott Lawrence, Erika Tolar, Stephen Schaeffer, James DeFelice, Adith Chandrasekhar, J. Fernando Naveda, Bob Krzaczek (standing) and Ian Gatley. Not shown: Jim Chauvin, Brian Perry, Joel Kastner.