Archiving, Distribution and Analysis of Solar-B Data Masumi Shimojo SOLAR-B Project NSRO/NAOJ and The SOLAR-B MODA Working Group Masumi Shimojo SOLAR-B Project NSRO/NAOJ and The SOLAR-B MODA Working Group 6 th SOLAR-B Science Meeting Kyoto, Japan 2005/11/086th Kyoto2 Topics Data Flow from SOLAR-B to Scientists Data Format of Level-0 of SOLAR-B SOLAR-B DB System for Scientists This talk is based on the MODA document The outline of SOLAR-B Mission Operation and Data Analysis. Data Flow from SOLAR-B to Scientists Data Format of Level-0 of SOLAR-B SOLAR-B DB System for Scientists This talk is based on the MODA document The outline of SOLAR-B Mission Operation and Data Analysis. 2005/11/086th Kyoto3 Data Flow from SOLAR-B to Scientists: 1 In nominal operations, all scientific data of SOLAR-B are downloaded at Uchinoura (Japan) and Svalvard (Norway). All data are transferred from USC and Svalvard to the SIRIUS database of JAXA/ISAS using Internet. We predict that it takes about one week till we get the complete set of SOLAR-B data from SIRIUS database. In order to check the status of instruments and make the operation plan, the operation team uses the data from Uchinoura. In nominal operations, all scientific data of SOLAR-B are downloaded at Uchinoura (Japan) and Svalvard (Norway). All data are transferred from USC and Svalvard to the SIRIUS database of JAXA/ISAS using Internet. We predict that it takes about one week till we get the complete set of SOLAR-B data from SIRIUS database. In order to check the status of instruments and make the operation plan, the operation team uses the data from Uchinoura. Japan SIRIUS Operation Team for QL 2005/11/086th Kyoto4 Data Flow from SOLAR-B to Scientists: 2 All scientific data are reformatted to Level-0 FITS files by each instrument team at JAXA/ISAS. Level-0 means that the data are not calibrated. All FITS files of SOLAR-B are archived in DARTS (the Data ARchive and Transmission System) of ISAS/JAXA. Scientists can get the SOLAR-B data using the ftp server of DARTS. All scientific data are reformatted to Level-0 FITS files by each instrument team at JAXA/ISAS. Level-0 means that the data are not calibrated. All FITS files of SOLAR-B are archived in DARTS (the Data ARchive and Transmission System) of ISAS/JAXA. Scientists can get the SOLAR-B data using the ftp server of DARTS. SIRIUS DB SOT Reformatter XRT Reformatter EIS Reformatter SOT FITS files XRT FITSfiles EIS FITS files DARTS FTP Server Scientists 2005/11/086th Kyoto5 Data Flow from SOLAR-B to Scientists: 3 The SOLAR-B FITS files are mirrored by the SOLAR-B project team around the world. Scientists may get the SOLAR-B FITS files from these sites. The SOLAR-B FITS files are mirrored by the SOLAR-B project team around the world. Scientists may get the SOLAR-B FITS files from these sites. LMSL MSSLNAOJ GSFC NRL Oslo Univ. 2005/11/086th Kyoto6 Topics Data Flow from SOLAR-B to Scientists Data Format of Level-0 of SOLAR-B SOLAR-B DB System for Scientists Data Flow from SOLAR-B to Scientists Data Format of Level-0 of SOLAR-B SOLAR-B DB System for Scientists 2005/11/086th Kyoto7 Data Format of Level-0 of SOLAR-B: 1 Level-0 means that the data are not calibrated. (count, position, polarization, etc...) SOLAR-B project does not provide Level-1 (calibrated) data. Instead, each instrument team provides the calibration (IDL) software and the data for calibration. We plan to provide the software and the calibration data using the SSW/SSWDB system Scientists have to calibrate the data using the software and the data, by myself. (The system is like the SXT and TRACE data-analysis system.) Level-0 means that the data are not calibrated. (count, position, polarization, etc...) SOLAR-B project does not provide Level-1 (calibrated) data. Instead, each instrument team provides the calibration (IDL) software and the data for calibration. We plan to provide the software and the calibration data using the SSW/SSWDB system Scientists have to calibrate the data using the software and the data, by myself. (The system is like the SXT and TRACE data-analysis system.) 2005/11/086th Kyoto8 Data Format of Level-0 of SOLAR-B: 2 The data format of Level-0 data is FITS format. The standard FITS format is used for the SOT and XRT data. The data file of EIS use the Binary Extension Table (It is like the data files of CDS) The unit of the file SOT-FG : 1 file per 1 image or several image (ex. Stokes) SOT-SP : 1 file per 1 slit XRT : 1 file per 1 image EIS : 1 file per 1 scan (include several images of the lines) The data format of Level-0 data is FITS format. The standard FITS format is used for the SOT and XRT data. The data file of EIS use the Binary Extension Table (It is like the data files of CDS) The unit of the file SOT-FG : 1 file per 1 image or several image (ex. Stokes) SOT-SP : 1 file per 1 slit XRT : 1 file per 1 image EIS : 1 file per 1 scan (include several images of the lines) 2005/11/086th Kyoto9 Topics Data Flow from SOLAR-B to Scientists Data Format of Level-0 of SOLAR-B SOLAR-B DB System for Scientists Data Flow from SOLAR-B to Scientists Data Format of Level-0 of SOLAR-B SOLAR-B DB System for Scientists 2005/11/086th Kyoto10 SOLAR-B DB System for Scientists: 1 The number of the FITS file of SOLAR-B is huge and it is difficult to look for the FITS files manually. JAXA/ISAS and NAOJ are developing the data search/provide system for the SOLAR-B data. The database system will have the web interface. The number of the FITS file of SOLAR-B is huge and it is difficult to look for the FITS files manually. JAXA/ISAS and NAOJ are developing the data search/provide system for the SOLAR-B data. The database system will have the web interface. SIRIUS DB SOT Reformatter XRT Reformatter EIS Reformatter SOT FITS files XRT FITS files EIS FITS files DARTS FTP Server Scientists Search/Provide System 2005/11/086th Kyoto11 SOLAR-B DB System for Scientists: 2 The database system.. The system have the web (GUI) interface and the CUI interface. The search keys of the search system are taken from the FITS keywords of the Level-0 FITS files. The system can search the data of three instruments from one interface. The system can display the result of search graphically. The database system.. The system have the web (GUI) interface and the CUI interface. The search keys of the search system are taken from the FITS keywords of the Level-0 FITS files. The system can search the data of three instruments from one interface. The system can display the result of search graphically. 2005/11/086th Kyoto12 Example of GUI interface :1 SOLAR-B Data Center Start Time [ 2007/05/02 12 : 23 : 00 ](UT) End Time [ 2007/05/02 14 : 30 : 00 ](UT) Target Position (arcsec) [ +130 ] Y(arcsec) [-120 ] Instruments SOT-FG, XRT, EIS SOT-FG Image Size >[ ] Y>[ ] Wavelength H CaK G-Ban Binning 1x1 2x24x4 Binning 1x1 2x24x4 RESET Search Back 2005/11/086th Kyoto13 Example of GUI interface :2 SOLAR-B Data Center SOT-FG Result The number of hits: 200 Total Size of data 120MB Show list Get all Data XRT EIS Display the FOV Display the time res. Get data of all insts. Back Start Time [ 2007/05/02 12 : 23 : 00 ](UT) End Time [ 2007/05/02 14 : 30 : 00 ](UT) Target Position (arcsec) [ +130 ] Y(arcsec) [-120 ] Instruments SOT-FG, XRT, EIS Result The number of hits: 300 Total Size of data 500MB Show list Get all Data Result The number of hits: 20 Total Size of data 50MB Show list Get all Data If push Show list button of SOT-FG, SOLAR-B Data Center Get Date Pos(X) Pos(Y) wave 2005/05/02 12:25: H 2005/05/02 12:26: H 2005/05/02 12:27: H 2005/05/02 12:28: H SOT-FG Delete the Window Get data Start Time [ 2007/05/02 12 : 23 : 00 ](UT) End Time [ 2007/05/02 14 : 30 : 00 ](UT) Target Position (arcsec) [ +130 ] Y(arcsec) [-120 ] Instruments SOT-FG, XRT, EIS 2005/11/086th Kyoto14 Example of GUI interface :3 Click the Display the FOV button. SOLAR-B Data Center Delete the window SOT-FG EIS XRT Start Time [ 2007/05/02 12 : 23 : 00 ](UT) End Time [ 2007/05/02 14 : 30 : 00 ](UT) Target Position (arcsec) [ +130 ] Y(arcsec) [-120 ] Instruments SOT-FG, XRT, EIS Click he Display the time res. SOLAR-B Data Center Delete the window SOT-SP SOT-FG Ca-K EIS XRT Thin-Al 20:05UT 20:10UT 20:15UT Start Time [ 2007/05/02 12 : 23 : 00 ](UT) End Time [ 2007/05/02 14 : 30 : 00 ](UT) Target Position (arcsec) [ +130 ] Y(arcsec) [-120 ] Instruments SOT-FG, XRT, EIS 2005/11/086th Kyoto15 Summary All SOLAR-B data are accumulated in JAXA/ISAS. The scientific data of SOLAR-B are reformatted to the FITS files. The FITS files are provided from JAXA/ISAS. The level-0 FITS files, the data of calibration and the calibrate software are provided from each instrument team. JAXA/ISAS and NAOJ are developing the mission- wide data search/provide system jointly. All SOLAR-B data are accumulated in JAXA/ISAS. The scientific data of SOLAR-B are reformatted to the FITS files. The FITS files are provided from JAXA/ISAS. The level-0 FITS files, the data of calibration and the calibrate software are provided from each instrument team. JAXA/ISAS and NAOJ are developing the mission- wide data search/provide system jointly.