IMOS PROFILING CTD DATAimos.org.au/fileadmin/user_upload/shared/IMOS...
27
IMOS PROFILING CTD DATA IMPLEMENTATION OF AN AUTOMATED POST-PROCESSING METHOD Cristian Munoz Mas, Craig Steinberg, Greg Coleman, Felicity McAllister, Paul Rigby, Guillaume Galibert, John Luetchford, Chris Bartlett, Juergen Zier, Mederic Mainson IMOS QC Summit Profiling CTD Processing Workshop Hobart, 11 th and 12 th December 2013
Transcript of IMOS PROFILING CTD DATAimos.org.au/fileadmin/user_upload/shared/IMOS...
IMOS PROFILING CTD DATA IMPLEMENTATION OF AN AUTOMATED
POST-PROCESSING METHOD
Cristian Munoz Mas, Craig Steinberg, Greg Coleman, Felicity McAllister, Paul Rigby, Guillaume Galibert, John Luetchford, Chris Bartlett, Juergen Zier, Mederic Mainson
IMOS QC Summit Profiling CTD Processing Workshop Hobart, 11th and 12th December 2013
CONTENTS
1. OBJECTIVE
2. BACKGROUND
3. PROFILING CTD DATA POST-PROCESSING
4. CURRENT APPLICATIONS
5. CONCLUSIONS
6. NEXT STEPS
7. DISCUSSION TOPICS
1. OBJECTIVE
Automate the CTD post-processing method
RAW DATA *.hex format
PROCESSED DATA *.cnv format
netCDF DATA *.nc format
2. BACKGROUND
NRS-BGC ANMN Standardised Profiling CTD Data Post-Processing Procedures v.1.5
SIMS, CSIRO, SARDI and AIMS. 08 March 2012.
Instrument output CTD Binary File *.hex
(e.g. YON201202080112.hex)
ASCII file *c.cnv
(e.g. YON201202080112c.cnv)
SBE Data Processing-Win32
Converts binary *.hex to *.cnv ASCII
SBE Data Processing-Win32
Post-processing *.cnv ASCII
ASCII file *cf.cnv (e.g. YON201202080112cf.cnv)
ASCII file *cfa.cnv (e.g. YON201202080112cfa.cnv)
ASCII file *cfal.cnv (e.g. YON201202080112cfal.cnv)
ASCII file *cfald.cnv (e.g. YON201202080112cfald.cnv)
ASCII file *cfaldb.cnv (e.g. YON201202080112cfaldb.cnv)
SBE SeatermV2
Download CTD Binary File *.hex
Matlab / imosToolbox
Converts cfaldb.cnv file to NetCDF format
NetCDf File *.nc (e.g. IMOS_ANMN-
NRS_CDEKSTZ_20120208T011200Z_NRSYON_
FV00_005554-NRS-BGC-CTD-SBE19plus-29_C-
20130819T234329Z)
3. PROFILING CTD DATA POST-PROCESSING Scope of the process
RAW data *.hex
Processed data *.cnv FV00
netCDF data *.nc FV00
SBE Post-processing Auto-ctd
IMOS Toolbox
Metadata
Metadata Local deployment
database
Config. file *.con / *.xmlcon
IMOS – eMII Data
center
3. SBE POST-PROCESSING PROCEDURE MANUAL METHOD
SEMI-AUTOMATED METHOD
AUTOMATED METHOD
3. SBE POST-PROCESSING Manual method
3. BATCH RUN SBE POST-PROCESSING Semi-Automated method
3. BATCH RUN SBE POST-PROCESSING Automated method
3. PROFILING CTD DATA POST-PROCESSING Scope of the process
RAW data *.hex
Processed data *.cnv FV00
netCDF data *.nc FV00
SBE Post-processing Auto-ctd
IMOS Toolbox
Metadata
Metadata Local deployment
database
Config. file *.con / *.xmlcon
IMOS – eMII Data
center
3. PROFILING CTD DATA POST-PROCESSING IMOS DATABASE
3. PROFILING CTD DATA POST-PROCESSING Scope of the process
RAW data *.hex
Processed data *.cnv FV00
netCDF data *.nc FV00
SBE Post-processing Auto-ctd
IMOS Toolbox
Metadata
Metadata Local deployment
database
Config. file *.con / *.xmlcon
IMOS – eMII Data
center
3. PROFILING CTD DATA POST-PROCESSING IMOS MATLAB TOOLBOX
4. CURRENT APPLICATIONS National Reference Station Biogeochemical Sampling
4. CURRENT APPLICATIONS Australian National Mooring Network
4. CURRENT APPLICATIONS Australian Institute of Marine Science CTD transects
• Importance of an Automated process:
saving time when processing data.
avoiding mistakes from the user.
• Multiple applications using the same standardized method.
• Multiple users using the same standardized method.
• SBE Post-processing procedure can be understood as a QC process in itself in some steps.
5. CONCLUSIONS
• Keep improving automated process.
Solve errors.
Add new variables for different applications.
6. NEXT STEPS
• Keep improving automated process.
• Possible inclusion as a pre-processing routine in the IMOS toolbox and/or QC routines.
6. NEXT STEPS
IMOS Toolbox
RAW data *.hex
SBE Post-processing Pre-process routine
Local deployment
database
Metadata
Metadata
Config file *.con/*.xmlcon
Display data
Metadata
Converted data
*.cnv FV00
netCDF data
*.nc FV01
netCDF data
*.nc FV00
IMOS – eMII Data center
Export data Interactive display
Quality Control Interactive display
Yes No
RAW format netCDF format
QC routines
IMOS Toolbox
RAW data *.hex
SBE Post-processing Pre-process routine
Local deployment
database
Metadata
Config file *.con/*.xmlcon
Metadata
Display data
Metadata
Converted data
*.cnv FV00
netCDF data *.nc FV01
netCDF data *.nc FV00
IMOS – eMII Data center
Export data Interactive display
Quality Control Interactive display
Yes
No
Mode Profile SBE Post-processing
netCDF format RAW format
QC routines
• Keep improving automated process.
• Possible inclusion as a pre-processing tool in the IMOS toolbox and/or QC routines.
• QC routines applicable to CTD profile data.
6. NEXT STEPS
• Keep improving automated process.
• Possible inclusion as a pre-processing tool in the IMOS toolbox and/or QC routines.
• QC routines applicable to CTD profile data.
• Need to archive and make available raw data sampled & processed.
6. NEXT STEPS
Instrument Output RAW data *.hex
Imos-toolbox
Converted data
*c.cnv
Processed data
*cfaldb.cnv
Processed data
*cfaldb.cnv +QC
eMII – Data Center
Converted data
*FV00.cnv
netCDF data
*FV00.nc
netCDF data
*FV01.nc
Sampled
QC
1. Include auto-ctd process as pre-processing routine within imos-toolbox.
2. Case YES: divide auto-ctd method as pre-processing routine and QC within imos-toolbox.
3. Current imos-QC routines applicable to CTD profile data.
4. Archive and make available raw data sampled & processed in different files.
6. DISCUSSION TOPICS
