Data centre support for the IGS-RT PP
description
Transcript of Data centre support for the IGS-RT PP
![Page 1: Data centre support for the IGS-RT PP](https://reader035.fdocuments.in/reader035/viewer/2022062222/56815a83550346895dc7eeaa/html5/thumbnails/1.jpg)
Data centre support for the IGS-RT PP
W. Söhne, H. Habrich, G. WeberFederal Agency for Cartography and Geodesy,
Frankfurt am Main, Germany
![Page 2: Data centre support for the IGS-RT PP](https://reader035.fdocuments.in/reader035/viewer/2022062222/56815a83550346895dc7eeaa/html5/thumbnails/2.jpg)
Outline
Introduction IGS-RT PP
• Data and product file centre• File types under consideration
Technical aspects• Data volume• Up- and downloading• Completeness • Limitations
Policy aspects• Upload policy
Conclusions & open questions
![Page 3: Data centre support for the IGS-RT PP](https://reader035.fdocuments.in/reader035/viewer/2022062222/56815a83550346895dc7eeaa/html5/thumbnails/3.jpg)
Introduction
Storing of highrate files nothing new for IGS• At CDDIS back to 2001, doy121• Initiative from the LEO needs• At IGN• Others
Term “highrate” is a relative one• Usually means 1 Hz sampling rate• But higher sampling rates in use depending on the
application Currently ~110 stations at CDDIS Clear statement from IGS concerning the
needs of long-term archive still missing
![Page 4: Data centre support for the IGS-RT PP](https://reader035.fdocuments.in/reader035/viewer/2022062222/56815a83550346895dc7eeaa/html5/thumbnails/4.jpg)
IGS-RT PP (1)
One activity of the IGS-RT PP: Data and product file centre
File types under consideration• Navigational data (broadcast ephemeris)• Observational data
- highrate RINEX files- hourly and daily RINEX files
• Derived files (products)- Orbits or orbit corrections- Clocks or clock corrections- Troposphere & ionosphere parameters
![Page 5: Data centre support for the IGS-RT PP](https://reader035.fdocuments.in/reader035/viewer/2022062222/56815a83550346895dc7eeaa/html5/thumbnails/5.jpg)
IGS-RT PP (2)
Why generation of highrate files?• Currently, considerable number of users are
“feeding” near real-time applications • For future scientific analyses
What is “near real-time”?• Typically hourly applications, with 20-30 minutes
computation time two times per hour computation possible
Why 15 minutes files?• Reduction of file size compared to an hourly
highrate file• “Convention”
![Page 6: Data centre support for the IGS-RT PP](https://reader035.fdocuments.in/reader035/viewer/2022062222/56815a83550346895dc7eeaa/html5/thumbnails/6.jpg)
IGS-RT PP (3)
Is that interval small enough? Are there other intervals of interest, e.g. 5 minutes files?• Advantages
- Better distribution of uploading over time- Other intervals can easily be created/derived- File naming convention fits
• Disadvantages- Interval size not known in IGS- Number of files growing
![Page 7: Data centre support for the IGS-RT PP](https://reader035.fdocuments.in/reader035/viewer/2022062222/56815a83550346895dc7eeaa/html5/thumbnails/7.jpg)
Technical aspects: data volume
Data volume of highrate observational files• Example BUTE115A45.08O: 1 Hz 15 minutes
GPS+GLONASS RINEX file, # obs. types 8, 20 SV in view: 2.4 Mb
• Hatanaka+compress: 170 Kb 5.7 Gb per station and year for the compressed
files >> 500 Gb per year• Selection of IGS stations for high-rate storing?
Data volume of product files• Clocks: 1 Hz, 1 hour: 2.5 Mb, compressed 170 Kb
5.7 Gb per solution and year• …
![Page 8: Data centre support for the IGS-RT PP](https://reader035.fdocuments.in/reader035/viewer/2022062222/56815a83550346895dc7eeaa/html5/thumbnails/8.jpg)
Technical aspects: up- and downloading
Data upload of observational files• Example 250 stations, 24 files per day (hourly):
6000 files uploaded * 100 Kb 0.6 Gb per day• Example 110 stations, 96 files per day (highrate):
10560 files uploaded * 170 Kb 1.7 Gb per day Critical aspects are
• Is the data centre able to handle all incoming files within, e.g., 5 minutes after the full hour?If not spreading of upload over a certain time span?!
• Is the data centre able to handle the parallel download requests in peak periods?
![Page 9: Data centre support for the IGS-RT PP](https://reader035.fdocuments.in/reader035/viewer/2022062222/56815a83550346895dc7eeaa/html5/thumbnails/9.jpg)
Technical aspects: completeness (1)
“Completeness” covers the aspects• Number of observation types• Number of epochs• Number of SVs per epoch• Ratio observed / predicted number of observations
Completeness can be affected by• Outages of single stations data streams• Outages of the broadcasting system• Handling of unhealthy SVs• Limitation of supported observation types
![Page 10: Data centre support for the IGS-RT PP](https://reader035.fdocuments.in/reader035/viewer/2022062222/56815a83550346895dc7eeaa/html5/thumbnails/10.jpg)
Technical aspects: limitations (1)
“Limitations” cover the following aspects• Resolution of
- Phase- Code
• Support of - GPS L2C, L5- GLONASS- Galileo- SBAS
![Page 11: Data centre support for the IGS-RT PP](https://reader035.fdocuments.in/reader035/viewer/2022062222/56815a83550346895dc7eeaa/html5/thumbnails/11.jpg)
Technical aspects: completeness (2)
Different levels of comparison of original and accumulated files possible• Character by character
- Difficult due to roll-over phase values • Completeness
- Epochs missing?- SVs missing?
• Parallel analysis- Differences between the results
![Page 12: Data centre support for the IGS-RT PP](https://reader035.fdocuments.in/reader035/viewer/2022062222/56815a83550346895dc7eeaa/html5/thumbnails/12.jpg)
Technical aspects: completeness (3)
Hourly file BUTE115A.08O (TEQC) Highrate file BUTE115A00.08O (BNC)
![Page 13: Data centre support for the IGS-RT PP](https://reader035.fdocuments.in/reader035/viewer/2022062222/56815a83550346895dc7eeaa/html5/thumbnails/13.jpg)
Technical aspects: limitations (2)
ALME: file from TEQC vs. file from BNC (RTCM 2.3) after RNXSMT (BSW5.0)
![Page 14: Data centre support for the IGS-RT PP](https://reader035.fdocuments.in/reader035/viewer/2022062222/56815a83550346895dc7eeaa/html5/thumbnails/14.jpg)
Technical aspects: limitations (3)
WTZR: file from TEQC vs. file from BNC (RTCM 3.0) after RNXSMT (BSW5.0)
![Page 15: Data centre support for the IGS-RT PP](https://reader035.fdocuments.in/reader035/viewer/2022062222/56815a83550346895dc7eeaa/html5/thumbnails/15.jpg)
Technical aspects: completeness (4)
579
80110
58 5662
99,9 -100,0%
99,5 - 99,6%
50,0 - 90,0%
100,0%
99,8 - 99,9%
90,0 - 99,0%
< 50,0%
99,6 - 99,7%
99,0 - 99,5%
99,7 - 99,8%
168
50010
2135 21 99,9 -100,0%
99,5 - 99,6%
50,0 - 90,0%
100,0%
99,8 - 99,9%
90,0 - 99,0%
< 50,0%
99,6 - 99,7%
99,0 - 99,5%
99,7 - 99,8%
0 1
001 0
4
14
2 2 99,9 -100,0%
99,5 - 99,6%
50,0 - 90,0%
100,0%
99,8 - 99,9%
90,0 - 99,0%
< 50,0%
99,6 - 99,7%
99,0 - 99,5%
99,7 - 99,8%
Completeness of 24 hourly high-rateRINEX files from RTIGS streamscoming in via udpRelay
Completeness of 233 hourly high-rateRINEX files from RTCMv2 streamscoming in via NTRIP/TCP
Completeness of 711 hourly high-rateRINEX files from RTCMv3 streamscoming in via NTRIP/TCP
![Page 16: Data centre support for the IGS-RT PP](https://reader035.fdocuments.in/reader035/viewer/2022062222/56815a83550346895dc7eeaa/html5/thumbnails/16.jpg)
Technical aspects: completeness (5)
Missing SVs G06, G21, G26, G30
Streaming interrupted Longer sections w/o GLONASS
![Page 17: Data centre support for the IGS-RT PP](https://reader035.fdocuments.in/reader035/viewer/2022062222/56815a83550346895dc7eeaa/html5/thumbnails/17.jpg)
Policy aspects (1)
Station A
Station B
Station C
Station N
Data centre 1
Data centre 2
Data centre 3
User a
User b
User c
User m
Mirroring
Mirroring
„Traditional“ situation transferring hourly files, daily files, ephemeris files
file creation at the station
![Page 18: Data centre support for the IGS-RT PP](https://reader035.fdocuments.in/reader035/viewer/2022062222/56815a83550346895dc7eeaa/html5/thumbnails/18.jpg)
Policy aspects (2)
Possible situation using real-time data streams for RINEX file creation
Station A
Station B
Station C
Station N
Broadcaster 1 User a
User b
Data centre 1
transferring files transferring streams
or
file creation possible at the data centre, at the users site, at a third party
![Page 19: Data centre support for the IGS-RT PP](https://reader035.fdocuments.in/reader035/viewer/2022062222/56815a83550346895dc7eeaa/html5/thumbnails/19.jpg)
Conclusions & open questions (1)
Derivation of highrate RINEX files from real-time streams suitable tool
Current interval for highrate files 15 minutes – is that small enough?
Recommendation #1: Storing of 1 Hz 15 minutes files, at least for the long-term archive
Derivation of daily files from streams instead of ftp transfer, at least for new or proposed stations?
RT-IGS PP CfP found five candidates (BKG, CDDIS, GA, KASI, Univ. of Padova)
![Page 20: Data centre support for the IGS-RT PP](https://reader035.fdocuments.in/reader035/viewer/2022062222/56815a83550346895dc7eeaa/html5/thumbnails/20.jpg)
Conclusions & open questions (2)
Limitations acceptable? If yes, how to point the users to these limitations effectively?
Derivation of RINEX files from real-time streams possible for everyone – needs some regulation or clarification: Who is allowed to upload files derived from real-time data streams?
Recommendation #2: highrate file creation and upload to the GDC in one hand, ideally at the broadcaster’s side