The potential for improvements to ICOADS: lessons from ... · KNMI 45-column Hollerith punch card...
Transcript of The potential for improvements to ICOADS: lessons from ... · KNMI 45-column Hollerith punch card...
![Page 1: The potential for improvements to ICOADS: lessons from ... · KNMI 45-column Hollerith punch card Punch card typists keying ship logbooks at KNMI in ~1950s Wallbrink and Koek (2010)](https://reader030.fdocuments.in/reader030/viewer/2022040218/5e0be37b47584f27190f053a/html5/thumbnails/1.jpg)
The potential for improvements to ICOADS: lessons from ship tracking
Elizabeth Kent, David Berry, Giulia Carella
![Page 2: The potential for improvements to ICOADS: lessons from ... · KNMI 45-column Hollerith punch card Punch card typists keying ship logbooks at KNMI in ~1950s Wallbrink and Koek (2010)](https://reader030.fdocuments.in/reader030/viewer/2022040218/5e0be37b47584f27190f053a/html5/thumbnails/2.jpg)
Marine climate data products from the 1980s & early 1990s
• Esbensen and Kushnir (1981) – TDF-11 • Isemer and Hasse (1987) – TDF-11 • Hsiung (1985) – CDS - FNMOC • GOSTA (1990) – UKMO MDB • Oberhuber (1988) – COADS R1
"Largetape". Licensed under Creative Commons Attribution-Share Alike 3.0 via Wikimedia Commons http://commons.wikimedia.org/wiki/File:Largetape.jpg#mediaviewer/File:Largetape.jpg
![Page 3: The potential for improvements to ICOADS: lessons from ... · KNMI 45-column Hollerith punch card Punch card typists keying ship logbooks at KNMI in ~1950s Wallbrink and Koek (2010)](https://reader030.fdocuments.in/reader030/viewer/2022040218/5e0be37b47584f27190f053a/html5/thumbnails/3.jpg)
Air-sea fluxes: Isemer and Hasse (1987)
The Bunker Climate Atlas of the North Atlantic Ocean
![Page 4: The potential for improvements to ICOADS: lessons from ... · KNMI 45-column Hollerith punch card Punch card typists keying ship logbooks at KNMI in ~1950s Wallbrink and Koek (2010)](https://reader030.fdocuments.in/reader030/viewer/2022040218/5e0be37b47584f27190f053a/html5/thumbnails/4.jpg)
GOSTA Atlas SST and MAT (1990)
Published 1990
![Page 5: The potential for improvements to ICOADS: lessons from ... · KNMI 45-column Hollerith punch card Punch card typists keying ship logbooks at KNMI in ~1950s Wallbrink and Koek (2010)](https://reader030.fdocuments.in/reader030/viewer/2022040218/5e0be37b47584f27190f053a/html5/thumbnails/5.jpg)
KNMI 45-column Hollerith punch card
![Page 6: The potential for improvements to ICOADS: lessons from ... · KNMI 45-column Hollerith punch card Punch card typists keying ship logbooks at KNMI in ~1950s Wallbrink and Koek (2010)](https://reader030.fdocuments.in/reader030/viewer/2022040218/5e0be37b47584f27190f053a/html5/thumbnails/6.jpg)
Punch card typists keying ship logbooks at KNMI in ~1950s
Wallbrink and Koek (2010) Data Acquisition And Keypunching Codes For Marine Met. Observations
![Page 7: The potential for improvements to ICOADS: lessons from ... · KNMI 45-column Hollerith punch card Punch card typists keying ship logbooks at KNMI in ~1950s Wallbrink and Koek (2010)](https://reader030.fdocuments.in/reader030/viewer/2022040218/5e0be37b47584f27190f053a/html5/thumbnails/7.jpg)
TDF-11 manual (1968)
![Page 8: The potential for improvements to ICOADS: lessons from ... · KNMI 45-column Hollerith punch card Punch card typists keying ship logbooks at KNMI in ~1950s Wallbrink and Koek (2010)](https://reader030.fdocuments.in/reader030/viewer/2022040218/5e0be37b47584f27190f053a/html5/thumbnails/8.jpg)
TDF-11: standard and supplemental data
![Page 9: The potential for improvements to ICOADS: lessons from ... · KNMI 45-column Hollerith punch card Punch card typists keying ship logbooks at KNMI in ~1950s Wallbrink and Koek (2010)](https://reader030.fdocuments.in/reader030/viewer/2022040218/5e0be37b47584f27190f053a/html5/thumbnails/9.jpg)
TDF-11: some data not kept
![Page 10: The potential for improvements to ICOADS: lessons from ... · KNMI 45-column Hollerith punch card Punch card typists keying ship logbooks at KNMI in ~1950s Wallbrink and Koek (2010)](https://reader030.fdocuments.in/reader030/viewer/2022040218/5e0be37b47584f27190f053a/html5/thumbnails/10.jpg)
TDF-11: some data not kept
![Page 11: The potential for improvements to ICOADS: lessons from ... · KNMI 45-column Hollerith punch card Punch card typists keying ship logbooks at KNMI in ~1950s Wallbrink and Koek (2010)](https://reader030.fdocuments.in/reader030/viewer/2022040218/5e0be37b47584f27190f053a/html5/thumbnails/11.jpg)
Lessons from ship tracking
![Page 12: The potential for improvements to ICOADS: lessons from ... · KNMI 45-column Hollerith punch card Punch card typists keying ship logbooks at KNMI in ~1950s Wallbrink and Koek (2010)](https://reader030.fdocuments.in/reader030/viewer/2022040218/5e0be37b47584f27190f053a/html5/thumbnails/12.jpg)
Ship tracking - January 1900
Unknown ID report from same deck (201) inserted
![Page 13: The potential for improvements to ICOADS: lessons from ... · KNMI 45-column Hollerith punch card Punch card typists keying ship logbooks at KNMI in ~1950s Wallbrink and Koek (2010)](https://reader030.fdocuments.in/reader030/viewer/2022040218/5e0be37b47584f27190f053a/html5/thumbnails/13.jpg)
Ship tracking – January 1900
Joining of records from e.g. logbook pages0120J774 & 0120J775
![Page 14: The potential for improvements to ICOADS: lessons from ... · KNMI 45-column Hollerith punch card Punch card typists keying ship logbooks at KNMI in ~1950s Wallbrink and Koek (2010)](https://reader030.fdocuments.in/reader030/viewer/2022040218/5e0be37b47584f27190f053a/html5/thumbnails/14.jpg)
January 1900 – adding info. from TDF-11 doc.
Joining of records from e.g. logbook pages0120J774 & 0120J775
![Page 15: The potential for improvements to ICOADS: lessons from ... · KNMI 45-column Hollerith punch card Punch card typists keying ship logbooks at KNMI in ~1950s Wallbrink and Koek (2010)](https://reader030.fdocuments.in/reader030/viewer/2022040218/5e0be37b47584f27190f053a/html5/thumbnails/15.jpg)
TDF-11 additional information
deck 118 – Japanese ship obs # 1 “Ship class”
deck 197 – Danish (and other) polar obs “code sheet page # and source”
deck 184 – GB Marine Obs. “Country of origin”
deck 196 – German marine “Ship class”
![Page 16: The potential for improvements to ICOADS: lessons from ... · KNMI 45-column Hollerith punch card Punch card typists keying ship logbooks at KNMI in ~1950s Wallbrink and Koek (2010)](https://reader030.fdocuments.in/reader030/viewer/2022040218/5e0be37b47584f27190f053a/html5/thumbnails/16.jpg)
Ship tracking January 1895
New track from data withno ID information. Allfrom same deck.
![Page 17: The potential for improvements to ICOADS: lessons from ... · KNMI 45-column Hollerith punch card Punch card typists keying ship logbooks at KNMI in ~1950s Wallbrink and Koek (2010)](https://reader030.fdocuments.in/reader030/viewer/2022040218/5e0be37b47584f27190f053a/html5/thumbnails/17.jpg)
Ship tracking – January 1960
ID 14101, deck 213
no ID, deck 254no ID, deck 926
ID 31314101deck 902
ID 14101, deck 213
![Page 18: The potential for improvements to ICOADS: lessons from ... · KNMI 45-column Hollerith punch card Punch card typists keying ship logbooks at KNMI in ~1950s Wallbrink and Koek (2010)](https://reader030.fdocuments.in/reader030/viewer/2022040218/5e0be37b47584f27190f053a/html5/thumbnails/18.jpg)
Decks containing common data
![Page 19: The potential for improvements to ICOADS: lessons from ... · KNMI 45-column Hollerith punch card Punch card typists keying ship logbooks at KNMI in ~1950s Wallbrink and Koek (2010)](https://reader030.fdocuments.in/reader030/viewer/2022040218/5e0be37b47584f27190f053a/html5/thumbnails/19.jpg)
Eltanin
Deck 897 in ICOADS • platform type = missing • ID = missing
![Page 20: The potential for improvements to ICOADS: lessons from ... · KNMI 45-column Hollerith punch card Punch card typists keying ship logbooks at KNMI in ~1950s Wallbrink and Koek (2010)](https://reader030.fdocuments.in/reader030/viewer/2022040218/5e0be37b47584f27190f053a/html5/thumbnails/20.jpg)
ICOADS Supplemental data SS Illinois, January 1880
Red circles = original ICOADS locations Green circles = locations interpolated between midday fixes using ICOADS supplemental data
![Page 21: The potential for improvements to ICOADS: lessons from ... · KNMI 45-column Hollerith punch card Punch card typists keying ship logbooks at KNMI in ~1950s Wallbrink and Koek (2010)](https://reader030.fdocuments.in/reader030/viewer/2022040218/5e0be37b47584f27190f053a/html5/thumbnails/21.jpg)
A reprocessing of ICOADS? New, sophisticated, dataset construction methods need more information: • platform identifiers • information on quality, metadata, observation types ICOADS (and TDF-11 before that) kept most of the available information • information may have been lost at many stages before this What would we gain from revisiting the earlier data? • prior to Release 2.5 “worse duplicates” were not kept • but input data to dupelim retained (now available by deck in lmr format) • Duplicate elimination and land masking could be rerun • More information extracted from supplemental fields • Examination of inconsistencies between decks -> improved quality
![Page 22: The potential for improvements to ICOADS: lessons from ... · KNMI 45-column Hollerith punch card Punch card typists keying ship logbooks at KNMI in ~1950s Wallbrink and Koek (2010)](https://reader030.fdocuments.in/reader030/viewer/2022040218/5e0be37b47584f27190f053a/html5/thumbnails/22.jpg)
Deck 194/201 comparison - 1860
![Page 23: The potential for improvements to ICOADS: lessons from ... · KNMI 45-column Hollerith punch card Punch card typists keying ship logbooks at KNMI in ~1950s Wallbrink and Koek (2010)](https://reader030.fdocuments.in/reader030/viewer/2022040218/5e0be37b47584f27190f053a/html5/thumbnails/23.jpg)
Deck 194/201 comparison - 1860
![Page 24: The potential for improvements to ICOADS: lessons from ... · KNMI 45-column Hollerith punch card Punch card typists keying ship logbooks at KNMI in ~1950s Wallbrink and Koek (2010)](https://reader030.fdocuments.in/reader030/viewer/2022040218/5e0be37b47584f27190f053a/html5/thumbnails/24.jpg)
“dupelim allowances”
![Page 25: The potential for improvements to ICOADS: lessons from ... · KNMI 45-column Hollerith punch card Punch card typists keying ship logbooks at KNMI in ~1950s Wallbrink and Koek (2010)](https://reader030.fdocuments.in/reader030/viewer/2022040218/5e0be37b47584f27190f053a/html5/thumbnails/25.jpg)
A reprocessing of ICOADS? A wealth of information is available in ICOADS supplemental data Reinstate original format data (e.g. Beaufort force) • revisit conversions Better tailor data formats to observation types, e.g. • historical logbooks • Ocean Weather Ships • Drifting buoys Track data within decks before merging and identifying duplicates Improved duplicate identification, allowing for position errors • and retain all duplicates for those that want to try themselves Pilot project to assess value of going back to “card images”