Forecast text editing rates and analysis

The Centre for Australian Weather and Climate ResearchA partnership between CSIRO and the Bureau of Meteorology

Jaime Clark and Bianca GibsonBrianna LaugherTennessee LeeuwenburgDecember 2012

www.cawcr.gov.au

The Centre for Australian Weather and Climate Research A partnership between CSIRO and the Bureau of Meteorology

Graphical Forecast Editor

GFE Text formatters

GFE Text formatters - editing

GFE Grid Initialisation

Forecast Database

GFEGFESmart Tools

GFE Grid Editing

Guidance Grid Editing & Smart Tools

Text Formatting

Forecast Production

Numerical Modelling

LAPS, GASP, ECMWFJapan, UK, US

Observations

Operational Consensus

Forecast (OCF)

Direct Model Output

Model Output Statistics

Automatic Forecast Generation System

Python Modules

Graphics

Product Generator Forecast Database

Forecast Products

Text Formatters

GFE Forecasting

Text Formatting

Forecast Production

Automatic Forecast Generation System

Python Modules

Graphics

Product Generator Forecast Database

Forecast Products

Text Formatters

GFE Forecasting

editedtext

original

gfemetrics

GFE Metrics

GFE Metricshttp://tardis.bom.gov.au/gfemetricsWeb front-end to “FDB” database created/maintained by ITBInitially developed by Brianna, further developed by Anthony Briggs

STATS/SUMMARY Can view graphs/tables of total edits: by time, by state, by formatter

VIEW Can view any edits made to any GFE forecasts Can query edits by state, date, formatter, period, issuance, locale Can search for edited/unedited/either containing particular words

TOOLS Can view graphs/tables of most used tools: by time, by state

Metric?

We initially hoped that improvements to the formatters would be reflected in lower edit rates... this appears not to be the case... but it's hard to tell...

Dimensions of edits

<—————— REPRESENTATIVENESS ——————> Is the generated text consistent with grid contents? “Wrong” results (genuine error) vs stylistic changes (subjective) “Possible shower” -> “Mostly dry”

<—————— POLICY ——————> Is the edited text consistent with Services policy? Policy may need resolving with Services “Fine”, “moderating”, “freshening”

<—————— MISSING DATA ——————> Is the info required to produce the edited text present in the GFE? Formatter missing rules - “Humid”, “Hot”, sea breeze GFE missing grids – climatology, uncertainty! Other info in forecaster brain (local expertise)

Can potentially

Won't fix

Can't fix(for everyone)

Should fixASAP

Limitations

Formatters are mainly just a group of phrases – Sky, Wx, Wind, Temp, Seas, Swell. Effort is directed at improving phrases – but there's no way to check edit rates of phrases

Hard to get a sense of any general trend when a query returns hundreds of results

Little/no time to devote to exploring regional differences

Monash Talented Students program

Science-Industry Student Research Placement Program“short-term, research-based industry placements”

Rationale(i) facilitating industry experience for upper undergraduate students; (ii) building collaborations in work integrated learning between Monash and

industry partners; (iii) furthering science and its broader communication.

About Bianca

Second year in undergraduate Computer Science

Analysing edits by formatter

Per forecast periodPer statePercentages

Marine edits by forecast period - percentage

Marine edits by forecast period - totals

Precis edits by forecast period - percentages

Town, metro and district edits by forecast period - percentage

NSW edits by formatter - percentage

NSW edits by formatter - totals

SA edits by formatter - percentage

TAS edits by formatter - percentage

VIC edits by formatter - percentage

WA edits by formatter - percentage

Two-Town comparisons

Hypothesis: Bigger towns (capital cities) get more attention than smaller towns

Does “the gap” represent editing that would be done if forecasters had infinite time? Or does it represent stylistic editing that could be done without?

Two-Town comparisons - VIC

Two-Town comparisons - TAS

Go-Live comparisons

Hypothesis: initial edits very low, increase as forecasters learn to use the GFE and have more time for text editing

“Month 0” is the month the GFE went live in that regionVIC: 5/2009NSW: 9/2010TAS: 6/2011SA: 11/2011WA: 10/2012 Each state has a different version of the GFE and different weather

conditionsThe dot and percentage on the far right represents edits in 11/2012

Go-Live comparison – all formatters (incl. warning)

Go-Live comparison - Precis

Go-Live comparison - Metropolitan

Go-Live comparison - marine

About Jaime

4th year in double undergraduate degree:

Science Major - Pure Mathematics Minor - Developmental Psychology Minor – Computational Mathematics

Computer Science Major - Intelligent Systems Major – Software Engineering

5 year course

Edits per formatter by phrase type edited

How do we do this?Formatter

Unedited forecast Edited forecast

Break into sentences

Identify the sentence type

Examining key known words within

the sentence to determine type

Unedited categorized sentences Edited categorized sentences

Compare categories: Identify if a change is present

day0-1 day3+ day0-1 day3+ day0-1 day3+ day0-1 day3+ day0-1 day3+NSW SA TAS VIC WA

0%10%20%30%40%50%60%70%80%90%

wxwindwx windwind skyskywx wind skywx sky

0100200300400500600700800900

Phrase type edited - Town

01-NOV-2012 --- 30-NOV-2012 :: TOWN_FORECAST

0100200300400500600700800

wxwindwx windwind skyskywx sky

Phrase type edited - District

01-NOV-2012 --- 30-NOV-2012 :: DISTRICT_FORECAST

0%10%20%30%40%50%60%70%80%90%

wxwindwx windwind skyskywx sky

100150200250300350400

Phrase type edited - Metropolitan

01-NOV-2012 --- 30-NOV-2012 :: METROPOLITAN_FORECAST

0%10%20%30%40%50%60%70%80%90%

day0-1 day3+ day0-1 day3+ day0-1 day3+ day0-1 day3+NSW SA TAS VIC

10001500200025003000

wxwindwx windseawind skyswellwx skywind sea

Phrase type edited - Coastal

01-NOV-2012 --- 30-NOV-2012 :: COASTAL_FORECAST

0%10%20%30%40%50%60%70%80%90%

wxwindwx windseawind skyswellwx skywind sea

0100200300400500600700800

wxwindwx windseawind skyswellwind sea

Phrase type edited – Local Waters

01-NOV-2012 --- 30-NOV-2012 :: LOCAL_WATERS_FORECAST

0%10%20%30%40%50%60%70%80%90%

wxwindwx windseawind skyswellwind sea

Phrase type edited - Marine

01-NOV-2012 --- 30-NOV-2012 :: MARINE_FORECAST

day0-1 day3+WA

wxseawindswell

day0-1 day3+WA

wxseawindswell

Precis: Investigating additions/removals

How do we do this?Formatter

Unedited forecast Edited forecast

Identify if the given key is present

Unedited keys identified Edited keys identified

Compare keys: Identify if a key has been added/removed

Ratio to un-edited Text

Morning – strongly removed

01-JAN-2012 --- 30-NOV-2012 :: PRECIS_TEXT

morning morning morning morning morning morning morning morning morning morningday0-1 day3+ day0-1 day3+ day0-1 day3+ day0-1 day3+ day0-1 day3+

NSW SA TAS VIC WA

IntersectedRemovedAdded

Afternoon – strongly added

afternoon afternoon afternoon afternoon afternoon afternoon afternoon afternoon afternoonday0-1 day3+ day0-1 day3+ day0-1 day3+ day0-1 day3+ day0-1

NSW SA TAS VIC WA

Late – strongly added

late late late late late late late late late lateday0-1 day3+ day0-1 day3+ day0-1 day3+ day0-1 day3+ day0-1 day3+

NSW SA TAS VIC WA

Developing – strongly removed

developing developing developing developing developing developing developing developing developing developingday0-1 day3+ day0-1 day3+ day0-1 day3+ day0-1 day3+ day0-1 day3+

NSW SA TAS VIC WA

Clearing – strongly removed

clearing clearing clearing clearing clearing clearing clearing clearing clearing clearingday0-1 day3+ day0-1 day3+ day0-1 day3+ day0-1 day3+ day0-1 day3+

NSW SA TAS VIC WA

Towns - Haze/Humid?

haze humidday3+

01-FEB-2012 --- 29-FEB-2012 :: PRECIS_TEXT

The forecaster trick to get humid into the forecast:

– Add haze into the grids.– Generate forecasts.– Remove haze from the grids.– Complete a search and replace

all: haze to humid.

The Centre for Australian Weather and Climate ResearchA partnership between CSIRO and the Bureau of Meteorology

Bianca Gibson: brgib1@student.monash.eduJaime Clark: jkcla3@student.monash.eduBrianna Laugher: b.laugher@bom.gov.au, x4690Tennessee Leeuwenburg: t.leeuwenburg@bom.gov.au, x4310

Web: www.cawcr.gov.au

Thank youwww.cawcr.gov.au

Forecast text editing rates and analysis

Documents

Transcript of Forecast text editing rates and analysis

Can Oil Prices Forecast Exchange Rates?

Modeling temporal variations of seismicity parameters to forecast earthquake rates in Japan

CAN EXCHANGE RATES FORECAST COMMODITY … · 2017-10-02 · CAN EXCHANGE RATES FORECAST COMMODITY PRICES?* ... When market participants foresee future commodity price shocks, ...

Can Oil Prices Forecast Exchange Rates? - … · Can Oil Prices Forecast Exchange Rates? ... oil price ⁄uctuations serve as an observable and ... the lagged oil price model. Section

CAN EXCHANGE RATES FORECAST COMMODITY PRICES?faculty.washington.edu/yuchin/Papers/CRRv3.pdf · 2009-07-03 · CAN EXCHANGE RATES FORECAST COMMODITY PRICES? ... When market participants

CAN OIL PRICES FORECAST EXCHANGE RATES? NATIONAL …To further study the link between oil prices and exchange rates, in addition to a simple regression of exchange rates on oil prices,

Forecasting Exchange Rates: a Comparative Analysis€¦ · Forecasting Exchange Rates: a Comparative Analysis ... ability to forecast, over a period lower than 12 months, rather than

No Slide Title · PPT file · Web view · 2016-03-042016-03-04 · Total Fertility Rates Fertility Assumptions in ICBS Forecast for Israel: Israel Moslems Israel’s Official Forecast

Impacts of Oil Shocks on Exchange Rates and … · Impacts of Oil Shocks on Exchange Rates and Macroeconomic Variables: ... could forecast future exchange rate ... or nonfundamental

RNS Rates – Five Year Forecast

How Well Do Monetary Fundamentals Forecast Exchange Rates? · How well do monetary fundamentals forecast exchange ... models of exchange rate determination were ... provides the solution

Photo editing services at affordable rates

CAN EXCHANGE RATES FORECAST COMMODITY PRICES? - FGV…epge.fgv.br/finrio/myreview/FILES/CR2/p20.pdf · 1 1. Introduction This paper demonstrates that the exchange rates of a number

CAN EXCHANGE RATES FORECAST COMMODITY …dornsife.usc.edu/assets/sites/462/docs/papers/published/... · 2013-11-26 · CAN EXCHANGE RATES FORECAST COMMODITY PRICES?* ... When market

Can Oil Prices Forecast Exchange Rates?...prices and exchange rates relevant? Our results suggest that, conditional on knowing the future value of commodity prices, we can forecast

wgcep.orgwgcep.org/sites/wgcep.org/files/Dawson_UCERF3Deformation... · Web viewBird, P. [2009] Long-term fault slip rates, distributed deformation rates, and forecast of seismicity

An Artificial Neural Network Model to Forecast Exchange RatesAn Artificial Neural Network Model to Forecast Exchange Rates. 59 ral network model by applying the idea of the optimiza-tion

TURBULENCE AHEAD DISENGAGE THE AUTOPILOT · Key Fleet Forecast Growth Rates 2015-2025 Global Fleet Forecast by Aircraft Class The global air transport jet and turboprop fleet will

The Australian Editing · 6 Editing practice: onscreen editing techniques 95 7 Editing practice: copy editing techniques 127 8 Editing practice: other elements 157 9 Editing practice:

How Well Do Professionals Forecast Exchange Rates ...inceo/Survey Based Expectations.pdf · How Well Do Professionals Forecast Exchange Rates? Evidence from Survey Expectations †