Lessons Learned in Transitioning

Solar-Interplanetary Research

models into Operational Services

Siqing Liu1, Bingxian Luo1, Jiancun Gong1, WengengHuang,1 Jingjing Wang1, Yuming Wang2, Chuanbing

Wang2

1National Space Science Center, CAS2University of Science and Technology of China

Outlines

Motivations

Progress

Lessons Learned

Motivations

Co-rotating Interacting Regions (CIRs) Forecast

– General information of background solar wind

– Time of CIR arrival at Earth

Oncoming CMEs Forecast

– Geoeffectiveness of CMEs (will arrive or not)

– Time of CME arrival at Earth

– Geomagnetic storm intensity

CME-driven Shock-associated SEP forecast

– SEP intensity forecast

– SEP spectrum

Pave the way for future space weather model

transitions

Motivations

Solar Magnet-ogram

Solar Observations

CME’s Corona-graph

Kinematic

Models

Empirical Models

PFSS + WSACoronal B Field

Solar Wind speed

CME detection and fitting ModelCME detection, CME size,

initial speed and direction

HAF + SEPMSolar Wind speed,

density, B Field, and energetic particles

Earth

Sun Near the sun Interplanetary Earth

Solar Interplanetary Forecasting System

Researchers of modelling: Yuming Wang, Chuanbing Wang,

Chenglong Shen, etc., from University of Science and

Technology of China

NSSC support staff (V&V, transition, IT, mamager, etc)

– Managers: Siqing Liu, Jiancun Gong

– Forecasters and model users: Jingjing Wang, Wengeng

Huang, Bingxian Luo, etc., at NSSC, CAS

– Engineers: Yanxia Cai, Guorui Lu, Zhaofeng Chen, Lei

Zhang, Lili Bao, etc., at NSSC, CAS

Supported by National programs.

Progress

Transition Team

Progress

01 02 03

06 05 04

Forecast needs

CIRs arrival

CME arrival

CME-driven shock SEP

Models/tools we have Input data we can get

Background SW models (PFSS+HAF)

CME detection and fitting models

(SEEDS+Cones)

SW propagation models (HAF)

Particle acceleration models (PATH)

Solar magnetic field map

Coronagraph images

System integration

Background solar wind model

CME-detection/fitting model

CME propagation model

SEP simulation model

Model improvement Model verification

Focusing on halo-CMEs (new

CME detection models)

Human-computer interaction

(CME fitting accuracy)

CME detection percentage

CME parameters accuracy

CME propagation evaluation

SEP spectrum

Chart of R2O process

Progress

Solar Interplanetary Forecasting System

1 2 3 4

PFSS modelSCC/HCCS/CSSS

WSA modelHAF model

Auto-detection(SEEDS)

Manned-detectionCone model

Ice-cream Cone

HAF model Diffusive Shock acceleration

model

Background solar wind (CIR)

CME detection and fitting

CME propagation

SEP Simulation

Photosphericfield

Source Surface field

Derivedcoronal holes

Source SurfaceSW speed

Part 1 : background solar wind - PFSS+WSA+HAF model

Problem: There are four different functions to relate the solar wind speed to the flux tube expansion factor.

Part 2 : CME detection and fitting – auto detection

Corona graphic observations

J-map and Hough transform

CME front traceCME collection and identification

Problem: The SEEDS algorithm yields poor performance for Halo-CME detections.

Part 2 : CME detection and fitting – manned detection

Find CME Draw line

Getting pointsDeriving CME trace

Part 2 : CME detection and fitting – Ice-cream cone model

CME front trace Ice-cream Cone model assumption

Fitting

SW density and Speed

SW pressure and magnetic field

Part 3 : CME Propagation Model - HAF

Source Surface field

Source Surface SW speed

Part 4 : SEP simulation (Diffusive Shock accelerative Code)

Progress: System Integration

CME Searching Trace CME front

Output from CME fittingInput for propagationDownload results

Case study : March 15 ,2015

CME observation

C9.1 flare erupted from AR2297 (S24W38)on 00:45UTC, March 15

Cone model fitting

Propagation direction : N01E07Angular width : 126 degreePropagation velocity: 1130 km/s

Case study : March 15 ,2015

IMF at Earth

DVE at Earth

Density and magnetic field

Pressure and magnetic field

Predicted Shock Arrival Time Difference (hrs) Confidence (%) Method

03-16T23:00Z -5.08 SEPC model

03-17T18:00Z 13.92 ---- WSA-ENLIL + Cone (NOAA/SWPC)

03-17T18:08Z (-15.0h, +26.3h) 14.05 55.0 COMESEP

03-16T18:00Z (-12.0h, +12.0h) -10.08 ---- Other (SIDC)

03-17T12:00Z (-12.0h, +6.0h) 7.92 60.0 WSA-ENLIL + Cone (Met Office)

03-17T11:39Z (-7.0h, +7.0h) 7.57 ---- WSA-ENLIL + Cone (GSFC SWRC)

03-17T11:48Z (-5.3h, +7.7h) 7.72 100.0 Ensemble WSA-ENLIL + Cone (GSFC SWRC)

03-17T10:55Z 6.83 71.6667 Average of all Methods

Case study : March 15 ,2015

Data from CCMC

Case study : June 18 and 21 ,2015

CME observation

M3 flare erupted from S24W38on 13:30UTC, June 18

Cone model fitting

Propagation direction : S21W32 ,S09W04Angular width : 194 degree,148 degreePropagation velocity: 630 km/s ,850 km/s

CME observation

M2 flare erupted from AR2371 (N13W00)on 01:42UTC, June 21

Case study : June 18 and 21 ,2015

IMF at Earth

DVE at Earth

Density and magnetic field

Pressure and magnetic field

Predicted Shock Arrival Time Difference (hrs) Confidence (%) Method

06-20T20:00Z -19.33 SEPC model

06-21T02:20Z (-4.53h, +7.12h) -13.33 31.0 Ensemble WSA-ENLIL + Cone (GSFC SWRC)

06-21T21:008Z 5.33----

WSA-ENLIL + Cone (NOAA/SWPC)

06-21T09:26Z (-7.0h, +7.0h) -6.23 31.0 WSA-ENLIL + Cone (GSFC SWRC)

06-21T08:00Z (-12.0h, +12.0h) -7.67 40.0 Other (SIDC)

06-21T16:00Z (-12.0h, +12.0h) 0.33 70.0 WSA-ENLIL + Cone (Met Office)

06-21T11:21Z -4.32 43.0 Average of all Methods

Case study : June 18 ,2015

Data from CCMC

Predicted Shock Arrival Time Difference (hrs) Confidence (%) Method

06-22T16:00Z -1.98 SEPC model

06-22T17:00Z (-12.0h, +12.0h) -0.98 90.0 Other (SIDC)

06-22T21:00Z 3.02----

WSA-ENLIL + Cone (Met Office)

06-22T21:43Z (-7.0h, +7.0h) 3.73 100.0 WSA-ENLIL + Cone (GSFC SWRC)

06-22T19:03Z (-5.15h, +3.33h) -1.07 100.0 Ensemble WSA-ENLIL + Cone (GSFC SWRC)

06-22T23:00Z (+7.0h) 5.02 100.0 DBM

06-22T22:50Z (-5.0h, +8.0h) -4.85 ---- ElEvo

06-22T14:00Z -3.98 ---- WSA-ENLIL + Cone (NOAA/SWPC)

06-22T19:48Z 1.82 97.5 Average of all Methods

Case study : June 21,2015

Data from CCMC

Case study : August 12 and 14 ,2015

CME observation

Associated with prominence erupted on 13:42Z, August 12on 13:30UTC, June 18

Cone model fitting

Propagation direction : N28E34 ,N34W00Angular width : 170 degree,34 degreePropagation velocity: 570 km/s ,490 km/s

CME observation

Associated with filament erupted below AR2399 (S07W47) on 06:30Z, August 14

Case study : August 12 and 14 ,2015

IMF at Earth

DVE at Earth

Density and magnetic field

Pressure and magnetic field

Predicted Shock Arrival Time Difference (hrs) Confidence (%) Method

08-14T23:00Z -8.72 SEPC model

08-16T04:00Z (-12.0h, +12.0h) 20.28 20.0 Other (SIDC)

08-16T09:09Z (-7.0h, +7.0h) 25.43----

Ensemble WSA-ENLIL + Cone (GSFC SWRC)

08-16T03:00Z 19.28 ---- WSA-ENLIL + Cone (NOAA/SWPC)

08-16T06:02Z (-7.0h, +6.66h) 22.32 91.0 WSA-ENLIL + Cone (GSFC SWRC)

08-16T00:01Z (-6.0h, +6.0h) 16.3 70.0 WSA-ENLIL + Cone (Met Office)

08-16T04:26Z 20.72 60.33 Average of all Methods

Case study : August 12 ,2015

Data from CCMC

Predicted Shock Arrival Time Difference (hrs) Confidence (%) Method

08-19T00:00Z(SWIDM: Will not arrive! )

---- SEPC model

08-18T12:00Z (-7.0h, +7.0h) ---- 10.0 WSA-ENLIL + Cone (GSFC SWRC)

08-18T05:00Z --------

WSA-ENLIL + Cone (NOAA/SWPC)

08-19T12:00Z (-12.0h, +12.0h) ---- 20.0 Other (SIDC)

08-18T18:00Z (-12.0h, +8.0h) ---- 40.0 WSA-ENLIL + Cone (Met Office)

08-18T17:45Z ---- 22.3333 Average of all Methods

Case study : August 14 ,2015

Data from CCMC

Difference (hrs) SEPC SWPC SWRC MET SIDC

March 15, 2015 -5.08 -13.92 7.57 7.92 -10.8

April 4, 2015 -33.17 -47.97 -28.17 -25.17

April 6, 2015 -24.0 -11.0 -15.7 -9.5 -6.0

May 6, 2015 21.0 -4.42 12.0

May 2, 2015 -11.83 15.17 41.17 3.17

June 9, 2015 -20.0 -1.92

June 18, 2015 -19.33 5.33 -6.23 0.33 -7.67

June 19, 2015 -4.85 16.15 1.22 21.15 3.15

June 21, 2015 -1.98 -3.98 3.73 3.02 -0.98

June 22, 2015 -9.95 10.05 5.35 8.05 -0.95

June 25, 2015 -20.5 11.5 22.5 1.5 -11.5

July 19, 2015 -31.0 4.5 2.5 5.5 -0.5

August 12, 2015 -8.72 19.28 25.43 16.3 20.28

Sep 04, 2015 -6.47 18.53 69.53 34.53

Sep 18, 2015 5.55 15.55 11.78 6.55 25.55

Oct 22, 2015 0.56 6.57 -6.62 2.57 -6.43

Nov 4, 2015 5.43 21.43 15.10 19.43 10.43

Absolute Mean 13.50 11.88 13.98 13.51 11.75

Case study: 17 CMEs

Lessons Learned

Research models can not be put into operational use

directly without evaluation, verification, validation and

some necessary modification

– Research models and operational model focus on

different targets (science question vs forecast accuracy)

– There may be several codes exist for a specific problem

(i.e., functions relating flux tube expansion factor to the

solar wind speed). Verification and Validation are needed

to choose the best (or the proper) one.

– Models should be optimized when driven by real time (or

quick-look) data rather than science data.

– Some models failed to meet the requirements of the

downstream models, which may significantly influence

the success of the whole project (such as the CME auto-

detection tool). Automation also brings some problems.

– Modifications are needed to connect different models (the

HAF resolution is changed then optimized codes are

needed to meet the requirements by SEP simulation).

Lessons Learned

Sustainable interaction between research community

and operation community is needed.

– Defining & developing what’s needed is non-trivial.

– Modelers may be unclear about what forecasters want.

Forecasters know exactly what are the most urgent

requirements in operational services.

– Forecasters may don’t know what models can (or could)

do, or the circumstances under which the models are

applicable.

– Forecasters do comprehensive verification and validation

works, focusing on the operational usage. The result

needed to be fed back to the modelers to optimize the

model.

– Iteration is required to derive good forecast products.

A “transition team” approach is workable (forecasters,

computation experts, scientists, managers).

Thanks for you attention.