Expert Operator Decision Making

Introduction

EMS User Conference

Prepared By

Robin Podmore, IncSysRobert Eubank, WECCGreg Lange, GCPUDFrank Greitzer, PNNL

Outline

• State of Applications• What is an expert?• Knowledge Capture and Transfer• Engineer Operator Gap• Expert Operator Decision Model (ODM)• Simulator Configurations• Conclusions

NORMAL

RESTORATIVE EMERGENCY

INSECURE

Preventative Action

Remedial Action

Island Blackout

Outage, Load Increase

Restoration Action

Power System States

State of Applications

• Normal and Insecure State– State Estimator– Operator Power Flow– Contingency Analysis – limited deployment– Security Constrained Economic Dispatch– Automatic Voltage Control

• Emergency State– Load shedding– What Other?

• Restoration State– What?

Objectives

• Report on past and planned advances in operator training since Blackout of 2003.

• Focus on understanding how an expert operator makes decisions.

• How can this understanding be applied to development of improved user interfaces and applications for insecure, emergency and restoration states?

Presenters

• Greg Lange, Grant County PUD• Robert Eubank – Training Instructor, WECC• Dr. Frank Greitzer – Scientist, PNNL• Robin Podmore - IncSys

ExpertsWhat is an “Expert”?

“Someone widely recognized as a reliable source of technique or skill whose faculty for judging or deciding rightly, justly, or wisely is accorded authority and status by the public or their peers. An expert, more generally is a person with extensive knowledge or ability in a particular area of study.”

Source: Wikipedia, 2008

• Stage 1 Novice – Explicit rules susceptible to context and nuance• Stage 2 Advanced Beginner – Nuance and context begin to be recognized

and incorporated• Stage 3 Competence - Transition from calculated effort to intuitive solutions• Stage 4 Proficiency – Scenarios are now being recognized as whole parts,

some analysis and conscious choice remains

• Stage 5 Expertise – Complete contexts are recognized and performance is fluid and unselfconscious.

Levels of ExpertiseDreyfus and Dreyfus 1986

•Dreyfus, H. L., & Dreyfus, S. E. (1986). Mind over machine: The power of human intuition and expertise in the era of the computer. New York: Free Press.

Novices vs Experts

Read

Analyze

Explore

Plan

Implement

Verify

Novice Pattern Expert Pattern

Explicit and Tacit Knowledge

• The knowledge in your operating department can be categorized as Explicit knowledge and Tacit knowledge.

• Explicit knowledge is formal and systematic. It can be easily communicated and shared, in operating procedures, disturbance reports, databases or a computer program.

• Tacit knowledge is highly personal. It is hard to formalize and difficult to communicate to others.

• “We know more than we can tell”. • Tacit knowledge is deeply rooted in action, experience,

and operator’s commitment to his profession and craft.

Tacit to Tacit Knowledge Transfer

• Sometimes one individual shares tacit knowledge directly with another.

• For example, when an operator in training (OIT) works side by side with an experienced operator through on the job training.

• He learns his tacit skills through observation, imitation and practice. But on its own, socialization or tribal learning is a rather limited form of knowledge creation.

• The apprentice learns the master’s skills. But neither the apprentice nor the master gain any systematic insight into their craft knowledge.

• Because their knowledge never becomes explicit, it cannot easily be leveraged by the organization as a whole.

Explicit to Explicit Knowledge Transfer

• An individual can also combine or apply discrete pieces of explicit knowledge into a new whole. For example, an operator may apply some well documented switching principles to develop a specific switching order.

• Compilation all the key papers on Restoration into the IEEE “Power System Restoration” Book.

• Development of Scenarios to illustrate operating principles in NERC Manual

From Tacit to Explicit

• The development of Explicit knowledge from Tacit knowledge is often one of the more difficult transformations.

• With Simulation the process becomes systematic. Scenarios provide a context for the master to express his knowledge. They stimulate the master’s memory on past operating incidents.

• When the master operator is presented with a challenging operating condition on the Simulator; he uses his tacit knowledge and experience to develop a specific set of operating actions.

• Actions of the master operator and how he responds to solve various problems can be recorded

• When the operating principles that the master operator was intuitively applying in developing and selecting these operating actions, can be documented and explained, this knowledge can be shared with other operating staff.

From Explicit to Tacit

• As new explicit knowledge is shared throughout the operating department, other operating staff begin to internalize it – that is, they use it to broaden, extend and reframe their own tacit knowledge.

• They adapt the procedures and principles that they have learned and observed under one set of operating conditions and apply them to other operating conditions.

• These operators eventually come to take it for granted as part of the background of tools and skills needed to do their jobs.

Building a Spiral of Knowledge

• Articulation (converting tacit knowledge to explicit knowledge) and

• Internalization (using that explicit knowledge to extend one’s own tacit knowledge base)

• These are the critical steps in building a spiral of knowledge.

• Both require the active commitment of the individual. • The master has to stretch and grow to communicate more

of what he knows. • The apprentice has to stretch and grow to apply the

explicit knowledge and extend his own operating skills.

Source: SOS International

The Engineer – Operator Gap• “Stuff You Don’t Learn in Engineering School – Skills

for Success in the Real World, by Carl Selinger, IEEE Press, 2004.”

• “Book smart and dumb to the world”• “I can train an operator to be an engineer, I can’t train

an engineer to be an operator.”• “Four year graduate engineers are not recruited for

system operators.”• “The rigorous technical Engineering curricula does

not allow time for us to teach soft skills.”• “New graduate engineers will not work shifts.”• “Applications designed by engineers for engineers.”• “Our EMS simulator is an engineers dream but an

operators nightmare.”

Engineer – Operator Gap

• “Engineering Schools do an inadequate job at producing well-rounded engineers with background courses in English, writing, economics, history and so on.”

• “I have fifty engineers and fifty operators. The engineers, cannot make a real-time decision.” Chief System Operator of a major RTO.

• “MIT is coming to ask us on how they can train there engineers to be leaders. Industry is telling them that there graduates cannot handle operations.” Dean, United States Naval Academy.

• “The IEEE Working Group Report is not readable by operators”

Critical SkillsSoft Skills

Broad business understanding

Financial understanding

Interpersonal skills

Strategic thinking and planning

Development

Communications

Decision making

Curiosity

Emotional IQ

Learning Agility

Calm under fire

– Experience / Technical Skills Power Dispatcher System Operator Relay Technician Underground Technician Transmission Expertise System Planning Engineering Design Engineer Electric Service Line Workers Gas Pressure Control Gas customer Field Service

Source: Wanda Reder, IEEE

Important Traits

SourceJ. Theotonio NERC

Crew Resource ManagementFoundation for Soft Skills

• Situational Awareness• Assertiveness• Decision Making• Communication• Leadership• Adaptability/Flexibility• Mission Analysis

How the Military Bridges the Gap between Officers and Senior

Enlisted

• Military Academies: One Senior Enlisted per Company.

• Prior enlisted can become officers.• Platoon Lieutenants and Sergeants work

closely with mutual respect.

Naturalistic Decision Making

• Typical industry training doesn’t match the way people think. Effective training for critical decision making must be informed by cognitive theories and models.

• Naturalistic Decision Making (NDM) is an area of cognitive science that describes how experienced individuals and teams perform in real-time, mission-critical environments.

• First NDM conference held in 1989.• Ninth NDM conference Covent Gardens June 1990

Example Applications of NDM

• Health Care – Surgeons, Nurses• Command and Control• Aviation• Business and Industrial Applications• Process Control• Material Requirements Planning• Nuclear Power Plant Emergencies• Anesthesiology• Naval Officers in Littoral Environments• Skilled Fighter Pilots

The Challenge of Decision MakingThe Challenge of Decision Making

High StakesHigh

Stakes

DynamicSettingsDynamicSettings

Time StressTime Stress UncertaintyUncertainty

Organizational Factors

Organizational Factors

Vague Goals Vague Goals

Multiple Players

Multiple Players

High Task Loading

High Task Loading

Critical Decision Making

NDM Thought Leaders

• Gary Klein – Recognition Primed Decision Model• Mica Endsley – Situation Awareness• Marvin Cohen – Consistent, complete, validated

“Story” with Quick Test• Sallie Gordon – Cognitive Task Analysis

Recognition Primed Decision ModelDeveloped by Gary Klein

Situation

Cues

ActionScript

MentalSimulation

MentalModels

That affect the

That let youcreate

That activate

Generates

Which youAssess by

Using your

Patterns

NDM Applications to Power

• RPDM introduced to Power Industry by Doug Harrington of Team Formation

• RPDM extended by Robin Podmore of IncSys, Frank Greitzer of PNNL and Chuck Johansen and Pam Eye of SOS to reflect:– Need for 100% Error Free Decisions – Quick Test– Distinction between Long and Short Term Memory– Endsley’s Three Levels of Situation Awareness

‹34›

Operator Decision Model

Situation

Cues

That affect the

That let youcreate

That activate

Generates

Which youAssess by

Using your

Which youValidate by

Using your

TimeAvailable?N

MentalSimulation

MentalSimulation

ActionScript

Real World Short termmemory

Long termmemory

Story(Patterns)

MentalModels

MentalModels

Y

‹35›

‹36›

Simulator Configuration Options

Generic Power System Model

Control CenterModel

EmulatedSCADA/AGC

ReplicatedEMS

Generic Simulator

Custom Simulator

ReplicaEMS

Simulator

GenericSCADA/AGC

Specific Power System Model

Specific Power System Model

Power SystemModel

Controls

Status, Analogs

Controls

Controls

Status, Analogs

Status, Analogs

‹37› 2008 © Copyright Incremental Systems Corporation

EXTERNAL

CHSAN UXBRDG COPMANOR COPLEY STNTON OAKDLE

AMUS

POOL

BEAVERASH

LOCHER

MOSES

JENKINHOMERVEXLEY

WYNHAM

ELLS

FARLIE

GRANGE

DAWSON

TANTON

RICTER

KINCAID

DOYLE

200

3*200500600

1200

702*400600

1200

BAKER

CRWFRD

AIRPRT NESTLE

230 KV

138 KV

69 KV

33 KV

80

400

‹38› PALCO Map

‹39›

Generic Simulator Advantages

• Wide Variety of Scenarios, developed and shared by industry leading instructors

• Students can learn to operate PALCO in a single training session

• Low cost method to obtain NERC mandated: “five days per year of training and drills using realistic simulations of system emergencies”

• Catalyst for capturing knowledge of experienced operators.• Clearing house for simulating incidents on real systems• Used by more than 50% certified operators

‹40›

PJM: Four Simulation Sessions - Four teams per session – Four operators per team – 36 logged on

Users

‹41›

Generic Simulators can be used torecruit new engineers and operators

‹42›

PG&E CETAC Drills (2009)

2009 Copyright Incremental Systems Corporation

• 5 weeks• 500 total operators• All major utilities in California• Generic PSM for principles• Custom PG&E Model and Northern California Area for Daylong drills• Drill based on report of earthquake off the coast of N. California

‹43›

‹44›

General Process

•Industry Standard Format: PSSE or CIM XML

•Rapid Building Process

•One-line diagrams automatically built

•Retains important aspects of external modeling, while

reducing the overall model size

•Recognizable Station names

•Electrically Sound model

•Great for verifying blackstart plan

•Easily customizable for system operator training

PSSE

“Internal” Area

“External” Area

200 KV and Above

Below 200 kV

PowerSimulator CIM Model

define

Retain All

Data

Retain All

Data

EquivalenceAnd Move

AutoBuilderAutoBuilder

‹45›

Station Built Automatically from PSSE Data

• Single Bus

• Single Breaker• Lower Voltage (under 200 kV) Equivalenced in the external area• EOP-005 Compliance ready

‹46›

Station Customized from PSSE Data

• Familiar layouts

• Custom Names

• Familiar Switching

• Transfer Busses

• Suitable for:•PER-002•PER-005•Blackstart Drills

‹47›

External Station Built Automatically fromPSSE Data

‹48›

Effect of External WECC Model Reduction

PowerSimulator performance with the reduced model is very good

‹49›

Internal Areas For WECC Models

‹50›

Custom Simulator Advantages

• Custom models can be developed as an application service.

• Dedicated support engineer is not required• Model is customized to represent operator’s own power

system.• Complies with PER 005 requirement – “Each RC, BA, TO

…. shall provide each System Operator with emergency operations training using simulation technology …. that replicates the operational behavior of the BES …”

• Clearing house for simulating incidents on real systems.• Need to OJT to cover differences in Custom Simulator

user interface and EMS user interface

‹51›

Alberta Electric System Operator Restoration Drills

2009 Copyright Incremental Systems Corporation

flash 360

http://www.powersimulator.net/?m=psmCENisku.html

‹52›

Types of Knowledge and Skills• Knowledge of power system equipment

and principles• Knowledge of Operating Procedures –

NERC, Region, Utility• Technical skills at operating A power

system• Technical skills at operating YOUR

power system

• Soft skills: team work, communications, multi-tasking, managing stress and uncertainty

• Knowledge of departments/personnel you need to work with.

• Knowledge and Skills for operating your EMS and related communication equipment

Suited for self study and Virtual Instructor

‹53›

Ready with PSD for a Day at School(We were totally safe and protected all the time)

‹54›

Iraq Ministry of Electricity National Dispatch Center

2009 Copyright Incremental Systems Corporation

• Located Al Ameen, Iraq• New and Incumbent Engineers• Generic PSM for principles• Custom Iraq Model• Remedial Action Schemes

‹55›

New Processes for Operator Training

• NERC PER 002: – Requirement for 5 days of emergency operations training with

simulators

• NERC PER 005: – Requirement for Job Task Analysis

– Requirement for emergency operations training using simulation technology …. that replicates the operational behavior of the BES …”

• Wide acceptance of Generic Simulators• Increasing acceptance of Custom Simulators• A Model for Expert Operator Decision Making• Cognitive Task Analysis for System Operators

‹56›

Can new methods by applied to System Development?

• Job Task Analysis• Generic Simulators • Custom Simulators• Model for Expert Operator Decision Making• Cognitive Task Analysis?

• Improved Tools for Normal and InSecure State• New Tools for Emergency State• New Tools for Restoration State