Randy Keyworth

Jack States

Ronnie Detrich

The Wing Institute

Professional Judgment: Professional Judgment:

Inevitability,Inevitability,

and Manageabilityand Manageability

Fallibility,Fallibility,

What is professional judgment?What is professional judgment?

Also known as…………..

informed clinical opinion

professional wisdom clinical reasoning

clinical expertise decision making

clinical decision making problem solving

clinical judgment data interpretation

What is professional judgment?What is professional judgment?

Critical interface between input and output when making clinical decisions.

Input information data research

Output strategies interventions treatment

What is professional judgment?What is professional judgment?

It is also…………

a complex set of behaviors governed by individual learning histories subject to complex contingencies

Why is professional judgment part of the Why is professional judgment part of the hierarchy of evidence? hierarchy of evidence? MedicineEvidence-based medicine is the integration of best research evidence with clinical expertise and patient values.

David L. SacketCentre for Evidence Based Medicine

PsychologyEvidence-based practice in psychology (EBPP) is the integration of the best available research with clinical expertise in the context of patient characteristics, culture, and preferences.

American Psychological Association

EducationThe integration of professional wisdom with the best available empirical evidence in making decisions about how to deliver instruction.

Grover J. Whitehurst, Assistant Secretary, United States Department of Education

social validitysocial validity

Why is professional judgment part of the Why is professional judgment part of the hierarchy of evidence? hierarchy of evidence? social validitysocial validity

Professional judgment is ingrained in virtually all professional standards and is a universally accepted component of professional skills.

Professional judgment is the age old “trusted method”, generally accepted, and expected by consumers.

Professional judgment has tremendous influence in shaping behavior.

Why is professional judgment part of the Why is professional judgment part of the hierarchy of evidence? hierarchy of evidence?

Everyone relies upon professional judgment when making clinical decisions…

when scientific research evidence is absent, incomplete or conflicting in conclusions

when scientific research evidence is present (review and interpretation… single subject research)

when responding to the complexity and uniqueness of behavior

inevitabilityinevitability

Why is professional judgment part of the Why is professional judgment part of the hierarchy of evidence? hierarchy of evidence? inevitabilityinevitability

Professional judgment is used in making decisions on critical components of behavioral interventions.

goals / prioritiesassessment strategiesinterventions

data interpretationremediation

Professional judgment is used in countless moment-to-moment decisions.

Why is professional judgment part of the Why is professional judgment part of the hierarchy of evidence? hierarchy of evidence?

Experience with types of interventions, problems, strategies with specific environments, individuals

Flexibility when the data is limited or absent when dealing with unique situations

Inexpensive compared to costs of research

Immediate often can’t wait for research to be conducted

functionalityfunctionality

The fallibility of professional judgment: The fallibility of professional judgment:

Professional Arrogance: ignoring the fact that mistakes are inevitable

Art vs. Science: under and over-reliance on science

Lack of Feedback: absence of accurate feedback

a false sense of accuracya false sense of accuracy

The fallibility of professional judgment: The fallibility of professional judgment: a false sense of accuracya false sense of accuracy

“art vs. science”

decisions made on the basis of intuition

empirical data not only waste of time, diminishes quality of service…it limits creativity and spontaneity

professionals “trust” their professional judgment

The fallibility of professional judgment: The fallibility of professional judgment: a false sense of accuracya false sense of accuracy

77% 22%Your intuition (gut feeling) about what will be effective

39% 92%

Your demonstrated track record of success based on data you have gathered systematically and regularly

37% 92%Results of controlled experimental Studies

CRITERIA FOR TREATMENT CHOICES CLIENT PHYSICIAN

Gambrill and Gibbs, 2002

The fallibility of professional judgment: The fallibility of professional judgment: a false sense of accuracya false sense of accuracy

“science vs. art”

failure to recognize the limitations of existing scientific research

failure to recognize the role and fallibility of professional judgment

The fallibility of professional judgment: The fallibility of professional judgment: a false sense of accuracya false sense of accuracy

Lack of Feedback

lack of systematic feedback from peers or colleagues

lack of organizational systems for providing feedback on outcomes related to decisions

difficulty with “complex systems” to ascertain cause and effect

The fallibility of professional judgment: The fallibility of professional judgment:

Decision makers are subject to contingencies that shape behavior

lack of time, resources to stay up on literature and research individual cases

external contingencies (culture, organization, systems, consumers, professionals, regulations)

lack of standards for professional judgment

conflicting contingenciesconflicting contingencies

The fallibility of professional judgment: The fallibility of professional judgment:

Biases that can interfere with judging progress and causation

Being swayed by hindsight

Being overconfident

Engaging in wishful thinking

Having an illusion of control

Critical Thinking in Clinical Practice : Improving the Quality of Judgements and Decisions, Eileen Gambrill

biasbias

The fallibility of professional judgment: The fallibility of professional judgment: bias bias

Biases that can interfere with judging progress and causation

Overlooking the role of chance (coincidences)

Overlooking confounding causes, such as regression effects

Attributing our success to our own attributes and failure to other factors

Seeking only data that support preferred views

Critical Thinking in Clinical Practice : Improving the Quality of Judgements and Decisions, Eileen Gambrill

The fallibility of professional judgment: The fallibility of professional judgment: bias bias

Biases that can interfere with judging progress and causation

Relying on observed rather than relative frequency

Overlooking the interaction between predictions and their consequences

Mistaking correlation for causation

Relying on misleading criteria such as testimonials

Critical Thinking in Clinical Practice : Improving the Quality of Judgements and Decisions, Eileen Gambrill

The fallibility of professional judgment: The fallibility of professional judgment: errors in reasoningerrors in reasoning

Common errors in reasoning that can effect perceptions and

decisions.

Circular Reasoning

Non-Sequitur

Post Hoc

Red Herring

Equivocation

False Dichotomy

Lying

Authority

Shifting the Burden of Proof

Self-Referential Fallacy

Ad Hominem

Sidestepping/Avoiding the Question

Suppressed Evidence (Stacking the Deck)

Statistics

Jumping to Conclusions

Traditional Wisdom

Analogy

Humor

Extrapolation

Circumstantial Evidence

Straw Man

Guilt by Association

Best-in-Field Fallacy

The fallibility of professional judgment: The fallibility of professional judgment: propagandapropaganda

Systematic manipulation of information

ad hominum

ad populum

bandwagon

big lie

card stacking

doublespeak

either / or

fear

glittering generalities

media manipulation

opinion as fact

plain folks

repetition

scapegoating

testimonial

transfer

The management of professional judgment: The management of professional judgment: raising your batting average raising your batting average

learn about sources of error

develop strategies for minimizing mistakes

track outcomes from your decisions

modify decisions according to data

create environmental contingencies for feedback,continuous learning, and reviewing research

maintain a healthy sense of skepticism …… and humility

Evidence-based special Evidence-based special education: Why now?education: Why now?

Tracking the term “autism cure” in the Google search engine…

August ’04: 308,000 results

February ’05: 528,000 results

August ’05: 1,830,000 results

February ’06: 5,290,000 results

April ’06: 15,200,000 results

Evidence-based special Evidence-based special education: Why now?education: Why now?

Tracking the term “autism cure” in the Google search engine…

5,290,000

1,830,000

528,000308,000

15,200,000

Aug 2004 Feb 2004 Aug 2005 Feb 2006 April 2006

GAP Analysis: Adequate GAP Analysis: Adequate ResearchResearch

Current Conditions: There is little agreement on what constitutes research

EXPERIMENTAL• Randomized Trials (Group Design) • Single Subject (Individual Design)

QUASI-EXPERIMENTAL • Non-Random Trial (Group Design)

DESCRIPTIVE• Correlational Research• Simple Descriptive• Comparative Descriptive• Case Study

QUALITATIVE• Case Studies• Interviews• Ethnography• Participant Observation• Document / Artifact Analysis

OPINION• Professional Opinion• Testimonials• Anecdotal

Why is professional judgment part of the Why is professional judgment part of the hierarchy of evidence? hierarchy of evidence? inevitabilityinevitability

Professional judgment is critical in making sound decisions…

when addressing client-centered outcomes (patient values)

“by patient values we mean the unique preferences, concerns and expectations each patient brings to a clinical encounter and which must be integrated into clinical decisions if they are to serve the patient” David Sacket

(not always “ideal solutions”…context of “consumer” resources, biases, choices, preferences)

The fallibility of professional judgment: The fallibility of professional judgment: biasbias

Questionable Criteria authority

status is correlated with accuracy

popularity and numbers acceptance of claims simply because many people accept

them

tradition what has been done in the past

newness being swayed by the “latest intervention”

manner of presentation swayed by style, persuasive style