Exploring psychological treatment

mechanisms using data from TAU

Dr Andy Fugard UCL Research Department of Clinical, Educational & Health Psychology

8 Jan 2014, University of Kent, EBM+ Workshop:

Evidence of mechanisms in evidence-based medicine

Mental health conditions

Picture from Blakemore and Mills (2014) using data from Kessler et al (2005)

Story of best practice:

1. Theory derived from

“basic science”, e.g.,

experimental psychology

2. Develop and manualise

treatments

3. Demonstrate efficacy

using RCTs

4. Systematic reviews

5. Implement whatever

works (EBP)

(How true is this tale…?)

Efficacy effectiveness via

Routine Outcomes Monitoring (ROM)

• RCTs are often run in very special contexts

• ROM evaluates effectiveness:

whether treatments work in “the real world”

• “Practice-based” evidence

• Usually multi-use data:

– Formal assessment checklists used as a clinical tool

– Progress monitoring (e.g., weekly)

– Various data for audit, driven by national datasets,

“key performance indicators”

The plan for this talk

Let’s take seriously the idea

that all data need to be

interpreted (not obvious from KPIs!)

What (if anything?) can we learn about mechanisms

using data from TAU*

* Spoiler: not completely answered in this talk!

Routine outcomes monitoring in the UK (Macdonald & Fugard, submitted)

YP-CORE (Twigg et al., 2009)

… two sides A4 …

Strengths and Difficulties Questionnaire

(SDQ; Goodman & Scott, 1999)

A challenge

WHAT DO PSYCHOLOGICAL

MECHANISMS LOOK LIKE?

Narrowing scope… the/a cognitive tradition

• Informational analysis following Neisser (1967): “…all processes by which the sensory input is transformed,

reduced, elaborated, stored, recovered, and used. It is concerned

with these processes even when they operate in the absence of

relevant stimulation … cognitive psychology is concerned with all

human activity rather than some fraction of it, the concern is from a

particular point of view”

• Cognition here doesn’t mean “conscious thought”

• There are ripples of debate, e.g.,

– Rowland (2010), Adams & Garrison (2012)

– “Cognitive processes ain’t (all) in the head!”

(Clark & Chalmers, 1998)

Marr’s (1982) multiple levels

Specification of input-output relations

Algorithms

Neural implementation

- What’s the system doing?

- What information is represented?

- How is it transformed? (Step-by-step.)

- How does it look in biology?

Lessons from the psychology of reasoning

• Mental models

• Mental rules/logic

• “Non-logical” Bayesian models

• Probability logics

• …

Often/usually difficult to distinguish representations

and processes empirically but something algorithmic

can be extracted (occasionally!)

What does biology look like?

Levels all the way down…

Multiple levels common in engineering, e.g.,

Open Systems Interconnection model (OSI)

Sun, Coward and Zenzen’s (2005) extension

Mental rules in cognitive therapy (Beck, 1976)

Procedures and mechanisms (Power 2010)

Variations of exposure (Telch et al, 2014)

Illustrative example study of mechanisms

(Goldin et al, 2014)

• CBT for social anxiety vs. waiting list

• CBT included

– “psychoeducation” (teaching the mechanism!)

– graduated exposure to feared social situations, within

session and as homework

– reappraising/reinterpreting situations

– uncovering and challenging negative core beliefs

Tasks: key tools for uncovering mechanisms

Self-report results

fMRI results

• (Long tables of differences between people and

conditions in BOLD signal)

• Example interpretation:

“The reappraisal-related right SFG region found in

this study overlaps with the medial PFC and has

been implicated in multiple executive cognitive

functions, including monitoring, mentalizing,

introspection, self-referential meta-cognitive

evaluations, and reappraisal of emotional

reactivity”

Loops: the mechanisms of reasoning about

mechanisms with patients

• Clinicians explain the theory to patients

• Jointly reason to a “formulation” including

– What’s up?

– Why me?

– Why now?

– Why still?

– What helps?

Template

(Dummett, 2010)

Example

(Dummett, 2010)

Causal maps – example drawn by a clinician

(De Kwaadsteniet et al 2010)

BACK TO ROUTINE OUTCOME

MONITORING

What works for whom?

• Defined in terms of self-, carer-, or clinician-

completed questionnaires

• Problem of regression to the mean and other

forms of non-treatment-mediated change partly

solved by using simulated control data

• No randomisation and matching imperfect

still problem of unknown variables

Time 1 Time 2

Avera

ge d

istr

ess

(Cooper, Fugard, Pybis, McArthur, & Pearce, in press)

Counselling

Estimated non-intervention

outcome (ENO)

Estimated

Effect

Outcome

Diagnosis

Treatment

?

Child Outcomes Research Consortium

• Founded 2002 by

clinicians and service

managers

• Members (service

providers) pay an

annual fee

• Over half UK CAMHS

are members + parts of

Scandinavia

Analysis of CORC data (Fugard, Stapley, Ford, Law, Wolpert & York, 2014)

School counselling data (work with Mick Cooper)

School counselling data (work with Mick Cooper)

What seems to cause outliers…?

Mechanisms lurking here?

• Data quality

• Return rates

• What’s covered in the

questionnaire

• Chosen outcome

variable

• Presenting problems

• Organisational factors

• Stage in episode of

care

• Therapeutic modality

• Therapist skill

• Goals set with young

person/family

Sketch of ongoing work

• Using data from a large NHS Foundation Trust

• Cases selected with a diagnosis of anxiety

• “Outcome” (end of treatment) based on clinician-

rated pre-post change

• Cases selected showing little vs. lots of change

• Case notes now being investigated using a

“thematic analysis”

WHAT’S THE TREATMENT?

Clinical coding in physical health

Example 1

– ICD-10 O300 (twin pregnancy)

Z37.2 (twin both live born)

– OPCS-4 R17.2 (elective lower uterine

segment caesarean delivery)

Example 2

– ICD-10 S7200 (fractured neck of femur)

W19.0 (unspecified fall at home)

– OPCS-4 W37.1 (primary total prosthetic

replacement of hip joint using

cement)

Z94.3 (left sided operation)

Children and young people’s mental health

(CYP IAPT) – long list but little detail

Procedure taxonomies, e.g., Common Language

for Psychotherapy (CLP) procedures

http://www.commonlanguagepsychotherapy.org

Example content of case notes

• Who’s involved, in sessions, in patient’s life

• Contact outside session, e.g., with school

• What the patient wants, e.g., parent there or not

• What the clinician did

• Session content, e.g., psychoeducation, breathing

exercises, giving/discussing homework…

• Clinician’s speculations about what the young

person understood the treatment

• Supervision discussions

• Also what hasn’t been mentioned in the notes

Gluing this all together:

any representations and algorithms…?

• TAU data can tell us about the elements of an

intervention, e.g., people involved, procedures

• The sequence of events

• Duration of phases of treatment

• The recorded wishes of patients and something

about to what extent wishes were granted

Instead of dichotomous can we vs. can’t we, think,

what kinds of mechanism can we reason about?