BABCP Annual Conference, Exeter, July 17th, 2009

THE EVIDENCE-BASE FOR CBT SUPERVISION:

SYSTEMATIC REVIEWS OF THE EMPIRICAL LITERATURE

DEREK MILNE, ALIA SHEIKH; SUE PATTISON; ANDREW WILKINSON

19/04/23 2

Background

Clinical supervision training regarded as essential.

Yet training not been given systematic attention (“Something does not compute”: Watkins, 1997) : concerns that evidence-base generally weak, etc.

One way forward: selective (BES) systematic review of controlled studies.

Best evidence synthesis (BES): a novel, affirming approach to the systematic literature review:

BES “combines the meta-analytic approach of extracting quantitative information in a common standard format…with a systematic approach to the assessment of study quality and study relevance” (Petticrew &Roberts, 2006, p.181);

Often used in education, to answer questions about what works (pragmatic focus);

Careful selection of studies - Milne et. al. 2006: 9 inclusion criteria, inc: ‘CS manipulated;

demonstrated effectiveness; in scientific journal’.

19/04/23 4

Objectives

To assess the degree of empirical support for supervisor training

To define training methods that are supported in the selected literature

To specify recommendations for the training of supervisors

19/04/23 5

Methodology Conducted systematic literature search of three

databases (MedLine, PsycInfo, ASSIA) Key words: “clinical supervision,” and “clinical

supervisor” Narrowed search using term “empirical” Studies selected according to specific inclusion

criteria, based on NICE(R) Manual for Systematic Reviews of the Clinical Supervision Literature (Version 11; Milne & James, 2000)

Final sample: 11 studies (N= 145 supervisors; 251 supervisees; 147 clients)

19/04/23 6

Methodology

Studies also evaluated according to NICE(R) (2001) coding manual

Coding system allows analysis of the methodological quality of studies (e.g., use of measures, threats to validity, etc) & of the methods (Independent variables)

2 studies independently coded by researchers, 1 at start (inter-rater agreement=81%) & 1 at end (82%)

19/04/23 7

Results: 2 Questions

Addressing the two questions posed by Whitman, Ryan & Rubinstein (2001):

What empirical support is there for supervisor training?

Which elements of such training are supported by evidence?

19/04/23 8

Q1: What empirical support is there for supervisor training?

11 studies provided empirical support for supervisor training (e.g. they had sound methodological validity and an average 67% impact on the 145 supervisors’ learning)

15 training methods- primarily: corrective feedback (all 11 studies); educational role-play & observational learning (both in 8); teaching (in 5 studies: NB: next slide arranged by frequency of methods use in training)

19/04/23 9

Q2: Which elements of such training are supported by evidence?

Feedback Educational role-play Modeling (live/video

demonstration) Teaching (verbal

instruction) Written assignments Behavioural rehearsal Providing a rationale Guided reading

Discussion Educational needs

assessment Direct observation Questions and answers Agenda-setting Homework

assignments Quiz

These methods were blended, to be:

Multiple (mean 5.1 per study);Structured (address training cycle);Complementary (experiential learning cycle)

Structured approach recommended

Loganbill and Hardy (1983): 3 necessary components of training: theoretical content, simulated experience, and in vivo practise with supervisees;Russell and Petrie (1994): combine didactic and experiential (“critical element in an effective supervisory training programme” ; p. 39). More recent reviews agree:Steinert et al.,2006 (systematic review; faculty); Falender et al., 2004; Kaslow et al., 2004 (consensus).

Example: Bambling, et.al. (2006), Psychotherapy Research, 16, pp. 317-331.

N=40 supervisors (RCT: CBT or Dynamic;

N= 127 therapists & N= 127 patients)

Trained separately (1 day workshops: teaching about instruments & role-plays)

Evaluated similarly: learning (self-rated adherence) & impact (BDI; Working Alliance Inventory; client satisfaction)

Discussion:Some problems with BES:

Ignores negative findings?

True, in order to address pragmatic questions; but mixed results, yet similar conclusions Problem of induction follows? True, but accidently coded weak/negative papers were also similar (failed to falsify); Also, BES review = falsifiable statement Generalize to non-LD/brief supervision? True, there very likely are significant differences: awaits more studies of ‘complex’ supervision

19/04/23 14

Conclusions & recommendations

BES systematic review = pragmatic method for addressing inconsistency between policy and practice

Results offer a ‘foundational’ evidence-base for CBT supervisor training, consistent with educational theory, best practise consensus and policies (add ‘complex’ supervision evidence as it emerges)

Mental health professionals should be prepared for the recognized specialization of supervision following the structured training format

19/04/23 15

Acknowledgements

This study was funded by the Higher Education Academy, Psychology Network

We are grateful to Helen Aylott, Nasim Choudhri, Christopher Dunkerley, Helen Fitzpatrick, and Sarah Wharton for their initial work on this review

Thanks also to Kathryn Mark, Barbara Mellors, and Thomas Cliffe for help in preparing the ppt

19/04/23 16

Authors

DEREK MILNE, School of Psychology, Newcastle University, Newcastle upon Tyne, NE1 7RU d.l.milne@ncl.ac.uk

ALIA SHEIKH, School of Psychology, Newcastle University, Newcastle upon Tyne, NE1 7RU a.i.sheikh@ncl.ac.uk

SUE PATTISON, School of Education, Communication and Language Sciences, Newcastle University, Newcastle upon Tyne, NE1 7RU susan.pattison@ncl.ac.uk

ANDREW WILKINSON, North Tyneside Primary Care Trust. Andrew.Wilkinson@northtyneside-pct-nhs.uk