Dr Tatiana Bachkirova

Nature of evidence, quality of research and self-deception in coaching and coaching psychology

2

The problem

• The knowledge base specific to coaching is thin

• Approaching it in coaching like way:

– Explore the situation around this problem

– Ask question about main concepts and assumptions

involved

– Explore options and solutions

– Consider potential pitfalls

3

What counts as good research?

• Systematic reviews and meta-analyses

• Randomized controlled trials

• Cohort studies

• Case-control studies

• Cross sectional surveys

• Case reports

4

Who cares about/does research on coaching?

• Specialised units in universities

• Commercial organisations

• Professional bodies

• Institute of Coaching

• Doctoral/MA students

• Practitioners

5

Going back to the basics.

What are the main concepts and assumptions?

• What do we want from research?

• What is evidence?

6

What is evidence?

• Evidence is not absolute and what it means is not a given.

• You can define it, but your definition will not be applicable,

nor necessarily useful, to all paradigms and to all fields of

knowledge and practices.

7

Meanings of evidence

• Context-free - evidence in philosophical normative sense (What is cause-effect relationship? What works in principle?)

• Context-sensitive - evidence in practical operational sense (What works here? What should be done? How to do it?)

• Colloquial - evidence as proxy for ‗most up-to-date information‘ on a subject

– ―anything that establishes a fact or gives reason for believing something‖ (Oxford American Dictionary)

8

Problems with RCTs

• All RCTs do is show that what you‘re dealing with is not snake oil. They don‘t tell you the critical information you need, which is which patients are going to benefit from the treatment (Williams, 2010)

• To account for heterogeneity among participants RCTs must be quite large to achieve statistical significance. What researchers end up with is the ―central tendencies‖ of a very large number of people — a measure that‘s ―not going to be representative of much of anybody if you look at them as individuals‖ (Williams, 2010).

9

More problems with RCTs

• RCTs may be especially ill-suited to psychological

interventions versus medical ones. In contrast to medications

that have a straightforward biochemical effect that‘s unlikely to

vary across individuals, psychological interventions tend to

interact with such factors as gender, age and educational level

(Hicks, 1998; Fishman, 1999; Williams, 2010)

10

Problems with RCTs,

specific for coaching

• What is a placebo for coaching?

How can people be provided with something which feels like coaching, they believe it to be coaching, but it is not?

• Difficulties with access

Researchers know how difficult it is to set up this kind of study with real leaders (Ellam-Dyson, 2012)

• Reductive nature of this design in principle.

It assumes that coaching is a neat intervention that can be attributed to the skills of the coach and delivered to a client who is a passive recipient. In fact clients have power in selecting a coach, keeping this service for a certain time and can significantly influence a coaching process.

11

Problems with outcomes

measures

• In therapy: subjective well-being, symptom reduction and life functioning (e.g. Mental Health Index, Howard et al, 1996)

• A fundamental difficulty of coaching outcome research is the extreme heterogeneity of issues, problems and goals, which can be picked out as themes in different coaching interventions (Greif, 2007)

– General measures (goal attainment and client satisfaction)

– Specific measures (e.g. result of business coaching)

12

Problems with a value of findings

• Positivist paradigm requires the search for general relationships between a small number of discrete variables across wide varieties of context, and yet it is these contexts, form a constructionist point of view, that have a large impact upon these relationships (Fishman, 1999, p. 235).

• What is found without consideration of context:

• ―Process variables that have been found consistently related to outcome… [include] the overall quality of the therapeutic relationship, therapist skill, patient cooperation versus resistance, patient openness versus defensiveness and treatment duration (Orlinsky, et al, 1994, p. 364)

• In coaching – very similar (Grief, 2007)

13

Checking the basics: what is science?

• One meaning of science is identified with modernistic positivism and refers to beliefs in the discovery, through the experimental method, of objective, universal, and decontextualized laws about the natural, biological, psychological, and social worlds.

• The other meaning of science refers to disciplined, critical, reflective thought that compares and contrasts evidence arguing for alternative interpretations or explanations of a particular phenomenon.

This is a meaning of science that James,

Dewey and other pragmatists had in mind

advocating a ‗scientific attitude‘. Many

different scholars, theorists and

professionals are ‗scientific‘ in the sense

of bringing intellectual discipline and

rigour to their work even if they function

outside the experimental, positivist

paradigm.

Fishman (1999, p. 206)

14

What are the alternatives?

• Open up to different paradigms (pragmatism, contextualism,

interpretative)

• Continue with all types of research, but also…

• Add much more case studies

– A detailed description in the context of a particular client and a

particular coach of how we weave together the technical,

relationship and process ingredients in a way to effect change

– And then systematically assess and analyse these individual cases

both quantitatively and qualitatively

15

Using research

In therapy:

• Best Research Evidence

• Best Clinical Experience

• Consistent with Family/Client Values

16 The path of the master coach to artistry (Drake, 2009, p. 8)

Using evidence and knowledge in coaching

17

Potential pitfalls

The aim of science is not to open the door to

infinite wisdom, but to set a limit to infinite

error.

Bertolt Brecht

18

Collective self-deception

• There is still strong modernistic forces in our culture, so being viewed as having a scientific basis of one‘s work endows it with a privileged status.

• How do we judge the quality of research?

• Politics of evidence (Drake, 2009). How do we position ourselves in the midst of the growing demands to serve multiple masters?

19

Aspect Business coaching model Developmental coaching model

Philosophy Pragmatism

Approach is eclectic: any method can be used - “what works”

Underlying purpose: current needs of the organisations as seen by SMT

Explicit purpose: Psychological evolution

Organisations will benefit as the result of individual development Approach is integrative: methods are integrated by a common philosophy

Orientation for outcomes

Short-term Long-term

Environment Organisation Wider than organisation: society, humanity, all natural world

Individual Separate from the organisation: a material to make use for organisational needs

An agent, who in the process of his/her own development may also change an organisation and wider environment

Coach A broker between the individual and organisation

Yes, to some degree, but also a facilitator of psychological evolution

Contract In theory – mutual agreement In practice – “who is paying for it”

True partnership

Psychological contract in addition to formal one

20

Individual self-deception

Basic types of cognitive errors:

• Overutilization of generally valid, intuitively reasonable strategies (we could over-rely on theory and research as opposed to observation)

• Availability heuristic (estimate the probability of an outcome based on characteristics such as most recent or emotionally loaded information, rather than a logical analysis)

• Overuse of our own schema for practice (we could too easily discard important information that doesn‘t fit with this schema)

21

More examples of self-deception in coaches

Coaches…

• never give advice, only ask questions

• don‘t do remedial coaching

• don‘t work with personal problems

• never have a value conflict with a client

• can always provide complete confidentiality

• don‘t coach if they have a conflict of interests

Video on the internet:

7 Steps to Retaining Clients 7 Times

Longer Than Other Coaches!

22

What contributes to self-deception

• We ‗bracket our attention‘ to avoid distress and to maximise

satisfaction

• Fear of not being good enough, of rejection

• High expectations, ambitions

• Security of being like everyone else

23

Minimising self-deception • In the business of developing others we also develop

• Improving quality of perception in principle: ‗perceptual re-

education‘ (Claxton, 1994)

• Awareness and reflexivity:

―Awareness, attendance to self, and articulation of engagement

with the world frees the mind from self-deceptive tendencies‖

(Frost et al, 2001).

24

Working with self-deception in supervision

• Explicit contract

– What aren‘t we talking about?

• Atmosphere of safety

– Permission to be imperfect (home for

a shadow)

• Embracing the paradox

– Modelling how to do it right by doing

it wrong

25

Helping each other to minimise self-deception

How do we explore our self-deception and our shadow needs in

a way that is loving and forgiving? We do need both: kind of

sharpness of being able to see through our hopes, fears and

self-editing and also the compassion to say: well, that is part of

us being human.

….Also something about not hunting

it down because by hunting it down

we‘ll make it a thing rather than

something that we are swimming in…

(RP 5)

26

Signs of progress

• That is part of the maturity of the coach, the degree of… not

that you‘re going to fall into traps less, but you notice it as you

are going down the hole, you notice that you‘re sliding down. It

is not that you can really stop these things from happening, it‘s

just that you catch them quicker and sometimes in the

moment… (RP 5).

27

References

• Bachkirova, T. (2013, forthcoming), Self-deception in coaching.

• Claxton, G. (1994). Noises from the Darkroom. London: Aquarian.

• Clay, R. (2010) More than one way to measure, APA, September 2010, Vol 41, No. 8, http://www.apa.org/monitor/2010/09/trials.aspx, Print version: page 52.

• Drake, D. (2009) Evidence is a verb: A relational view of knowledge and mastery in coaching, International Journal of Evidence Based Coaching and Mentoring, Vol 7, No 1, pp.1-2.

• Ellam-Dyson, V. (2012) Coaching psychology research: Building the evidence, developing awareness, OP Matters, No 16, pp.13-16.

• Fishman, D. (1999) The Case for Pragmatic Psychology, New York: New York University Press.

• Frost, C. Arfken, M. & Brock, W. (2001). The psychology of self-deception as illustrated in literary characters, Janus Head, 4, pp. 331-357.

28

More references

• Grief, S. (2007) Advances in research on coaching outcomes, International Coaching Psychology Review, Vol 2, no 3, pp. 222-245.

• Hicks, C. (1998) The Randomised Controlled Trial: A Critique, Nurse Researcher, Vol, 6 No 1.

• Howard, K, et al (1996) Evaluation of psychotherapy: Efficacy, effectiveness, and patient progress, American Psychologist, No 51, pp. 1059-1064.

• Lane, D. A., & Corrie, S. (2006). The modern scientist-practitioner: A guide to practice in psychology. London, UK: Routledge.

• Orlinsky, D., Grawe, K. & Parks, B. (1994), Process and outcome in psychotherapy, In A. Bergin & S. Garfield, eds, Handbook of psychotherapy and behavior change, (4 ed) New York: John Wiley, pp. 270-378.

• Williams, B.A. (2010). Perils of evidence-based medicine. Perspectives on Biology and Medicine, 53, 1, 106–120.