Using a Learning From Practice to Improve HIA - HDA England - 2004

8/14/2019 Using a Learning From Practice to Improve HIA - HDA England - 2004

1/111

Using a learning from practice styleto help practitioners improve

health impact assessment

This bulletin summarises the learning from

practice work programme carried out at

the Health Development Agency (HDA). It

provides information on how the learning

from practice process was undertaken for

specic health impact assessment (HIA) topics

and how effective it has been, and offers

recommendations on how the HIA learning

from practice approach may be improved.

The learning from practice work

programme sought to help practitioners

achieve improvements in the practice and

understanding of HIA through a series of

workshops and summary bulletins. This

bulletin identies learning points drawn from

this process, based on the individual projects

that made up the work programme and their

evaluation. The programme consisted of:

Four learning from practice workshops on

four different HIA topics

Four bulletins derived from the ndings of

the workshops, one on each HIA topic

A report describing how the workshops

were undertaken

A report describing the evaluation of the

work programme.The work programme centred around four

pilot learning from practice workshops.

Practitioners from around the country were

invited to these to hear about, and share,

examples of promising HIA practice. Each

workshop had a different theme, and

each was evaluated on the day using an

evaluation form. A series of learning from

practice bulletins was produced, based on

the material provided by the case studies

presented and discussions held at the

workshops. These bulletins covered the four

workshop topics:

Inuencing the decision-making process

through health impact assessment(Taylor

et al., 2003a)

Evaluating health impact assessment

(Taylor et al., 2003b)

Deciding if a health impact assessment isrequired (screening for HIA) (Taylor et al.,

2003c)

Addressing inequalities through health

impact assessment(Taylor et al., 2003d).

A report describing how the workshops

were run has also been produced (Gowman

et al., 2003). The work programme ended

with an independent evaluation of the

series of HIA workshops and the resources

produced, and their impact on practitioners

(Crozier, 2004).

Further information on any of these elements

can be found at www.hiagateway.org.uk

Introduction


2/11

2

Identifying participants

Participants were identied and registered

for the workshops by email, via email groups

known to the HDA, up to three months inadvance. The only requirement was that the

participants should have some experience in

undertaking HIA, and preferably (although

not essential) some experience in the subject

area being discussed. Participants were asked

to volunteer case study material suitable for

presentation at the workshops.

Numbers and timing

Thirty-one people attended at least one

workshop, with a range of 914 participants

at each workshop (several people attended

more than one workshop). For various

reasons a small number were not able to

attend each workshop, despite having

booked. The workshops ran from 9.30am to

3.30pm, with the rst half hour for arrivals

and coffee.

Use of a facilitator

A central component of the learning from

practice method is the role of a facilitator

who can draw out responses from the whole

group and encourage debate and discussion.

At all four workshops an external facilitator

was contracted to undertake this role, and

to provide advice on how to structure the

workshops based on the evidence about how

people learn.

Structure of the day

The aims of the workshops were to:

Identify examples of projects that

demonstrate aspects of promising practice

Identify particular elements and processes

that need to be in place to make such

activities successful

Actively disseminate and share this learning

with those who are in the process of

planning and providing similar provision

Test the learning from practice workshop

model and assess its benets for future

application.

Each workshop had a similar structure:

Welcome and introductions

Rationale for and outline of the day,

including background information on

the theory of the learning from practice

approach, and topic-specic background

Aims of the day, both HDAs and

participants aims

Learning from practice case study

presentations, each 1015 minutes

long, detailing a practitioners personal

experience of working on a specic topic

Group clarication of the denitions/

understanding of the topic being discussed

Facilitated small-group work to discuss and

draw out learning from the case studiespresented, and from participants own

experiences

Group discussion on the benets to

practitioners of the topic under discussion

Group discussion where lessons learned

from the morning were distilled into what

promising practice looks like

Action planning by participants about

changes they may make to their practice

On-the-day evaluation forms completed.

A full explanation of how the workshops

were run, including background and rationale

information, is given by Gowman et al.

(2003).

Evaluation ndings (Crozier, 2004) andour reections

Identifying participants

The evaluation reported that identifying

participants for the workshop using regional

contacts and email contact lists worked well.

Nearly all participants valued the networking

from these meetings (Crozier, 2004).

Numbers

The evaluation reported that between 12

and 14 participants at each workshop was an

appropriate number to undertake the main

tasks required, particularly when it came

to splitting into small groups. The authors

recalled that lower numbers than this (eg one

workshop with nine people) interrupted the

planned structure of the workshops as it was

not easy to split into two separate groups to

discuss the two case studies presented.

Learning points from running the workshops


3/113

Timing

The authors noted that the timing of the

workshops required consideration. People

could not start too early because of travel

time from distant destinations, and people

tended to drift away after 3pm to travel

home. Depending on where participantscome from, it is important to consider

shorter days to allow people travelling time

from and to their home. If the workshops

move beyond the pilot stage, less time will

be needed for evaluation and this may help

shorten the day.

Use of a facilitator

The evaluation results showed that the

majority of participants appreciated the use

of the facilitator, but a small minority did not.Facilitating the workshops was difcult as

there were a number of strong personalities

present, who sometimes clashed. The topics

covered by the workshops included areas

where there is not full agreement among the

HIA community about how things should be

done, raising the potential for conict. Having

a professional facilitator who knew how to

deal with such situations was critical. But this

view was not shared by all participants, and

organisers must be aware that a minority

of participants do not want to be led by

a non-expert and see this as insulting to

acknowledged leaders in the eld (Crozier,

2004).

The authors, however, believe the learning

from practice style is not an expert-driven

process, and is based on practitioners

discussing and sharing their leaning and

experience so we believe facilitation by non-

experts in the particular subject area is a valid

approach. The use of a facilitator allowed the

organisers to participate more fully and derive

benet from the day.

Structure of the day

The evaluation reports that the majority of

respondents were satised with the structure

and implementation of the workshops, and

felt that the workshop structure and features

within it had enabled their personal objectives

for attending to be realised. The format isexible enough to allow small additions or

subtractions to the workshop format to suit

the topic/participants/timing (Crozier, 2004).

Recommendations for runningworkshops

As networking was so valued by

participants, the networking opportunity

should be highlighted in any initial email

advertising.

We recommend that a suitable number is

1214 participants, and that a small charge

for attendance at the workshop would

reduce the risk of participants pulling out

at the last minute. Overbooking participant

numbers by 25% ensures that if people do

pull out, adequate numbers remain to run

the workshop successfully.

We recommend that an individual with

excellent facilitation skills is required for the

smooth running of this type of workshop.The facilitator can deal with conict,

produce ideas on workshop format and

structure, and allow the organisers to

participate more fully in the proceedings

while adhering to the learning from

practice style.

We recommend that start and end times

should be chosen to suit the travelling

distances of participants, and workshops

should err on the side of being shorter

rather than longer.

We believe the structure of the workshops

(as described above) is appropriate and

suitable for future workshops.


4/11

4

The learning from practice approach used in

the workshops is underpinned by a theoretical

model, the certainty and agreement matrix

(Stacey, 1999). The workshop structure was

developed on the basis of evidence reviewsabout changing peoples practice (NHS

Centre for Reviews and Dissemination, 1999;

Ollerearnshaw et al., 2000). Both sources are

discussed below.

Theoretical model

This work is underpinned by a business model

called the certainty and agreement matrix

(Figure 1). The matrix can be used to choose

an appropriate action within a complex

system. It allows managers to choose from

an array of approaches to deal with a specic

issue, or helps them to make a decision

(Stacey, 1999).

The user answers two simple questions and

notes where the answers to those questions

intersect on a matrix. The matrix then

suggests a suitable approach to deal with the

issue, eg guiding a change process. The two

questions are:

Are issues close to agreement what level

of agreement is there in the group/team/

organisation about the desirable outcomes?

For example do we all agree that this is the

outcome we want from dealing with thisissue or from making this decision?

Are issues close to certainty can cause

and effect linkages be shown? For example

if we do this, then that will follow. This is

usually the case when a similar decision

has been made in the past, allowing

extrapolation from past experience to

predict future outcome with a good level

of certainty.

Previous work at the HDA (Taylor and

Quigley, 2001, 2002) highlighted that

there was moderate certainty about how

to do HIA, because of the many different

methods available to undertake an HIA, each

with different stages and using different

information within each stage. Similarly, there

was only moderate agreement about the

desired outcomes for HIA commentators

have described many different reasons for

doing an HIA, with each HIA attempting to

achieve something unique.

Learning points about learning from practice

Figure 1 What change process is appropriate? (Adapted from Stacey, 1996)

Low

High

High Low

Agreement

Certainty

Zone ofcomplexityPolitical

compromise

Standardsguidance

Experiment

Innovation

Creativity

Trial and error


5/115

Receiving a moderate answer for both

certainty and agreement placed HIA

within the zone of complexity (Figure 1).

Standards and guidance, tools often used

by government to attempt change, are not

suitable in the case of HIA. This allowed

us to consider other ways of workingto change practice within the zone of

complexity that use creativity, innovation,

and trial-and-error to develop practice

within the eld of HIA.

A learning from practice approach that

had previously been used in isolated topic

areas at the HDA appeared suitable as

an innovative method for use within HIA

(McCormick, 2002). The basic premise of

this approach is that people with academic/theoretical experience and those with

practical experience are brought together in

an informal learning environment, such as a

workshop, to share the learning they have

gained through study and/or experience.

It was anticipated that individuals working

towards a common goal would be able to

ask questions, be challenged, support one

another, and so develop better practice and

knowledge.

Experience of how people learn

The learning from practice approach was

also informed by relevant documents relating

to the evidence about how people learn

and make changes to their practice (NHS

Centre for Reviews and Dissemination,

1999; Ollerearnshaw et al., 2000). Principles

outlined in these documents about how

people learn were used in designing the HIA

workshops: The more complex the learning (eg HIA),

the more benet is gained from using

interactive approaches. The learning from

practice workshop approach is interactive

To change practice you need to avoid

instructions from the centre and instead

involve the practitioners to

- describe their own experiences

- identify what will work best in their

own circumstances

- discuss and learn from external

guidance and models rather than

importing them wholesale.

Again, the learning from practice approach

allowed these features to be practised

through case studies presented by

practitioners, small-group discussions, and the

presentation of research and theory where

available.


Learning style

The results of the external evaluation clearly

showed that the majority of participants

valued, and responded well to, this type of

learning. They found the workshops useful,

stimulating and engaging, and valued the

facilitated discussion. They appreciated

learning from peers and experts in the eld,

and the good use of case studies. There

was a small minority for whom participatory

learning with facilitated discussion was not

their preferred method of learning they

would have preferred a more teacher

student, lecture-type approach (Crozier,

2004). However, as noted in the reviews of

evidence about how people learn, practice

development work is less likely to be effective

using a didactic teacherstudent approach.

Background material and case studies

A small minority of participants reported

in the evaluation that they disliked the

initial background work that informed the

workshops because of its lack of rigour

(Crozier, 2004). In the pilots, case studies

were invited from participants and delivered

in the workshops. These were the hooks

on which discussion on how to do projects

well was hung, and were generally limited to

what was done. A minority of participantswanted these examples to be systematically

identied best practice. Some participants did

not believe they had adequate time before

the workshops to read over the background

material and case studies (Crozier, 2004).

We believe that a mix of good and not-so-

good (real-life) projects stimulates discussion,

is more in keeping with the learning from

practice style (rather than being expert-

driven), and allows practitioners to becomemore involved in the process by letting them

present their own work as recommended by

the experience of how people learn.


6/11

6

Participation of experts

The evaluation reported that participants

valued the opportunity to hear and reect on

other peoples experiences both from and

between experienced practitioners and other

experts. However, organisers also need to be

aware that a small number of participantsmay give more information than they receive.

This is, of course, the purpose of the day to

share learning and some people will have

more experience than others. This was not an

issue for all people in this situation, but some

felt they had had all of their learning sucked

out of them. Others acknowledged that, as

an expert, they would expect this to happen

in most situations, and were happy to share

their knowledge (Crozier, 2004).

We suggest that in some situations it may

be appropriate to consider a master-class

approach, where top experts only are invited

to some learning from practice events, levelling

out knowledge and skills differences. This

would cater for experts who would like to

develop their skills and knowledge further

within this style of learning approach. However,

this suggestion is opposed to the learning

from practice style, where learning is shared

among participants of different knowledge

and skill levels. The best solution may be that

any promotional material should make it clear

that experts experience and knowledge will

be drawn on heavily this would manage

participants expectations more carefully, and

those who do not wish to participate in such a

process can choose not to.

Impact on practitioners work

The external evaluation also investigated

whether, and in what ways, the workshops

had an impact on practitioners work. In

general, respondents struggled to identify

any specic impacts from the workshops.

When pressed, those whose work involved

teaching about HIA mentioned that they had

used the learning to inform the content of

their teaching, or had passed the bulletins

on to students. Some were able to identify

a moment within a workshop that had

helped them come to some new realisationabout the subject; others could not (Crozier,

2004). The action planning component of

the workshop elicited good responses from

participants about how they may change their

work in the future (Gowman et al., 2003),

but it is difcult to determine if these changes

were carried forward. Because of the small

amount of impact data collected, we believethat further evaluation would be useful to

determine if practice has improved, and that

in the meantime these workshops offer a

promising approach to achieve improvement

in HIA practice. The ability of participants to

recall specic changes to their practice was

limited because the workshops occurred up

to ten months earlier. This time lag was a aw

in this component of the evaluation.

Recommendations about the learningfrom practice approach

Staceys (1999) certainty and agreement

matrix is a suitable model for making

decisions on how to deal with an issue

in this case, how to improve practice

within HIA.

Using evidence on how people learn is

highly recommended for designing the

workshop structure.

Case studies should be sought from

practitioners attending the workshops,

rather than best practice case studies

being provided from the centre. The

case studies are the hooks on which

discussions on how to do projects well are

hung, and allow practitioners to present

their own work.

Providing participants with background

information and case studies well

before the workshops is essential we

recommend at least one week before.

Promotional material must state clearly that

experts and/or experienced practitioners

experience and knowledge will be drawn

on heavily.

As a small minority do not enjoy the

learning from practice approach,

consideration should be given to providing

other methods of learning (other than the

learning from practice workshop approach)

to cater for such peoples learning

preferences.


7/117

Bulletin production

The production of the bulletins was a

central component of this work, and they

were a planned outcome of the workshops,although this was not listed as an aim during

the workshops as we wanted participants

to focus on the learning from practice style

(Gowman et al., 2003).

The bulletins drew on the information

provided during the workshops, particularly

from the case studies and group discussions.

They were designed to provide practical

recommendations for improving the

application of HIA within the four topicareas covered; to provide practical case study

examples; and to outline the importance

of achieving such practice. Information

exchanged and presented during the course

of the workshops, including the case studies,

informed the content of the bulletins. All the

workshop participants were provided with

an opportunity to comment on the content

before publication.


The evaluation reported that the majority

of participants believed the bulletins were

clear, well presented and succinct. However,

a small number did not believe that the

workshops were the appropriate way to

develop such a publication, and that the HDA

would have been better going through a

more formal, expert-driven process. However,

we believe that the material produced fromthe learning from practice workshops was

also of considerable value, and an expert-

driven process can discount the views of

practitioners. Also, the evidence as to how

people learn recommends against such an

approach, pointing instead to practitioners

being heavily involved in developing the

material.

The production of bulletins is not a required

output from such learning from practice

workshops, and it would be up to future

organisers to decide if bulletins were needed

for the topic they are covering. We believe

that the bulletins are useful in providing a

lasting reminder for those attending the

workshops, as well as for the many people

who did not attend. Also, some people

prefer a written style of learning, and the

publications provide for their needs. Funding

bodies or managers often like to see a hard

copy report, and the bulletins also full

this function. The most important function

of a bulletin, though, is that it provides a

background document for future workshops

on the same topic potentially allowing this

learning style for a particular topic to be rolled

out across the country.

Recommendations on producingbulletins

Production of a bulletin is recommended

where a workshop has covered a topic

for the rst time using the learning from

practice style.

Bulletins should be produced after the

workshop the case studies, literature and

theoretical bases can all be merged with

other learning identied at the workshops

to produce a balanced publication. The

involvement of practitioners in this process

is critical, if you are interested in changing

practice.

Producing the learning from practice bulletins


8/11

8

How we evaluated the workshops andbulletins

The evaluation took place in three stages, the

rst two undertaken by the HDA, and thethird by an independent researcher:

On-the-day feedback forms were lled in

Participants were asked to comment on

the draft bulletins

Participants were followed up, up to 10

months later, and asked to take part in an

interview.

The HDAs rationale for undertaking the

evaluation was:

The HDAs role is to determine what

interventions work. Testing interventions

including workshops is crucial for

developing the evidence base in public

health

Use of the learning from practice approach

as a method for improving practice is

increasing, and it is important that any

learning from pilot work be transferred to

future work.

On-the-day feedback formThe on-the-day feedback form was

distributed to workshop participants at

the end of each of the four workshops.

Participants completed the three-sided

form and handed it back to the facilitator.

The form did not collect the name or any

other identifying details of respondents. The

questionnaire included items on:

How well participants felt the workshop

had achieved its stated objectives

Views on the content and presentation of

the workshops

Whether the learning gained would be

useful in participants own work

Administration and organisation of the

workshop

Any other comments.

Commenting on the draft bulletins

The bulletins were circulated to all participants

while they were in a draft stage, allowingtime for comments to be incorporated into

the structure or content.

Follow-up interviews

Follow-up interviews of participants were

developed and carried out by an independent

contracted researcher in consultation withthe HDA, and a full report is available

(Crozier, 2004). All the workshop participants

were invited to participate in the follow-

up interviews, and those who agreed were

interviewed in November 2003. The stated

aims of the research were to:

Assess whether the workshop aims and

objectives were achieved

Assess the acceptability and

appropriateness of the approach used Evaluate the intended and unintended

impacts of the workshops

Identify how the workshop process,

including its evaluation, could be improved

Determine the usefulness of the resources

created.

The results were presented anonymously

to reduce any potential inuence on

participants responses. The research was

made up of:

Analysis of on-the-day evaluation sheets

and rough notes from the workshop

Semi-structured interviews with workshop

participants and organisers conducted by

telephone or face-to-face

A structured questionnaire sent by email

to participants overseas or who were

otherwise not contactable by telephone.

Areas covered by the research included:

Marketing and customer care before,during and after the workshops

Motivations and expectations for

attending

Views on the workshops structure,

content, level, time for each item

covered

Impacts of attending the workshop

(intended and unintended)

Learning from practice bulletins

Overall assessment of the workshops and

their contribution to HIA.

Learning points about evaluation


9/119


The on-the-day feedback form drew an

overwhelmingly positive response about the

workshops, whereas the follow-up interviews

identied a small minority of participants who

were unhappy with the learning from practiceapproach. There was little sign in any of the

on-the-day feedback forms of the concerns

that some participants (a small minority) later

expressed in the in-depth interviews.

We believe that the on-the-day feedback

forms were inadequate to gain a full

understanding of all participants views,

nor did the responses provide a realistic

impression of the value of the workshop.

However, they did accurately gauge thegeneral thrust of the follow-up interviews,

that the majority of participants valued the

learning from practice style and found the

workshops useful.

The evaluation reported that feedback was

also positive when workshop participants

were given the opportunity to comment on

the learning from practice bulletins, with only

minor changes to the bulletins suggested by

participants.

The follow-up interviews provided much more

detailed information, and drew out the full

spectrum of experiences of the workshop

participants. Overall, the majority of

participants valued the learning from practice

style of teaching, and found the workshops

useful. The interviews provided useful data

for planning future workshops, for gaining

insights into how to improve the learning

from practice approach, and for highlightingissues that a minority of participants may

experience. However, the evaluation

described the delay between carrying out

the workshops and the eventual follow-

up interviews (up to 10 months) as a

major aw. This is likely to have inhibited

participants ability to recall and respond in

detail to the questions put to them by the

interviewer, and means that the ndings of

the follow-up interviews must be treated

with caution.

Recommendations for evaluation

On-the-day feedback forms should beshort, provide basic feedback and cover

short-term and administrative matters.

Future workshops should be evaluated

externally, especially as additional impact

information is required for this style of

learning.

Individual responses/data from the action

planning session (eg a practitioner saying

they will write up their case studies) should

be recorded so that they can be followed

up later in an external evaluation.

External evaluation should occur relatively

soon after the event no longer than

two to three months after to allow

participants enough time to change or

implement new practice, but not allow

time for memory recall to fade.

If bulletins are produced from the

workshops, they may need to be evaluated

separately to avoid delaying the workshop

evaluation. External evaluation is appropriate for

gathering the information required,

provided it is carried out in a timely

fashion. Future ways to improve an

evaluation could include covering these

additional questions:

- did participants contact anyone from the

day again?

- did they make new contacts?

- did they tell anyone else about what

they had done/learnt?

- did they do anything differently as a

result of attending?

- did they use the materials, examples,

points made on the day?

- did they nd the day more or less useful

than traditional dissemination of case

studies, and if so why, how, etc?


10/11

10

Further informationThe HIA Gateway website

(www.hiagateway.org.uk) provides access

to HIA-related resources, networks and

information to assist those participating in

the HIA process. The site is designed for bothbeginners and seasoned HIA practitioners.

The four summary bulletins, the workshop

report and the evaluation of the learning from

practice work programme (detailed in the

References) are all available on the website

( Resources Other materials General

guidance).

References

Crozier, A. (2004). Evaluation of health impact

assessment learning from practice workshops.London: Health Development Agency.

Gowman, N., Taylor, L. and Quigley, R. (2003).Learning from practice: report of a seriesof workshops for health impact assessmentpractitioners. London: Health DevelopmentAgency.

McCormick, G. (2002). Report to the TeenagePregnancy Unit on promising practice project.London: Health Development Agency.

NHS Centre for Reviews and Dissemination(1999). Effective health care: getting evidenceinto practice. Effective Health Care Bulletin 5 (1):1-16.

Ollerearnshaw, S., King, E. and Wright, S. (2000).The effectiveness of different mechanisms forspreading best practice. London: Ofce of PublicManagement.

Stacey, R.D. (1999). Strategic managementand organisational dynamics: the challenge ofcomplexity. New York: Financial Times/PrenticeHall.

Taylor, L. and Quigley, R. (2001). Looking back to

move forward: a retrospective process evaluationof ve HIA case studies. Presentation at the4th Annual HIA Conference, Liverpool, 1617October 2001. London: Health DevelopmentAgency (unpublished).

Taylor, L. and Quigley, R. (2002). Health impactassessment: a review of reviews. London: HealthDevelopment Agency.

Taylor, L., Gowman, N. and Quigley, R. (2003a).Learning from practice bulletin: Inuencing thedecision-making process through health impactassessment. London: Health DevelopmentAgency.

Taylor, L., Gowman, N. and Quigley, R. (2003b).Learning from practice bulletin: Evaluatinghealth impact assessment. London: HealthDevelopment Agency.

Taylor, L., Gowman, N., Lethbridge, J. andQuigley, R. (2003c). Learning from practicebulletin: Deciding if a health impact assessmentis required (screening for HIA). London: HealthDevelopment Agency.

Taylor, L., Gowman, N. and Quigley, R. (2003d).Learning from practice bulletin: addressinginequalities through health impact assessment.London: Health Development Agency.


11/1111

Authors:

Robert Quigley, Q-Research & Assessment Ltd and Lorraine Taylor, Health Development Agency

Copies of this publication are available to download from the HDA website (www.hda.nhs.uk).

Health Development Agency

Holborn Gate

330 High Holborn

London

WC1V 7BA

Email: [email protected]

URL: www.hda.nhs.uk

Health Development Agency 2004

ISBN 1-84279-259-8

About the Health Development Agency

The Health Development Agency (www.hda.nhs.uk) is the national authority and information

resource on what works to improve peoples health and reduce health inequalities in England.

It gathers evidence and produces advice for policy makers, professionals and practitioners,

working alongside them to get evidence into practice.

Using a Learning From Practice to Improve HIA - HDA England - 2004

Documents

Transcript of Using a Learning From Practice to Improve HIA - HDA England - 2004