Full appendices to the research report on neighbourhood ... · research project. The full research...

GfK NOP Social Research

Full appendices to the research report on neighbourhood planning referendums: referendum question testing

These appendices contain discussion guides, the questions tested in each location, examples

of stimulus materials and breakdowns of the make-up of the mini-depth sample for this

research project. The full research report is available in a separate document.

1.1 Discussion guides

A) Focus group discussion guide

Interview Objective: Explore how voters react to and understand the proposed referendum question. Specifically: Identify any areas of the question that are problematic and the reasons for any problems. Investigate the extent to which the question meets the Commissions’ question assessment

guidelines: being easy to understand; to the point; unambiguous; ensuring that voters don’t consider one response more favourably than another; avoid misleading voters.

Explore potential ways of redressing any problems through changes to the question and other channels of information.

Please note: this discussion guide is intended to guide the discussion however; the exact flow

and question wording will be tailored by the focus group participants. Resultantly, not all

questions may be asked in the order below, or using the wording below.

Moderator please note: throughout the mini group people will make suggestions for

improvements. Please get a feel for whether these are essential or nice to have.

1. Moderator introduction and initial question test 10 mins

(Aim: introduce research to participants and ask participants to complete the referendum question)

Thank you for agreeing to take part in this group discussion.

Introduce self and GfK NOP.

Explain purpose of the group discussion: you have been invited along this evening to

discuss your thoughts on a referendum question to do with neighbourhood planning.

Reassure participants that the discussion is confidential, and the MRS Code of Conduct.

Our discussion will last around 1.5 hours.

Explain: I’d like to start our discussion by asking you to complete an exercise.

Please look at the sheet on your chair. Without talking, please answer the referendum

question as if you were doing so for real. Moderator please clarify: What is on the page is

what you would see on your ballot paper if you were to go along to a polling station to

vote, or what you would receive by post if you were registered as a postal voter.

2


Please do not move on to the worksheet until everyone has finished.

There are no right or wrong answers to the question. If you would be happy to tell people

how you answered the referendum question, please feel free to do so during the

discussion. If you do not want to tell people how you answered, you do not have to. The

discussion will focus on your understanding of the question.

Stimulus: here we will show QUESTION 1 relating to neighbourhood plan TYPE 1 (refer to

rotation).

Now you have all answered the referendum question, please answer the three questions

on the worksheet:

o On the first page, please summarise what you think the referendum question was

asking in your own words.

o On the second page, please underline anything you found clear in blue and

anything you found unclear in red.

o On the third page, please write down what changes, if any, you would like to see

made to the referendum question to make it clearer.

Moderator to clarify: By the question we mean everything on the paper

from the explanatory paragraph to the question itself

Please complete your worksheets without talking. If you have any questions, raise your

hand and I will come to you.

2. Participant introduction 5 mins

(Aim: meet the participants and gather initial views regarding the referendum question)

Thank you for completing the exercise. Moderator - explain purpose of research:

o During the discussion we will be looking at your views towards the neighbourhood

planning referendum question you have just completed. GfK NOP is carrying out

research for the Electoral Commission to test the referendum question with

voters.

o Does everyone know what a referendum is? Just to clarify, referendums are held

to ask the people in a country or area to vote on a proposal.

This research is not about your views about how you would vote or what you think about

local planning in your area. It is about the neighbourhood planning referendum question

itself and how well you understand it.

The questions that we are looking at are questions that people could be asked in any area

across England in the future. However, I should stress that we’re only in [group location]

because it’s a good place to talk to people. We don’t know of any plans for a

neighbourhood planning referendum in your area, and any plans that we discuss today are

just examples we’ve made up for this research.

Respondent introductions:

o Please tell me your first name and a couple of things that you wrote in your

exercise.

3


3. Completing the question: accuracy 5 mins

(Aim: explore how confident participants feel in answering the question)

Moderator note: IF REQUIRED during the discussion, you can reassure participants

that they will be given more information on this topic as the group continues.

Who feels confident that they gave the response they wanted to?

o What makes you feel confident about this?

o Do you have any doubts that you have given the response you wanted to?

Who does not feel confident that they gave the response they wanted to?

o Why do you not feel confident about this?

o What could make you feel more confident about your response?

What do you think the question was asking you?

What do you think you have voted for?

How did you decide whether to vote yes or no?

o Moderator please note: this question is included to give insight into any potential

misconceptions when answering the question but we are keen not to enter a

discussion on general views around neighbourhood planning.

Based on the discussion so far:

o Who feels they have given the response they wanted to? Why?

o Who feels they have not given the response they wanted to? Why not?

Who feels unsure about whether they have given the response they want

to or not?

4. Completing the question: literal meaning 10 mins

(Aim: understand how far participants understand the literal meaning of the question; the words on the page)

What words would you use to describe the referendum question?

o Moderator to clarify if needed: By the question we mean everything on the paper

from the explanatory paragraph to the question itself

Prompt if needed: o Positive words: easy, clear, to the point, straightforward.

o Negative words: difficult, unclear, longwinded, confusing.

Ease of understanding

Overall, how easy was the question to understand?

o What were the main things that made it easy to understand?

o What were the main things that made it difficult to understand?

o Which words and phrases did you find easy to understand?

What makes them easy to understand?

How do these words help you understand the question?

Which words and phrases did you identify as difficult to understand?

o Why were these difficult to understand?

o What do you think they mean?

4


o How far did these affect your understanding of the question?

o How could they be changed and improved?

Did the question give you the right amount of information? Why / Why not?

o What more does it need to tell you?

o Is that information that needs to be on the ballot paper or could it be somewhere

else?

o Is there anything that it doesn’t need to tell you?

o Is that information that you don’t need at all, or is it information that you’d rather

find out somewhere else?

o Moderator please note: this is a chance to get initial spontaneous views around

information provision at this stage but we will cover this in greater detail later

within the guide.

Neutrality

Are there parts of the question that sounds particularly positive or negative?

Which parts are these? What kind of person do you think wrote this question?

Does the person who wrote the question want you to vote one way or the other?

Summary and improvements Do you agree or disagree that the referendum question: Moderator to probe each bullet

o Uses plain language?

o Is clear in what it is asking?

o Is biased towards one option?

o Is to the point?

o Is misleading to voters?

How would you describe the language used in the question at the moment?

o Why is this?

o What makes you say this?

5. Testing the alternative wording 10 mins

(Aim: test comprehension of other papers including alternative wording)

Stimulus: here we will show QUESTION 2 relating to neighbourhood plan TYPE 1 (refer to

rotation).

Stimulus: here we will show the alternative wording.

What do you think about this question?

What’s different between this question and the first/second one you looked at.

o NB: whether the alternative wording is shown first or second is TBC.

Is it asking the same thing?

Moderator clarify: these are two different ways of asking the same question.

What works well about this wording? Why?

What works badly about this wording? Why?

5


Stimulus: here we will directly compare both questions. Moderator to show the questions

side by side.

Moderator to note: We do not want to make people choose a preference – we just want to

understand what they like or dislike about both of the questions.

Where problems were identified with the first question please explore this issue with the

alternative wording/ compare this problem across the different versions.

Looking at the different options:

o How do these questions compare?

o Which parts of the questions work better / worse? Why?

o Is one of the questions clearer? Is this the whole question, or just parts of it?

o Is one of the questions more to the point? All, or parts of it?

o Is one of the questions more balanced than the other? All or parts of it?

o Is one question easier to answer? Why?

6. Understanding the question in context 15 mins

(Aim: explore the context/ awareness/ knowledge when answering the question)

Can anyone remember there being a referendum in your local area before?

o What was this about?

Why might there be a referendum question about planning in your area?

o Has anyone heard of this?

Has anyone heard of: Moderator, ask about relevant planning type:

o A neighbourhood plan

o A neighbourhood development order

o A Community Right to Build Order

(If not) What do you think it might be?

Looking at the question again

o What do you think planning in this context refers to?

What makes you say this?

o What do you think is meant by a ‘local plan’

What makes you say this?

o What do you think is meant by: Moderator, ask about relevant plan:

NDPL: ‘planning policies for the development and use of land?’

NDO: ‘development’

CRTBO: ‘development’

Probe for all:

o What comes to mind?

o What kind of policies / planning permissions do you think

this might refer to?

o What will be the outcome if [a yes answer] is given?

What would happen as a result?

What would this mean for your local area?

Would this happen automatically or would there be other steps?

o What will be the outcome if [a no answer] is given?

6


What would happen as a result?

What would this mean for your local area?

Would this happen automatically or would there be other steps?

Who feels confident that they understand what the referendum is about?

o Why/ why not?

Is there any information that would help you better understand the question?

Moderator: ask ONLY if time available:

Which specific local areas are likely to be affected by this? Is the question more about the local area developing generally, or is it about a specific

piece of development? What kind of thing do you think might be in a neighbourhood plan?

7. Information provision 15 mins

Stimulus: here we will show some information regarding neighbourhood plan TYPE 1 (see

rotation). Moderator please clarify that the examples are made up for the purposes of the

research.

(If not already discussed) did everyone know what a referendum was before they came along

this evening?

For those who don’t: a referendum is a vote on a single political issue.

For each statement shown ask:

o Did you know about this before?

o Has this changed how you understand the question?

Moderator: ask below questions at appropriate points in the stimulus delivery

Definition of neighbourhood plan / neighbourhood order / Community Right to

Build Order

Had you heard about this before?

Did you understand the intention behind the plan?

Which parts are easier / more difficult to understand?

People involved in a plan / order?

Did you understand who is responsible for developing a neighbourhood plan?

Which aspects of the people involved in a neighbourhood plan are easier / more difficult to

understand?

The outcome

7


Is this the outcome that you had expected from the neighbourhood planning referendum?

Do you understand who gets a say in the process?

Which aspects of the outcome are easier / more difficult to understand?

8. Reflecting on & refining the question, exploring alternatives 10 mins

(Aim: explore how the information impacts on understanding and identify how the question could be improved)

Thinking about the information we have just looked at…

Does this change the way you would respond to the question? How/ why?

If you were responsible for designing a question for people to vote on this topic, what

would you do?

o The question itself?

o Other information?

Moderator: ensure discuss any changes to the way would answer the question.

Moderator: please go through each question sentence by sentence to ensure understanding

and identify whether it is clear/ where improvements are needed.

o Do you think the wording of the question needs to be changed to make it easier to

answer? How?

Overall, which elements of the question would you keep?

Which would you change/ why?

o Which changes do you think are essential? Why?

o Which changes would be just nice to have? Why

Overall, which pieces of information (if any) are important for people to know before they

answer the question?

o Which information is essential? Why?

o Which information is just nice to have? Why?

Where would you like to see that information?

To what extent do you think this should be included on the actual ballot paper?

o How well do you think this would work?

o Are there better places that you could be told this information?

How would you like to be told about this?

Individual exercise: Imagine you were responsible for improving the question. Please write

down what your preferred wording would be for the question.

Discuss responses as a group.

9. Information needs 10 mins

(Aim: explore what information people would like and how this should be delivered)

8


What information would you expect to be told about the neighbourhood planning

referendum?

What information would you like to know in order to enable you to vote? Why?

o From the information we have looked at today, which information would you tell

people in your area about this referendum?

o What do you think would be the most important thing to tell people? Why?

o Which information is essential? Why?

o Which information is just nice to have?

Individual exercise: write down which three pieces of information are most crucial for people

to have before voting in the referendum.

Discuss responses as a group.

Who is responsible for giving people this information?

What would be the best way to provide this information?

o Prompt:

o Advice in postal vote packs

o Adverts in the local paper

o Newspaper articles

o Notices in public places

10. Summary and Close 5 mins

(Aim: wrap up the discussion)

Thinking about everything we have looked at this evening, is there anything you would

suggest we do to ensure that people understand the neighbourhood planning referendum

question?

Moderator: go around the group and get an answer from each participant.

Thank and Close

9


B) Mini-depth interview discussion guide

Mini-depth Discussion Guide FINAL

Interview Objective: Explore how voters react to and understand the proposed referendum question. Specifically: Identify any areas of the question that are problematic and the reasons for any problems. Investigate the extent to which the question meets the Commissions’ question assessment

guidelines: being easy to understand; to the point; unambiguous; ensuring that voters don’t consider one response more favourably than another; avoid misleading voters.

Explore potential ways of redressing any problems through changes to the question and other channels of information.

Please note: this discussion guide is intended to guide the interview however; the exact flow

and question wording will be tailored to the participant. Resultantly, not all questions may be

asked in the order below, or in the wording below.

Moderator please note: throughout the depth people will make suggestions for improvements.

Please get a feel for whether these are essential or nice to have.

1. Moderator introduction and initial question test 5 mins

(Aim: introduce research to participants and ask participants to complete the referendum question)

Thank you for agreeing to take part in this 20-minute interview.

Introduce self and GfK NOP.

Explain purpose of the interview: you have been invited along to discuss your thoughts on

a referendum question to do with neighbourhood planning.

This research is not about your views about how you would vote or what you think about

local planning in your area. It is about the neighbourhood planning referendum question

itself and how well you understand it.

The questions that we are looking at are questions that people could be asked in any area

across England in the future. People have started putting together a neighbourhood plan

in Hackbridge, but we’re really just in Sutton because it’s a good place to talk to people.

We don’t know of any plans for a neighbourhood planning referendum in your area, and

any plans that we discuss today are just examples we’ve made up for this research.

Reassure participants that the discussion is confidential, and the MRS Code of Conduct.

Participant introduction: first name, job/ hobby.

Explain: I’d like to start our discussion by asking you to complete an exercise.

Stimulus: here we will show QUESTION 1 relating to neighbourhood plan TYPE 1 (each depth

moderator to take a different question on each interviewing day).

Please answer the referendum question as if you were doing so for real.

Can you tell me in your own words what you think the question was asking you?

o What do you think you have voted for?

10


How confident do you feel that you have given the response you wanted to? Why?

2. Completing the question: literal and contextual meaning 5 mins

(Aim: understand how far participants understand the literal meaning of the question; the words on the page)

Moderator: as you go through the question with respondent please mark it up to highlight

things that are positive and things that are negative – we will then revisit this at the end of

the interview.

Literal:

Overall, how easy was the question to understand? Why?

o What were the main things that made it easy to understand?

o What were the main things that made it difficult to understand?

o Were there any words and phrases did you find difficult to understand?

o Did the question give you the right amount of information? Why / Why not?

If not, what sort of information would you find useful?

o Are there parts of the question that sound particularly positive or negative?

What words would you use to describe the referendum question?

Prompt if needed:

o Positive words: easy, clear, to the point, straightforward.

o Negative words: difficult, unclear, longwinded, confusing.

Contextual:

What impact do you think a neighbourhood planning referendum would have?

o What will be the outcome if [a yes answer] is given?

o What will be the outcome if [a no answer] is given?

How confident do you feel that you understand what the referendum is about? Why?

Moderator: ask ONLY if time available:

Which specific local areas are likely to be affected by this?

Is the question more about the local area developing generally, or is it about a specific piece of development?

What kind of thing do you think might be in a neighbourhood plan?

3. Information provision 5 mins

(Aim: test comprehension of alternative wording and explore impact of information provision)

Stimulus: here we will show some information regarding the Neighbourhood planning

referendum

Show statements then ask:

o Did you know this before?

o Has this information changed how you understand the question?

11


o Is there a particular bit of information that’s made a difference?

o Does this change the way you would respond to the question? How/ why?

Mod NOTE: please verify that people understand what is being shown, and that this

has improved their understanding

If not, what other information would be useful?

Had you heard about a neighbourhood plan / neighbourhood order / Community Right to

Build Order this before?

Which parts are easier / more difficult to understand?

Did you understand who is responsible for developing a neighbourhood plan?

Which aspects of the people involved in a neighbourhood plan are easier / more difficult to

understand?

Is this the outcome that you had expected from the neighbourhood planning referendum?

Which aspects of the outcome are easier / more difficult to understand?

Moderator: ask ONLY if time available and these issues have not already been raised:

Which specific local areas are likely to be affected by this? Is the question more about the local area developing generally, or is it about a specific

piece of development? What kind of thing do you think might be in a neighbourhood plan?

4. Improvements, information needs and close 5 mins

(Aim: explore what information people would like and how this should be delivered and end mini-depth)

Improvements:

Moderator: please refer to marked up question. Explore any thoughts on question length,

type of language, words used.

Looking at both of the questions we have looked at:

o Do you think the question needs to be changed to make it easier to answer?

How?

o Overall, how would you improve the questions?

Information needs:

Moderator: please only briefly cover this.

What information would you expect to be told about the Neighbourhood planning

referendum before it took place?

What information would you like to know in order to enable you to vote? Why?

Thank and Close

1.2 Questions tested in each location

12


This section contains the questions tested in each of the five research locations.

1.2.1 Hackbridge: NDP

A) CLG wording

13


1.2.2 Honiton: NDO

A) CLG wording

14


B) Alternative wording

15


1.2.3 Ringmer: NDP

A) CLG wording

16



17


1.2.4 Newcastle: CRTBO

A) CLG wording

18



19


1.2.5 Nottingham: CRTBO and NDO

A) CLG wording – Community Right to Build Order

20


B) Alternative wording - Community Right to Build Order

21


C) CLG wording - Neighbourhood Development Order

22


D) Alternative wording - Neighbourhood Development Order

23


1.3 Stimulus material

The stimulus materials were individualised for the relevant plan type and planning authority in

all five locations, meaning that there were a number of different versions pertaining to

different relevant authorities. We have included one example per plan type here to highlight

the information that participants would have seen. Our examples show the Neighbourhood

Development Plan, the Neighbourhood Development Order and the Community Right to Build

Order. Participants were shown the relevant example for the plan tested in their area.

1.3.1 Neighbourhood Development Plan

Showcard 1

24


Showcard 2

25


Showcard 3:

Showcard 4

26


1.3.2 Neighbourhood Development Order

Showcard 1

Showcard 2

27


Showcard 3

Showcard 4

28


1.3.3 Community Right to Build Order

Showcard 1

29


Showcard 2

Showcard 3

30


Showcard 4

Showcard 5

31


1.4 Mini-depth quotas achieved

This table shows how the 80 mini-depth interviews break down across a range of

characteristics; gender, age, socio-economic grade (SEG), marital status, tenancy, ethnicity,

voting behaviour and community involvement.

Quota Target

Rin

gm

er

Ha

ck

bri

dg

e

Ne

wca

stl

e

No

ttin

gh

am

Ho

nit

on

Ha

rd t

o

rea

ch

1

TO

TA

L

Gender

Male 2 8 4 8 7 3 32

Female 6 6 12 6 8 10 48

Age

17-24 1 2 8 2 2 2 17

25-34 1 2 5 1 3 12

35-44 3 1 1 2 3 10

45-59 3 4 1 4 5 1 18

60+ 4 4 4 2 5 4 19

Social grade

B 3 2 2 4 4 15

C1 4 7 8 5 5 7 36

C2 1 1 2 3 3 10

D 1 2 1 2 2 8

E 1 3 3 2 1 1 11

Marital status/children in home

Married/cohabiting (children at home) 1 1 1 1 2 6

Widowed/Single/Divorced/Separated/Living alone (children at home)

3 1 1 3 2 10

Living with parents/Living with friends (children at home)

0

Married/cohabiting (children NOT at home) 1 7 3 3 9 6 29

Widowed/Single/Divorced/Separated/Living alone (children NOT at home)

5 3 5 9 1 2 25

1 Hard to reach respondents were interviewed across the five locations.

32


Living with parents/Living with friends (children NOT at home)

1 7 1 1 10

Tenancy

Home owner 6 5 5 5 10 4 35

Council tenant 1 1 3 1 1 7

Housing association tenant 1 1 1 3

Private rented accommodation 1 6 2 5 2 6 22

Other 1 1 7 1 2 1

Ethnicity

Ethnic minority group 2 5 7

Voting behaviour

Always or usually vote in local elections AND regularly or sometimes attend community groups

3 1 3 5 1 3 16

Always or usually vote in local elections AND occasionally or never attend community groups

3 11 9 4 9 5 41

Sometimes or never vote in local elections AND regularly or sometimes attend community groups

1 1 1 2 2 3 10

Sometimes or never vote in local elections AND occasionally or never attend community groups

1 1 3 3 3 2 13

Full appendices to the research report on neighbourhood ... · research project. The full research...

Documents

Transcript of Full appendices to the research report on neighbourhood ... · research project. The full research...