Project 4: Student teacher engagement with research

Elspeth McCartney, Helen Marwick

University of Strathclyde

ECER Conference

3rd September 2014, Porto

3

• Across Europe there is a renewed emphasis on developing ‘evidence based’ classroom practices

• For example, the work of the cross-European Evidence Informed Policy and Practice in Education in Europe (EIPPEE) group http://www.eippee.eu/cms/

The (re)turn to research evidence

4

• There are contended issues of what constitutes ‘evidence,’ on the feasibility of filling the evidence gap, and of the complexities of implementation

• However, most accounts agree on the need for teachers to become ‘discerning consumers’ of research (BERA 2014 p5)

The (re)turn to research evidence

5

• ‘Critical Policy’ evidence, synthesising (often) large-scale data and existing research studies into policy statements

• ‘What works’ research, evaluating and synthesising studies on the effects of specific classroom approaches

• Much research is as yet poorly articulated with teachers’ needs/poorly ‘translated’ for use

Two broad types of ‘evidence’

6

• Policy makers and service commissioners seek ‘best’ evidence from randomised controlled trials (RCTs) which provide a ‘counterfactual’ (control) condition

• RCTs offer the possibility that results would generalise to similar situations

• However, there is then a need for real-world follow-up implementation studies

The turn to (medical) trials models

7

• Not all teachers are educated within Higher Education (England)

• There is limited agreement on underpinning disciplines for teachers, beyond their curriculum subject areas

• There is limited formal instruction in applied research methods in many ITE courses

The (re)turn to varied initial teacher education (ITE) formats

8

• However, there is also a current, international, focus on developing the quality of teaching and the teaching workforce. Reviews of high-quality school systems internationally have identified teacher professionalism as a key factor (Schleicher, 2012)

• Professionalism includes both personal qualities leading to ethical and responsible action, and access to a knowledge and evidence base that allows the development of educational practices that are likely to be effective.

The (re)turn to varied initial teacher education (ITE) formats

9

• Professionalism includes both personal qualities leading to ethical and responsible action, and access to a knowledge and evidence base that allows the development of educational practices that are likely to be effective.

The (re)turn to varied initial teacher education (ITE) formats

10

• The UK HEA is an independent body funded by HE institutions that champions learning and teaching within HE

• In 2013 it undertook research on the distinctive contribution of HE to ITE (Florian & Pantić, 2013)

• This resulted in a strategic research priority: ‘Supporting research-informed teacher education in a changing policy environment’

The UK Higher Education Academy (HEA) response

11

• The ultimate aims of this programme are:• to develop educational opportunity and

achievement for the diverse modern classroom

• for teachers to respond effectively to developmental, social, cultural and/or linguistic factors that impact, often adversely, on child attainment and wellbeing

The UK Higher Education Academy (HEA) response

12

• This project (Project 4) was commissioned as a strategic social science project within this priority research programme

• It investigated student teachers’ views on and use of research evidence

HEA Project 4

13

• Little is known about student teachers’ views as they progress towards practice

• Teachers’ views of research evidence and its uses are not uniformly welcoming (Helmsley Brown & Sharp 2003)

• Students are transitioning into the new research context, and may have similarly mixed views

Because …..

14

1 identify examples of research evidence on the influences of developmental, social, cultural and/or linguistic factors on child attainment and wellbeing, and of their inter-relationships – i.e. critical policy research.

 

2 identify examples of What Works research evaluating classroom practices, also for children with developmental, social, cultural and/or linguistic factors that may impact adversely on their attainment and wellbeing – what works research.

Specific project aims were to:

15

 

3prepare and pilot workshop materials to engage participant students in appraising selected reports of relevant research, using questionnaire and group discussion methods to discuss the barriers and facilitators they perceive in using research

4identify key themes emerging from pilot workshops

 

Specific project aims were to:

16

• A mixed-methods approach was used, employing survey methods(questionnaires) where number of responses to closed questions could be summated, alongside thematic analysis of open comments and group interview responses.

Methodology

17

• Education research evidence was sourced from University of Strathclyde Humanities and Social Science Faculty staff, as a scholarly community

• Also from targeted literature searches

Aims 1 and 2: sourcing evidence

18

• Faculty staff sent policy research examples, but no What Works research examples.

• Targeted searches found both kinds of evidence.

• One example of each was used in a workshop.

Evidence retrieved

19

• Key sources of policy research were:

• The Joseph Rowntree Foundation http://www.jrf.org.uk/publications

• The National Foundation for Educational Research http://www.nfer.ac.uk/publications /

• The Teachers College Record http://www.tcrecord.org/

Sources for policy evidence

20

• Key sources of What Works research were:

• The Campbell Collaboration Library of Systematic Reviews http://www.campbellcollaboration.org/lib/

• The Cochrane Collaboration reviews http://www.cochrane.org/cochrane-reviews

• The Education Endowment Foundation http://educationendowmentfoundation.org.uk/toolkit/

• The ‘What Works’ Clearinghouse http://ies.ed.gov/ncee/wwc/findwhatworks.aspx

•  

Sources for What Works evidence

21

• Pilot students could be studying for a:• Post Graduate Diploma in Education

(PGDE), primary or secondary,• BEd degree, primary; or • BA Childhood Practice (BACP), a part-time

degree for individuals already working with pre-school children

Selecting evidence to discuss

22

• Workshops were designed so that HEIs could select relevant examples for their students, and update as required.

• Studies relevant to Strathclyde students were chosen for the pilot.

Selecting evidence to discuss

23

• Sosu & Ellis (2014). Closing the Attainment Gap in Scottish Education. York: Joseph Rowntree Foundation. http://www.jrf.org.uk/sites/files/jrf/education-attainment-scotland-summary.pdf

• New, relevant to Scotland

Policy example: all pilot courses

24

• All were US What Works Clearinghouse Quick Reviews:

• PGDE: Closing the Social-Class Achievement Gap: A difference-education intervention improves first-generation students’ academic performance and all students’ college transitions.

• BEd: Reciprocal Teaching: Students with learning difficulties.

• BACP: Head Start Impact Study: Final report• All http://ies.ed.gov/ncee/wwc/pdf/quick_reviews

What Works examples: variedby course

25

• Each workshop comprised five activities:

1. an individual Pre-workshop Questionnaire, asking about current uses of research evidence, its sources and utility, barriers and facilitators to its use, and views about research;

2. an activity on Views of Research, asking whether statements about educational research were or were not close to the participant’s personal views, with group discussion;

3. group discussion focussed on questions about Research Summary 1, the selected policy research summary;

Aim 3: workshop activities

26

4. group discussion focussed on questions about Research Summary 2, the WWC Quick Review, and

5. an individual Post-workshop Questionnaire, asking about how research could be made more useful, about students’ experiences of participating in the workshops, and about any personal changes resulting from workshop participation.

• A Follow-up Questionnaire was sent a month later asking about any further changes in views following the workshop.

Workshop activities

27

• Workshop materials were piloted with a post-graduate student with experience as an ITE teaching associate, as a primary teacher and as a teacher of children with learning difficulties, to clarify procedures and timing

• Then with BA Childhood Practice students

Piloting workshop materials

28

• The workshop materials proved feasible, and uncovered relevant information

• Final versions of the materials are available on the HEA website

Outcomes

29

• Recordings of discussion were transcribed, and thematically analysed

• Nine key themes emerged• Further piloting could uncover further

themes

Aim 4: emerging themes

30

• A qualitative analysis was undertaken, with too few participants in the pilot workshops to allow quantitative measures

• For discussion, we will consider five of these themes, and their implications, and consider some views on research

What did participants say?

31

• Teacher’s role - including barriers to undertaking research (e.g. work pressure, time constraints, and the belief that research is the job of academics), as well as the belief that it is the teacher’s role to adapt research to classroom context.

• “What I’ve heard in schools I’ve been in is that research is the job of the academics in the universities but we just want a summary of ‘How’s this going to help me, what I need to do, why does it help, and how do you know it will?”’ (W1)

Theme One

32

• Identifying practical application/s of the research, potentially useable within personal practice.

• “For example like the after-school club not just being ICT because that is the traditional one. … But this is saying not necessarily, ‘supported study’ is more helpful so if we can change what clubs we’re offering, it has a big impact.” (W1)

Theme Three

33

• Challenging/questioning the research example’s methodologies - such as selection of population characteristics.

 • “ What do they mean by ‘learning disabilities’,

what’s ‘adequate (pre-coding) proficiency’, who - what pupils do they have?” (W1)

 • “What was the temperament of the children,

things like that?” (W2)

Theme Five

34

• In/accessibility of the research message• “It’s not normal language (academic jargon) and

how it’d be spoken. There’s so much pressure in schools and the pace goes so quickly, there's no time to sit and think, ‘What does that mean?’, and if I need to go to the effort to find out what it means, I’m not going to bother reading it at all.” (W1)

• “It’s not user-friendly, isn’t it not?” (W2)

Theme Six

35

• Personal position in a job hierarchy and top-down decision making.

 • “If I take anything back and try to discuss it

you’re kind of closed down; nobody’s willing to listen.” (W2)

Theme Eight

36

• The policy research extract was considered accessible, with its messages understood

• The ‘What Works’ Quick Reviews were not considered accessible

• Although written for a professional audience, they require understandings of how study quality is evaluated, and how systematic reviews are constructed, which challenged the students

Types of research

37

• Students responded to What Works Quick Reviews by attempting to relate findings to personal experience (Theme 2), and by challenging their research methodologies (Theme 5). They expressed discomfort in discussing the Quick Reviews when they had not understood them (Theme 7), which they said would impact upon workshop participation in less supportive contexts.

Types of research

38

• However, the different responses to policy and What Works research has resonances with some teachers’ views, summarised by Connolly, (2009)

This may not be a common reaction

‘Practitioner’ research is

associated with emancipatory,

democratic and theoretically-

informed approaches that

encourage reflective practice.

Whereas ‘What Works’ research is

seen as an oppressive, dictatorial,

descriptive and theoretically naïve

approach that stifles reflective

practice.

41

• The distinction may also be due to the underpinning evaluative principles of ‘What Works’ research being unfamiliar to students

• This could make it difficult for education students to engage with current ‘What Works’ research initiatives, and to understand the outcome metrics used by researchers such as the Education Endowment Foundation (EEF)

Issues for ITE

42

• The relatively new applications of What Works approaches to classroom practice may need to be addressed in ITE courses

• This has implications for HE, and its distinctive contributions to ITE

• There are also implications for co-professional education and co-professional working

Issues for ITE

43

• On post-workshop questionnaires, participants answered ‘yes’ to all questions about whether they had changed their views following workshops. For example:

• “Developing my understanding of analysis/methodologies.” (W1)

• It has encouraged me to think more about, are there any weaknesses to research?” (W2)

• “Finding out that research and implementation of practices are not always evaluated and clearly written.” (W2)  

• On the follow-up questionnaire, a student noted that she had accessed the web sites suggested, and found them very useful.

Impact of workshops

44

• Similar workshop approaches, or indeed replicating the workshops with course- relevant research examples, may be a good means of encouraging student engagement with research, and allowing them space to discuss their views.

Issues for ITE

45

• The workshops proved a viable means of assessing and discussing students’ views

• There was a suggestion that discussing research examples could influence student views

• A possibility was raised of different understandings of policy research and What Works research, which has implications for ITE educators.

Conclusions

46

• Thank you for listening.

Thank you

47

• BERA (2014). The Role of Research in Teacher Education: Reviewing the Evidence. London: BERA

• Connolly, P. (2009) Paradigm Wars, Evidence and Mixed Methods in Educational Researchhttp://www.paulconnolly.net/publications/ pdf_filesTERN_Presentation_2009.pdf

• Florian, L. & Pantić, N. (2013), Eds. Learning to teach. Part 2: Exploring the Distinctive Contribution of Higher Education to Teacher Education. York: HEA

• Helmsley-Brown, J. & Sharp, C. (2003). The use of research to improve professional practice: a systematic review of the literature. Oxford Review of Education, 29 (4) 449 – 470.  

• Schleicher, A. (2012), Ed. Preparing Teachers and Developing School Leaders for the 21st Century: OECD Publishing.

Key readings