Primary Processes and the Pursuit of Quality: Past and Present Perspectives
-
Upload
shanecolvin -
Category
Economy & Finance
-
view
361 -
download
0
description
Transcript of Primary Processes and the Pursuit of Quality: Past and Present Perspectives
www.nifustep.no Page 1
Primary processes and the pursuit of quality: Past and present perspectives
Bjørn Stensaker
www.nifustep.no Page 2
Past perspectives and debates….
Evaluation as a phenomenon in general emerging after WWII
a) Overarching belief: ”Social engineering”, positivism, quantification
b) Two forms of evaluation schemes:
- PPBS (future oriented, policy-relevant techniques: cost-benefit analysis, system analysis, etc)
- Social programme evaluation (the controllable experiment)
www.nifustep.no Page 3
Past perspectives and debates cont.
Critique of positivism (is man really rational? Is generalisation possible when the context change?
Change of perspective:
- from a belief that those in charge knows best..(”outside perspective”)
- to a belief grounded in enlighenment of those implementing reforms/practitioners (interpretative, hermeneutic perspectives -> extreme case: social constructivism) (”inside perspective”)
- from that time, one can identify a continuing debate between proponents of the two perspectives
www.nifustep.no Page 4
Current perspectives and debates
Central issue in the debate: how to deal with values? (what criteria should be used as ”benchmark”?)
- validisation: is it true?
- valorisation: is it good?
Dominant evaluation perspectives in use:
- Evaluation ”in context”
- From summative to formative evaluation
- Re-instate rationality (evidence-based approach)
- Pluralist approach (using competing theories and perspectives)
www.nifustep.no Page 5
Evaluation in Higher Education today
Evaluation schemes a reflection of a changing policy context:
- downsizing and transforming welfare states (relevance, ”the evaluative state”)
- resource problems (growth in number of HE-students) (efficiency)
- accountability issues (effectiveness)
Quality = relevance, efficiency and effectiveness put together?
www.nifustep.no Page 6
Evaluation: a key to understand the primary processes of HE
Evaluation used to:
- determine reputation (rankings..)
- resource allocation
- appraisal of knowledge (peer review)
- certification of students
- legitimation of academics
- ranking of students, academics and institutions
- promotion to academic posts
- etc, etc….
www.nifustep.no Page 7
The different underpinnings of peer review
Is peer review:
- based on general consensus of values and understandings (within the discipline)
- just a process to enforce the autonomy, creativity and integrity of the individual academic?
- a conservative process emphasising values of the past, unable to recognise and appriciate innovation?
- or, a power struggle between competitive groups within certain disciplines or fields
www.nifustep.no Page 8
The development of peer review:
”extended peer review”
- new groups of academics involved (need for an ”outside view” of new academic spesialities/tribes and territories)
- people from business, industry or society more involved
- students are given a voice as members of evaluation panels
- ”the ordinary man” included in evaluations where ”big issues” are dealt with (technology assessment, etc)
More focus on legitimacy of the process? Evaluation as a political process
www.nifustep.no Page 9
From Method (evaluation) to Content (quality)
How to evaluate what academics do? (directly, indirectly, quantitative, qualitative)
What form of QA is most appropriate for HE? (control, enhancement, finding a balance?)
How can academic standards be secured and assessed? (examinations, relevance)
How can the outcome of HE be assessed? (employment, relevance, ”happiness”, democracy)
What are the links between QA and the improvement of teaching and learning? (is there a causal model, poor methodology)
www.nifustep.no Page 10
Studies of quality focuses on four central themes:
Course evaluation
Grading and outcomes
National monitoring practices
System standards
www.nifustep.no Page 11
Course evaluations
Are current course evaluation techniques effective?
- ”happy sheets” (problem with response rates)
- ”student engagement questionnaires”
What is measured in student assessment of teaching?
Shevlin et al (2000): is there a ”halo effect” in student evaluation of their teachers? Charismatic teachers receive better assessments…(but, can this research be trusted, or is the result dependent on the design of the study..?)
Wiers-Jenssen et al (2002): what trigger student satisfaction? Academic quality..(but also institutional size, buildings, etc..)
www.nifustep.no Page 12
Grading and outcomes
How do/should lectures assess their students?
- coursework receive higher grading than formal examinations
- differences between academic fields
- argument for external examiner schemes (Norway, Denmark and the UK)
- cultural differences between countries (Bologna)
- In general, grading not high on the institutional/public policy agenda (how are teachers trained in grading procedures…?)
www.nifustep.no Page 13
National monitoring practices (NMP)
Research and teaching monitored independently, and with very different consequences
What is the effect of NMP?
- shifts in power distribution
- bureaucratisation
- permeability (exposure of HE)
- public relations (active use of rankings to establish a positive image externally)
Very few studies show a link between NMP and improvement in teaching and learning
www.nifustep.no Page 14
System standards
What is the role of evaluation in HE?
- an instrument to instigate change?
- an instrument to monitor change?
Why this focus on ´quality´?
- how should ”good” QA-systems look like?
- what is the role of various actors in such systems?
- how can current QA-systems be transformed?
Conclusion: quality is an integrated part of the governance of the HE-sector