Innovations

Higher Education40: 69–97, 2000.© 2000Kluwer Academic Publishers. Printed in the Netherlands.

69

Implications for evaluation from a study of students’ perceptionsof good and poor teaching

DAVID KEMBER ∗ & ANTHONY WONGHong Kong Polytechnic University, Hung Hom, Kowloon, Hong Kong (∗Correspondingauthor: Educational Development Unit, Hong Kong Polytechnic University, Hung Hom,Kowloon, Hong Kong; Phone: (852) 2766 6274; Fax: (852) 2334 1569; [email protected])

Received December 1 1998; accepted March 20 2000

Abstract. Many standard teaching evaluation questionnaires have been criticised as beingbased upon didactic models of teaching, and there are also concerns about extraneous factorsbiasing responses. These issues are examined in the light of a study of students’ perceptionsof good and poor teaching from interviews with 55 Hong Kong undergraduate universitystudents. The interview transcripts suggested that perceptions of teaching quality form as aninterplay between the student’s conceptions of learning and the beliefs about teaching of thelecturer. The students’ beliefs about learning can be placed on a continuum between passiveand active learning. Their perception of the instructors’ beliefs about teaching range betweentransmissive and non-traditional teaching. The quality of teaching is then conceived in fourcategories which are the quadrants formed by the intersections of the representations of beliefsabout learning and perceptions of teaching. The quadrants are examined in turn to reveal howstudents with active and passive beliefs about learning conceive quality in transmissive andnon-traditional teaching. The results suggest that responses to questionnaires would be biasedby the students’ conceptions of learning. They also confirm the significance of implicit modelsof teaching in questionnaire design.

Evaluation of teaching

The use of student feedback questionnaires to evaluate teaching has becomealmost ubiquitous. Near the end of every semester innumerable studentsall over the world complete questionnaires about their instructors and theteaching they have experienced. The questionnaires are meant to providefeedback to the teachers with a view to them remediating revealed areasof weakness. In addition, or as an alternative, the ratings of teaching maybe used judgementally to inform decisions about hiring, contract renewal,tenure, promotion and funding of courses.

Each time student feedback questionnaires are introduced into a new situ-ation there is an addition to the chorus of lecturers questioning the abilityof students to make appropriate judgements about teaching and the reli-ability and validity of the questionnaires. Over the years these concerns have

70 DAVID KEMBER AND ANTHONY WONG

stimulated a substantial body of research into various aspects of the validityand reliability of student feedback questionnaires. Marsh (1987) providesa thorough and comprehensive review of this work. More recent and moresuccinct reviews have been produced by Feldman (1996a, 1996b).

In research terms the question of the ability of students to give validjudgements on teaching has been commonly formulated as an examinationof whether factors beyond the control of the teacher influence or contam-inate the student ratings. The search for ‘bias’ has included factors such asclass size, level of the course, academic discipline, gender of the teacher,major or subsidiary subject and the grade of the teacher. A large numberof research studies have looked at bias by typically looking for correlationsbetween ratings on questionnaires and other variables. Marsh’s (1987) reviewrecognised that variables such as class size, prior subject interest, workload,difficulty and expected grade could have a small effect of between 5 and25% of the variance on student ratings. The level of the ‘bias’ dependedupon the academic environment and varied according to the nature of theitem. For example, questions about interaction and rapport were particularlysusceptible to class size effects.

Kwan (1999) pointed out that the proportion of total variance was anappropriate way to examine the theoretical effect of other factors. However,their practical significance was better measured by the effect size on theratings, since it is the rating values which are used in judgements. He useda multivariate design to examine the effect of five course variables uponratings of six dimensions of teaching. Academic discipline and class sizewere found to have the effect sizes with the largest differences between sub-groups. Husbands (1996, 1997) found that there were variations in the ratingsof individual teachers both within the same course by mode of teaching andbetween courses for the same teaching mode. Course characteristics appearedto have a greater effect on these discrepancies than teacher characteristics.The strongest effects were from enrolment size, number of lecturers teachinga course and general satisfaction with the course.

A further issue of validity, which has attracted attention recently, comesfrom the use of standard questionnaires throughout a university or faculty.When this is the case, as it most commonly is, the selection of itemsdefines the dimensions of teaching which are evaluated. There then arisesthe important question as to whether the items and dimensions measuredare appropriate and valid measures of the teaching and learning situation.In technical terms the issue is that of construct validity. Furthermore, the useof a standard instrument throughout an institution implies that it is applicableto all modes and types of teaching.

GOOD AND POOR TEACHING 71

As there are so many questionnaires in existence, however, it is notstraightforward to generalise. Many instruments have been developed withoutrecourse to attempts to ensure construct validity of the dimensions examined.Tagomori and Bishop (1995) examined 200 questionnaires used by accreditedschools of education in the US and found that over 75% had items whichwere confusing, ambiguous or subjective. That such a high proportion hadproblems with the relatively straightforward issue of item design suggeststhat few could have paid much attention to the more complex question ofconstruct validity.

Even with well designed and researched instruments, the validity of itemsand dimensions is problematic. Hinton (1993) argued that the lack of a univer-sally accepted model of good teaching is a handicap to the developmentof questionnaires which can claim to be valid measures of good teaching.Marsh (1987) describes a typical approach to the selection of items for goodinstruments as being based upon a logical analysis of the content of effectiveteaching, supplemented by literature reviews of dimensions others have used.The most frequently cited set of dimensions is probably that of Feldman(1976), which appears to have influenced the design of a number of otherinstruments.

Recent writing has criticised the selection of dimensions in the mostwidely used US instruments as being based upon too narrow models andmodes of teaching and learning. A hint to the issue can be gained just bylooking at Feldman’s (1976) list of nineteen instructional rating dimensions.No less than eleven of the nineteen begin with the word ‘Teacher’s’, whichadds credence to Centra’s claim (1993, p. 47) that ‘the typical student ratingform is devised to reflect effectiveness in lecture, lecture and discussionand other teacher-centred methods’. D’Apollonia and Abrami (1997) arguedthat typical feedback questionnaires are based upon models of instructionfocussing upon traditional didactic teaching. McKeachie (1997) pointed outthat student rating forms gather information about conventional classroomteaching. Almost all ignore the learning which takes place outside theclassroom, which is probably the majority for many students.

Evidence that standard feedback questionnaires do not relate to innova-tive teaching comes from the Action Learning Project. This has supportedninety action research projects in universities in Hong Kong, all of whichhave been or are being evaluated and reported (Kember, Lam, Yan, Yum andLiu 1997). Hardly any of the project teams utilised the standard evaluationquestionnaires available in their universities, simply because the qualitiesevaluated did not relate to the specific changes or innovation introduced bythe project. Instead each project had to devise customised forms of evaluation,


most commonly employing a combination of interviews and questionnaires(Kember, 2000).

Kolitch and Dean (1999) examined a typical US evaluation instrumentagainst two models of teaching. They found it compatible with a transmissionmodel but not with an engaged-critical one. The article went on to questionthe neutrality of instruments which did not acknowledge forms of educationinvolving high levels of student involvement.

This study examined both these strands of criticism of student feedbackquestionnaires. It looked at the issue of bias by examining a form of influ-ence which has not been widely examined by others – namely the way inwhich the student’s conception of learning influences the ratings they giveto dimensions of instruction. As this form of bias was found to interact withthe beliefs about teaching of the lecturer, the study also related to criticismsof forms of evaluation based upon implicit models of teaching. It shouldnevertheless be pointed out that these two issues did not frame the studyas research questions or hypotheses. Rather, the study started with an openqualitative exploration of students views on good and poor teaching. Theissues of conceptions of learning and beliefs about teaching emerged throughthe analysis as frameworks to interpret the data.

The study investigated perceptions of teaching quality from an openframework, seeking the students’ perspective. We attempted to determinehow students perceive good and poor teaching. The aim was to try to discoverthe dimensions, constructs or descriptors which students themselves use toconceive the quality of teaching. The questioning was, therefore, carefullydesigned so as not to impose our perspectives or framework upon the students.In interviews, students were asked to describe types of teaching they learntmost or least from.

Method

This study was part of a project which aimed to conduct an open examinationof the holistic experience of university students in Hong Kong. A sample ofstudents were interviewed to provide an in-depth picture of aspects of thestudent experience.

The interviews were semi-structured, with questions which focused uponthe following topic areas.• Courses and teaching methods• Workload• Study habits and methods• Interpersonal contact with peers and tutors• Language use


• Out-of-class learning• Learning in the classroom• Leisure activities• Overview

Between two and four questions were prepared for each topic to ensure thateach interview covered the topics. Once a topic was initiated, however, theinterviewer conducted the interview in an open interactive manner. All inter-views were conducted in Cantonese and were tape-recorded. The interviewslasted for about one hour each. The interviews were translated into Englishand transcripts were then introduced into the NUD·IST program for codingand analysis. NUD·IST is a computer program for assisting with the analysisof qualitative data, with facilities for indexing, text-searching and combiningdata from different independent studies (Richards and Richards 1991).

Sample

The total sample were asked to complete a diary which was not used in thisparticular part of the overall project. This total sample was drawn from seven-teen classes, chosen across the seven universities in Hong Kong, so as to givea representative spread of disciplines and year of study. The classes selectedcontained students enrolled in full-time undergraduate degree courses. Aclass was defined as between 25 and 50 students. For larger classes, sub-unitssuch as tutorial groups were selected.

The sub-sample for the interviews was a random sub-sample of theselected classes. Four or five students were selected from each class, soin total 55 students were interviewed. If those originally selected were notwilling to be interviewed a further selection was made. This did not happenoften, though, as students were generally quite willing to be interviewed. Thestudents were all enrolled in full-time courses so the large majority wouldhave either enrolled directly after completing secondary school or within ayear or two of doing so. The remaining students would mainly be those whofirst completed a sub-degree course to qualify for university entrance. Mostof the students were, therefore, in their late teens or early twenties. Virtuallywithout exception, the students were from Hong Kong and had Cantonese asa first language.

Analysis

The analysis for this paper concentrated upon sections of the transcriptconcerned with learning in the classroom. In the interviews, students wereasked to describe types of teaching they learn most and least from. Most of thedata utilised in this paper were drawn from this section of the interviews. The


researchers did, though, refer to the entire transcript in each case to ensurethat any extracts used were consistent with views expressed on other topics.

Transcripts were initially sorted with NUD·IST so that responses tosimilar questions were grouped, but with the respondents still identified. Theprocess is akin to forming a variable-by-respondent matrix, though in thiscase the volume of data was far too large to portray on a two-dimensionalsheet. Having performed this initial ordering of the data, it was then possibleto compare responses from all interviewees to similar questions.

Once the data had been ordered it was possible to search across inter-viewees and courses for common responses or constructs with respect tosimilar questions or variables. Categories and constructs which emerged fromthe data were discussed within the overall project team for conceptual verifi-cation. No attempt was made to categorise each and every statement intoa limited number of categories. The process was more in line with seekingimportant conceptual insights. The quotations selected for this paper are thosewhich represented significant constructs which appeared across the range ofstudents and courses.

Framework for perceiving teaching quality

In our initial examination of the transcripts it became obvious that searchingfor sets of universally agreed good and poor qualities would not be fruitful.There did seem to be a measure of agreement over some courses or partic-ular qualities which were liked or disliked. There also seemed to be quitemarked differences between some of the interviewees as to the way good andpoor teaching was described. There were clear instances of some studentsconstruing a form of teaching in a positive light while others disliked it. Theway students perceived the quality of teaching appeared to depend upon boththeir own beliefs about teaching and learning and those of their teacher.

Eventually, after close examination of the transcripts, we found a frame-work which enabled us interpret the student perceptions and to rationalisethe instances which initially appeared contradictory. Our analysis of the datasuggested that students’ perceptions of the quality of teaching form as aninteraction between the student’s own beliefs about learning and their percep-tion of the teacher’s beliefs about teaching. Students’ beliefs about learningcould be classified as passive or active. The distinction was better portrayedas a continuum between the two poles rather than two mutually exclusivecategories.

This description of student beliefs about learning differs from, but isconsistent with, other categorisation schemes for describing beliefs about,or conceptions of, learning. Säljö (1979) advanced a five-category set of


descriptors for conceptions of learning. In hierarchical order the categorieswere: learning as the quantitative increase in knowledge; learning as memor-ising; learning as acquisition of facts, procedures, etc. which can be retainedand/or utilised in practice; learning as the abstraction of meaning; andlearning as an interpretative process aimed at the understanding of reality.Marton, Dall’Alba and Beaty (1993) subsequently identified the same fiveconceptions and added a sixth, which applies more particularly to adultstudents, namely changing as a person.

There are also a number of developmental models of intellectual capacity.The one which has probably been most widely cited in application to univer-sity students is that of Perry (1970, 1988). Perry characterised students’cognitive and intellectual development by nine observed positions rangingfrom the unswerving belief in the correctness of authorities, through a morerelativistic understanding, to the ability to evolve and evaluate personalcommitments.

The correspondence between these categorisation schemes and ourcontinuum is that those in the lower positions of either hierarchy wouldclearly have a logical preference for more passive styles of learning. Iflearning is conceived as some form of reproducing remembered information,or knowledge is seen as black and white, learning can only be a processof absorbing the authoritative word of the teacher. Attaining a higher stageof development in Perry’s hierarchy would have to be accompanied by aconception of learning which seeks meaning or understanding and this canonly be achieved through an active engagement with the learning task.

In view of later reference to the approaches to learning constructs, it isappropriate to consider the relationship between active and passive concep-tions of learning and deep and surface learning. It should be clear that theconstructs are not the same, but they are likely to be related. When Gibbs(1992) describes course design and teaching methods which foster a deepapproach he concludes that ‘Students need to be active rather than passive’(p. 11). Biggs (1999) compares cases of students using the two dimensionsof low to high level task engagement and the extent to which the teachingmethod requires student activity. He argues that for teaching methods thatrequire little activity, like the standard lecture, there is likely to be a range oflevels of engagement. Committed students can be using a deep approach butthose with limited motivation will take a surface approach. As the teachingapproach requires more activity the distinction between the students is likelyto lessen.

This framework will be seen to fit well with the analysis of our study. Thestudents with active conceptions of learning could all be classified as usinga deep approach. Those who had a passive conception of learning are less


clear cut. Some clearly were taking a surface approach. Others were seekingunderstanding from passive learning situations like lectures.

Academics’ beliefs about teaching were seen by the students as trans-missive or non-traditional, and again could be ranged on a spectrum. Thisagain is consistent with research into academics’ conceptions of teaching.A review of thirteen studies (Kember 1997) synthesised the outcomesinto a spectrum with five well identified positions ranging from impartinginformation to conceptual change/intellectual development.

Our category descriptors will become clearer as we explore the fourpossible combinations of beliefs. The one which is the most diffuse is thenon-traditional belief about teaching. For the moment we will define it asany conception which is not transmissive. This is not purely an ‘any other’label as many of the students did seem to define the category as teachingwhich was not transmissive or didactic. Others had a more positive visual-isation. There was not always complete consistency between students, butthe common elements were that students were required to take an active rolein the learning and teaching process and that there were significant levelsof interaction between the students and the teacher, and usually among thestudents too.

Having determined two categories each for beliefs about learning andteaching, they were then related in a two dimensional framework. The frame-work implied four quadrants which formed as the beliefs about learning andteaching intersected (Figure 1).

The framework is shown as two intersecting continua. A student’s beliefabout learning or their perception of their teacher’s belief about teachingcould be positioned at points along each axis. The intersection of the twopositions results in a perception within one of the four quadrants of thediagram.

When we returned to the transcripts we found that the framework enabledus to make sense of the students’ descriptions of good and poor teaching.Instances where some students judged a form of teaching to be good whileothers rated it poorly could now be explained by whether there was compat-ibility between the student’s belief about learning and the lecturer’s beliefabout teaching. Essentially we now had to interpret good and poor teachingwithin each of the four quadrants formed by the intersections of beliefs aboutlearning and teaching. Diagrammatically we have portrayed this in Figure2 by placing boxes within each of the quadrants of our two dimensionalgrid. Three of the boxes are allocated two compartments to allow us to placethe dimensions of good and poor teaching derived within each of the quad-rants. The quadrant for passive learning and non-traditional teaching onlyhas a compartment for poor teaching. This is because there is incompati-


Figure 1.

bility between the forms of teaching utilised by those with non-traditionalbeliefs about teaching and the expectations of students with passive beliefsabout learning. The passive learners see all non-traditional teaching in anegative light because it challenges their desire to act as passive receiversof knowledge.

There is also an element of incompatibility between the beliefs aboutactive learning and transmissive teaching. However, this quadrant does havea compartment labelled ‘necessary teaching’ because the students ascribed tothis conception recognised that didactic forms of teaching were necessary attimes as part of a mix with other teaching strategies.

The next step was to return to the transcripts and examine the students’descriptions of good and poor teaching. In each instance it was necessaryto discover the student’s belief about learning and determine the student’sperception of their teacher’s belief about teaching. This enabled us to sort


Figure 2.

the descriptions of good and poor teaching into the compartments in Figure2. Once this sorting had taken place we could look for commonality amongthe descriptions. For most of the compartments it proved possible to classifythe descriptions into a limited number of dimensions which the students usedto describe good and poor teaching. These dimensions are shown in theirrespective compartments in Figure 3.

In the next main section of this article the compartments are examined inturn to illuminate the dimensions the students used to describe good and poorteaching. Before doing so, however, it is worthwhile making four generalpoints. Firstly, the dimensions placed within a compartment are not neces-sarily exclusive to that compartment. There are dimensions which operateacross quadrants. Some forms of teaching were universally judged to be poorand elements of good teaching could be perceived similarly by those withdiffering beliefs.


Figure 3.

Secondly, we do not claim that the list of dimensions within each compart-ment is comprehensive. In the interviews the students were asked to describethe most striking aspects of good and poor teaching rather than to give afully detailed description. Our analysis concentrated upon finding a suitableframework for interpretation. In searching the transcripts we concentratedupon commonly found dimensions rather than recording every single one.

Thirdly, the unit of analysis was the dimension of good or poor teaching.This meant that each interview yielded several dimensions for analysis andthese could be located in two quadrants. For example there is a student quotedat some length whose views on poor teaching were classified in the activelearning/transmissive teaching quadrant while opinions about good teachingwere located in the active learning/non-traditional quadrant.

This mode of analysis makes quantification even more difficult than itnormally is for qualitative studies of this type. What can be said is that


dimensions of teaching were only included in the analysis if described byat least four students. The quotations illuminating the dimensions are typicalof those placed in a category, though wording obviously varied considerably.In terms of the quadrants, about three-quarters of the descriptions analysedwere categorised in the passive learning/transmissive teaching quadrant. Therest were roughly evenly divided between the remaining three.

Finally, we feel it important to stress that students’ beliefs about learningand their perceptions of teaching are not fixed and immutable. There was not agreat deal of evidence of this within our transcripts because the students wereinterviewed only once. The data are, therefore, in the form of a snapshot,revealing beliefs at that point in time. They would not show evidence ofchange. Many other studies, though, which have involved the introductionof innovative forms of teaching, do show evidence of students adapting tothe new teaching style (e.g. Gibbs 1992; Weeks and Scott 1992). It is quitecommon for those who try novel forms of teaching to report initial discomfortor resistance from students. Typically these students have become used todidactic forms of teaching so have become accustomed to behaving passivelyin class. Eventually many recognise that the more innovative style results inbetter learning outcomes and come to prefer it.

The context

At this point it is appropriate to say something of the context so that thereader is better able to understand the findings. It is usually accepted thatthe predominant mode of teaching within secondary schools in Hong Kongfollows a didactic model (Biggs 1996). Class sizes are around forty pupils formost classes. Curricula and frequent examinations are set by the educationdepartment. Examinations are taken very seriously as entry to both the finaltwo years of senior school and university is highly selective (Salili 1996).

The diligence of the students in studying for examinations is also influ-enced by the Confucian tradition which means that Chinese culture showsgreat respect to scholars and teachers and values hard work, effort andendurance (Yang 1986). Salili (1996) reviewed the literature on achieve-ment motivation among Chinese students. She noted the cultural impact ofa strong sense of collectivism manifest in loyalty to the family and othersocial groups. Academic success would be seen as a source of pride for theentire family unit. Learning and education are valued implicitly as well asfor social advancement (Ho 1981). The selective entry to the final part offormal education imply considerable material rewards to those who followthe tradition of diligent study and material wealth is certainly valued andsought by Hong Kong society.


Students, therefore, enter university with considerable experience ofdidactic teaching and a history of diligently learning material to pass exami-nations. There has been a widely prevailing view in Hong Kong universitycircles that the students are only capable of learning by rote from didacticteaching. This position is no longer so prevalent as it has been strongly chal-lenged, by evidence which shows that it a misperception (Kember and Gow1991; Kember 1996; Watkins and Biggs 1996). However, it is probable that,in Hong Kong as in many parts of the world, a high proportion of universityteaching still follows a didactic model.

Passive learning/transmissive teaching

The descriptions of good and poor teaching start with the quadrant containingtransmissive teaching and passive learning. Given the above description of thecontext it is hardly surprising that the large majority of the reports of goodand poor teaching were allocated to this quadrant. The greatest number ofdimensions were also found within this quadrant, presumably because of thevolume of description.

Good teaching

In following the description it will be seen that the qualities associated withgood teaching follow logically from the model of teaching and learningassociated with the quadrant. Teaching is seen as a process of transmittingknowledge which is absorbed by the body of students sitting in the classroom.It is, therefore, perfectly rational to find good teaching described in terms ofgood presentation. The students expected material to be presented in a logicalsequence and clearly presented. They expected the lecturer to strive to presentthe material in such a way that it could be understood.

Systematic, step-by-stepThe first of the categories of good teaching perceived by the passive studentsis the category of logical organised delivery, characterised by a systematicstep-by-step covering of the content.

There are some lecturers who can teach you step by step. They can showyou the details.

One of the components of passive belief about learning is an inclination tofollow. The kind of teaching that allows students to follow step-by-step easilywill be perceived as good teaching.


Clarity of informationWhen students visualise teaching as a process of the lecturer supplyingmaterial while they write it down, the clarity of information in teaching willbe much appreciated.

In fact, there are a few subjects in which the teaching is quite good. Iremember that a subject named [. . . ] I took last year isgood. The lecturerused transparencies in his/her teaching. The transparencies were very welldesigned, with so many pictures. I think those were very attractive for meto listen to in the class. . . .

In fact, this student demanded not only good transparencies, but also handoutsthat were easy reading, and books that helped to answer the multiple choicequestions in the examination. Teaching that included all these was designatedas good.

Trying to make students understandA further related dimension of good teaching perceived by passive studentswas that of the lecturer aiming for student understanding. It is related toand quite commonly found in conjunction with the above dimension. It was,though, spoken of separately from clarity of information and thus meritedclassification as a discrete dimension.

Q: Is there any better kind of lecture?

Yes. Like the subject of [. . . ]. Notes are provided in this subject and theteacher has made so much effort to try to make us understand. Althoughwe don’t really quite understand, we would try our best to listen in thelecture.

This quotation shows that a passive approach to learning does not alwayscorrespond to surface learning. In many of the earlier quotations, though, itwas clear that the students were predominantly employing a surface approach.

Poor teaching

Just as the qualities of good teaching were logically related to the model ofteaching and learning framing the beliefs inherent in the quadrant, so toowere the dimensions of poor teaching. In fact they were the opposite tothose of good teaching. Poor organisation was expressed as either a lack ofclarity in course goals or circuitous structuring of material. Lack of claritywas commonly seen as arising from presentation which was too rushed.


Poor organisationThe poor organisation dimension was often expressed as a failure to provideclear guidance about the syllabus for the course, as in the following example:

I’d better say that the teaching method that I learn least from. For example,if the lecturer doesn’t tell us his/her requirements, I think I will not studythe subject on my own initiative. If the objectives of the subject are notclear enough, I will be less willing to study.

In other words, the student would like to see that the lecturer would organisethe lecture or the materials to be taught around a number of specificrequirements which could be followed accordingly.

Round and round in circlesAnother teaching style criticised by the students was that of the teachertalking about irrelevant things in the lecture. For example:

In some lectures, the teachers may lead you up and down the garden path[translated from a Cantonese expression]. That is, they don’t teach theimportant thing but say something irrelevant. For instance, one of myteachers liked to say something irrelevant such as his journey, uncles andAustralian farms when talking about the topic related to Australia. As aresult, most of us fell asleep in that period.

When the teacher did get back to the original topic and said somethingimportant in the lesson, the students then found it difficult to grasp:

However, the teacher suddenly said something important suddenly. Thismakes it very difficult to get the meaning.

RushingIn our interviews, many students commented that their teachers frequentlyfailed to keep the speed of their teaching at an appropriate pace for thestudents’ learning – invariably the teaching was too fast, as in the followingexample:

On the other hand, the most ineffective means is that the lecturer speakstoo fast and always skips the transparencies to be gone through. Simultan-eously, we have to write the notes by ourselves. Thus, listening andwriting can’t be done well.

The intersection of the passive belief about learning and the transmissivebelief about teaching is evident in many of the comments which could beplaced in this category. The students visualised teaching as a process of the


lecturer supplying material while they wrote it down. If it was delivered toofast it could not be noted down properly:

Their performance in teaching is obviously not acceptable. I won’t blametheir teaching method as they may not have received teacher training.However, they should write clearly, speak at a normal and attainable speedand lend the transparencies to the students. Although I have spoken up inthe course evaluation, no improvement has been made.

On the other hand, the desire for clear writing and a reasonable speed ofspeaking seems sensible, regardless of the belief about learning of the student.Another student had a different concern, which should also be applicable forall combinations of beliefs:

I think they are going too fast. Perhaps, it has much to do with the syllabusitself. But I think they should give me some time to digest, to thinkthrough one thing before they initiate something new.

Monotonous readingA few students said that they did not like the style of plain reading becausethey were bored by it.

Q: Can you describe a class that you think is the worst?

Most of the lectures in the course are worst such as the [. . . ]. It is soboring. The lecturer just reads by following the book. He just talks byfollowing the texts written on the transparencies. We just don’t need toattend his class.

From this excerpt and numerous others, it becomes clear that when a lecturebecomes very boring, students follow the passive path, in spite of their discon-tent. There are frequent reports of avoidance behaviour such as falling asleep,reading something else or skipping class. There are no instances of studentsattempting to change the lecturers’ approach whatever the level of their frus-tration, but this is hardly surprising given the status relationship betweenstudent and lecturer and the Confucian tradition of respect for a teacher (Ho1986).

Some of the students said that they did not like plain reading because theteacher did not care whether the students understood or not.

The one who just read the lecture notes continuously.. . . After he hasplaced his transparency correctly, he read it out continuously. He does notcare whether you understand or not. He may ask whether you understandor not, however, we won’t have a problem understanding it since he is justreading the notes out. . . .


Passive learning/non-traditional teaching

For the intersection of the passive learning and transmissive teaching quad-rants, we are dealing with compatible beliefs about teaching and learning.The students with this belief about learning feel comfortable with the trans-missive approach to teaching. They recognise that some transmissive teachingis better than others and the categories given above represent their depictorsof the types of good and bad teaching.

In this next quadrant, however, passive learning intersects with non-traditional teaching. Because there is a mis-match between the students’belief about learning and the teachers’ belief about teaching, the students’reaction is inevitably negative. All teaching of this type is viewed asundesirable.

Different

One of the examples given in this section might have been placed underthe disorganised teaching category section in the quadrant discussed above.The student describes the teaching as disorganised, but it could equally beinterpreted as different to the systematic step-by-step approach the studentprefers.

As far as teaching method is concerned, I think there is a great disparity.Roughly, there are two different types. One is just the same as insecondary school. They teach and guide us step by step. The other isa much more foreign approach. The lecturers talk about anything theythink is important. They don’t teach us systematically. I think I can’tcatch up with the latter approach. I think I prefer a more systematicapproach, which can teach me step by step. I can understand more withthis approach. But if they jump from one part to another, I won’t get much.I may be quite confused.

The student preferred a systematic approach of knowledge transmission andwanted a detailed outline to follow in the lecture. The transcript suggests thatthe lecturer was not necessarily disorganised, but organised in a different wayto the step-by-step transmission approach favoured by the student.

. . . For example, when we learn something about a topic, there would beseveral sub-topics. The lecturer [using the foreign approach] would onlygive us the overarching topic and keep jumping across various sub-topics.I don’t understand them at all.


Active learning/transmissive teaching

This is the other quadrant featuring incompatibility between the teachers’belief about teaching and the students’ belief about learning. There was,therefore, considerable evidence of students diagnosing poor teaching whichcould be placed in this quadrant. However, there were also cases of studentsrecognising that some degree of transmissive teaching was needed at times.We felt that the more positive sub-section was better described asnecessaryteaching. This fitted better to the students attitude to this type of teachingwithin this quadrant, as the students were not strongly positive about it butdid see its place.

Necessary teaching

So far we have only discovered one positive category within this quadrantwhich corresponds to teaching incorporating a variety of approaches andstrategies. The quotations make it clear that it is the way students make useof the elements in the teaching mix which gives those elements value.

Multi-faceted teachingThe following student construed transmissive teaching as a necessarycomponent in a mix.

We have lectures and tutorials. I think they have a different nature. Welearn passively in lectures. The lecturer or professor teaches us. Thenwe read books by ourselves. We do assignments in tutorials. Then wereport to other classmates. Lectures provide the theoretical frameworkand tutorials gives us a chance to practice. They are interdependent. Bothare needed. If we only have lectures and you do not read books, it ismeaningless in your studying. Tutorials act as a guide. Then we can applywhat we have learned.

We can see that this student is concerned not only with receiving knowledgepassively, but also with making use of the knowledge received for furtherunderstanding. Whether the knowledge could be acquired depended verymuch on the student’s effort in preparation and reading.

We have lectures but there is nobody to force or encourage us to learn.In the past, the teachers will show us what we have to study. Now, wehave to read books by ourselves after lectures. If you do not read books,you may not understand what the lecturer teaches. You have to put moreeffort in reading books if you want to know more in-depth. What we learnin lecture is very superficial, we have to find and read more books if wewant to learn in-depth.


So, here we find a student who was able to construct various kinds of teachingas potentially useful sources or means for active learning. Without under-stating the responsibility of the teacher for student’s learning, there wasstress on the student’s responsibilities such as preparation, doing reading afterlectures, doing research for tutorials and making use of knowledge acquiredfor further understanding.

Poor teaching

In this sub-section methods of transmissive teaching will be considered asconducive to poor teaching exclusively for those students who have an activebelief about learning.

Blocking class discussionStudents with an active belief about learning treasure classroom discussionor interaction. They are unhappy with teachers who block discussion in theclassroom.

Once there was a classmate in my group who was so careless that he hadturned the direction of some questions into another point of philosophy. Ithink he should not turn it like that. However, the consequence was thatthe lecturer criticised us strongly. I think it’s all right to criticise, but theproblem was the attitude, and the way of speaking. The way the lecturerreacted, room for discussion was limited.

Merely giving notesWhile those with a passive belief about learning value almost exclusivelynotes or transparencies in teaching, those with a more active belief aboutlearning downplay the importance of them. Poor teaching is when the teachergives them notes and leaves. ‘I just know that somehow someone will appearat a certain time in a classroom where s/he will give us notes and leave’.

Frustration at lack of participationThe student who supplied the above quotation represents an interesting case,and one which can be used to contrast incompatibility within this transmissiveteaching quadrant with much more positive reactions to the non-traditionalteaching which was compatible with her active belief about learning.

. . . you could feel that every word you said would be seriously valued.But in [. . . ] it depends on individual professors. Some professors may bemore traditional, and they don’t like to be challenged. All students needto do is to jot notes.


This student eventually became so frustrated at the mis-match between herdesire for active forms of learning and the prevalent transmissive teachingapproach, that she applied for an exchange scheme.

Q: As you mentioned you studied in US in your third year, is it the courserequirement?

It’s not a course requirement. I applied for it myself. I knew that I wouldhave to spend four years in [. . . ] but I found it’s quite boring after twoyears of study. At that time our college wanted to select some students tojoin an exchange study in New York. So I applied for it . . .

Q: As you said your study was quite boring at the first two years, why hadthat feeling? In which aspects?

I thought it’s because the environment and the relationship between theteachers and students were very boring. Most students would choose tostay here for three years. The main reason I chose to join the exchangeprogramme was that I chose to stay here for four years. But I didn’t wantto waste one year. Also, there was a chance for me. So, I decided to changethe environment of study and tried to study outside of Hong Kong.

Active learning/non-traditional teaching

This is a rare example of a student acting to change a learning environmentfrom a passive to an active one. The fact that she found it necessary to go tothe U.S. to do so, graphically illustrates the difficulties students face if theyfeel the teaching learning environment is not conducive to the type of learningthey wish to engage in. In fact, this student did choose to be passive in somekinds of learning environment, but only because she did not want to risk heracademic result.

Active engagementIn the U.S., she did choose to be active because she found that what she saidwould be valued:

. . . participation was counted. Also when I studied with people who camefrom different places, I might be the only Asian, I thought I needed tobe more active. As an exchange student, which was an important factor, Ithought I needed to be more out-spoken. . . I thought I had a responsibilityto do something good, representing [. . . University] or Hong Kong. I wasan active person at that time.


She chose to be active not only in U.S., but also in an appropriate contextwhere she knew that active participation would be valued, just as she did inthe class of a subject at her university in Hong Kong.

. . . the professor, just as when I studied in U.S., asked you to speak all thetime and you had to speak more.

She found compatibility, therefore, by selecting courses according to theattitude of the professor or the nature of the teaching.

Multi-faceted teachingThe other aspect of that same professor’s teaching which was positivelyregarded was again the use of a variety of teaching strategies.

The means of teaching were diverse, such as playing games, watchingvideos, etc. We needed to prove our talents in various aspects, not justfrom our paper. He would focus on whether we learned it seriously ornot.

Although she designated this subject as the most demanding subject she hadever studied, she said she would nevertheless encourage others to take it.

. . . He didn’t follow our department’s rule and the subject and evaluationscheme were not so well-structured. Instead, he used some abstract termsto guide his teaching. For example, he asked us to design one thing orto do homework and assignments. He was concerned with our innova-tion and creativity . . . Many students quit after the first lesson because itseemed that there was nothing to learn. But I completed it happily.

This student can be considered as a deep achieving learner, for she did displaya motive which is strategically aimed at efficiently achieving a good result.Other students also saw the value of this multi-faceted approach:

There is a topic in the tutorial. There are about twenty people. We aredivided into eight groups. Each of us has a partner. We find data together.We remember what the professor teaches us and make use of the knowl-edge to further our understanding.

. . . We must study [. . . ]. Usually, there is a mass lecture and also atutorial. What do I learn? That is I think I can learn from the [. . . ] tutorialas [. . . ] prefers to use the tutorial as it is closer to the students and thereis more discussion. That is, you focus on one topic in your presentation,and that’s why you can learn more. For my major subjects, since I amhurried in doing my assignments, I may not be able to learn anything. Onthe contrary, I can learn from doing papers as papers focus on one topiconly. I have to do the research and read books and I can learn from it.


It may be noted that within this quadrant we have only reported dimensionsof good teaching. This is simply because we did not find any dimensions forpoor teaching which could be attributed to this quadrant. Students with anactive belief about learning would probably report instances of transmissiveteaching when asked to give an example of poor teaching. Presumably thestudents would find some aspects of non-traditional teaching undesirablebut would not remember them as vividly as instances of bad transmissiveteaching. Given the way the interview questions were asked, it is thereforequite reasonable to find this compartment empty within our data.

Implications for the evaluation of teaching

Our initial exploration of the data was on the assumption that there would beidentifiable sets of characteristics which distinguish good from poor teaching.We had originally assumed that there would be good agreement amongstudents as to the elements of good teaching. However, these assumptions,which also underpin the use of student feedback questionnaires as a means ofevaluating teaching, were not consistent with our findings.

Qualities examined

There is considerable variety in the qualities examined by the innumerablestudent feedback questionnaires in existence. It is, therefore, difficult to makegeneralisations about the degree of match between questionnaire dimensionsand the qualities our interviewees felt were important indicators of teachingquality. However, the following qualities, which typically appear in studentfeedback questionnaires, are likely to be seen as important by those with apassive belief about learning. Students with an active belief about learningwould seem to see these qualities as relevant but judge them to be of lowerpriority than passive students.• organisation• clarity of structure• specification of clear objectives• pace• clarity of communication• workload• level of difficulty

The following qualities may be seen as a higher priority by more activestudents. Passive students, however, might give a low rating to teachers whodisplay these qualities.• stimulation of interest


• promotion of interaction in class• variety in teaching approaches• display of enthusiasm

The other way in which typical questionnaires fail to relate to descriptionsof teaching seen as good by those with an active conception of learning isin the types of teaching about which questions are asked. Looking throughthe typical quotations relating to these respective quadrants, we found refer-ences to learning through modes such as self-directed learning, role play,collaborative learning and project-based learning, yet few standard question-naires make specific reference to teaching and learning modes like these.Similarly, few questionnaires include items asking whether a variety ofteaching approaches were used. Indeed, many questionnaires, implicitly ifnot explicitly, focus almost exclusively upon the standard lecture.

This study, therefore, reinforces claims made in the literature reviewed atthe start of the article namely, that evaluating innovative or student-centredteaching is problematic with most standard questionnaires, which imply ateacher-centred model of the learning process relying heavily upon didacticinstruction. However, the extent to which questionnaires focus upon teacher-centred models of education also seems to depend on whether they aredesigned primarily to provide feedback or are used judgementally. Question-naires for judgmental purposes are more likely to focus upon the instructorand his or her teaching. Following the analysis of Kolitch and Dean (1999),the end result is more likely to be a questionnaire compatible with atransmission model of teaching than an engaged-critical one.

This point can be illustrated by evaluation procedures within our ownuniversity. The first student feedback questionnaire service to be used widelywas a voluntary scheme in which the results were confidential to the lecturermaking the evaluation request. This service was, therefore, solely to providefeedback on teaching and learning within subjects. Wording of items consist-ently focussed upon student outcomes from their experiences in taking thesubject. Two examples from the ‘learning outcomes’ dimension are quoted.

I have been able to understand the subject matter.

I feel that I have learnt and understood the material in this course.

At a later date, the university made plans to introduce staff appraisal sorecognised that it would need evidence of lecturers’ teaching performance.A working group was set up to devise a method for doing so and the outcomewas a revised version of the original feedback questionnaire. The wording ofeach item was revised to include the term ‘staff member’. The items in the‘learning outcomes’ dimension became:


I have understand the subject matter taught by the staff member.

The staff member’s method of teaching has helped my understanding.

The resulting items are still less teacher-centred than in many instructor ratinginstruments, but the effect of the changes has been a shift from items whichwere neutral to the mode of teaching and learning to one placing responsi-bility for teaching upon the staff member. There has been a correspondingshift in moving from items consistent with both teacher- and student-centredmodels of learning towards one which is clearly more teacher-centred.

Changing beliefs

Questionnaires which appear to lecturers to seek feedback on teacher-centredmodels of education may encourage conservative approaches to teaching.This is particularly likely if there is a belief that student ratings will showbias against more innovative teaching because of a preference for passiveforms of learning. If the questionnaire is linked to a staff-appraisal processthe conservatism may be acute.

Student resistance is often cited as a dis-incentive for instructors to intro-duce more innovative forms of teaching. Gow, Kember and Sivan (1992,p. 142) include a quotation which is very typical of the attitude of someacademic staff.

. . . we do have some resistance from students which I suppose ah

. . . students really like to have lectures, use slides, videos, overheadprojection, and other aids. And then as many handouts as possible. That’swhat they like. That’s what they feel comfortable with. If you decide thatI’m not going to do it that way and then make them think, and thenmake them read and do some work and come back and discuss it afterthe lectures, they really don’t like it.

We would not like our study to encourage beliefs like this and hencediscourage teachers from attempting curriculum innovations. There is a greatdeal of evidence which suggests that student resistance can be a significanteffect but is not normally enduring as students can and do come to adapt tomore innovative forms of teaching. A good example comes from attemptsto introduce reflective writing (Kember, Jones, Loke, McKay, Sinclair, Tse,Webb, Wong, Wong, Yan and Yeung 1996). Initially the students found it veryhard to adapt to a course which was much less prescriptive than those they hadexperienced before. They also found it difficult to adopt a reflective style ofwriting, since its personal introspective form was diametrically opposed to theimpersonal third-person forms of academic writing, citing academic sources,


they had been previously taught to follow. In interviews the students said theyexperienced considerable discomfort in the early stages of the course, so hadstudent feedback ratings been sought at that stage, they would probably havebeen quite negative. However, by the latter stages of the semester, most of thestudents felt that they had made reasonable progress towards adapting to thisquite different form of teaching. A substantial proportion had become quitepositive about it, as they could see how reflection upon practice helped themdeal with ill-defined problems encountered in their work and how reflectivewriting prepared students for lively tutorial discussion. By the end of thesemester student feedback ratings were high.

Bias in responding to questionnaires

Another key concern about the validity of student ratings raised in the intro-ductory review was that extraneous factors might result in bias. Our studycould be interpreted as showing that students’ conceptions of learning couldbias their rating of teaching. Those who conceive learning as an activeprocess are unlikely to give high ratings to teaching which is purely didactic.Students who have a passive conception of learning are likely to be biasedagainst teaching which requires active engagement. This latter conclusionis of greater concern because our study showed that a higher proportion ofstudents had a passive conception of learning and their potential bias coulddiscourage innovation in teaching.

Within the research into bias in student feedback questionnaires, thestudies closest to our findings are those which examined various person-ality and attitudinal characteristics of instructors. Feldman (1986) synthe-sised findings from the studies to show that, when students or colleaguesassessed the instructors, there were significant correlations between feed-back questionnaire ratings and several characteristics of instructors. Theattitude and personality characteristics which were investigated, though, werethose which have been commonly measured through standard personalityinventories. Beliefs about learning or teaching or related constructs didnot feature in Feldman’s review nor that of Marsh (1987). Other studiesfrom a rather different paradigm do, though, lend support to our findings.Entwistle and Tait (1990) asked psychology and engineering students tocomplete a combination of questionnaires examining study orientations andevaluating courses. They found that students who used a deep approach tolearning preferred a teaching/learning environment promoting understanding,which could presumably be equated to our non-traditional forms of teaching.Students in a non-academic orientation did not have a preference for teachingwhich promoted understanding, but those in engineering preferred forms ofteaching conducive to rote learning, which parallels our transmissive pole.


However, the authors cautioned against over generalising from their findingsas there were some differences between disciplines.

The findings of a qualitative study by Van Rossum and Taylor (1987) couldalso be interpreted as being consistent with our conclusions. They showedthat students’ conceptions of learning were related to their descriptions ofgood teaching. Those with reproductive conceptions of learning saw goodteaching as ensuring that facts were absorbed. As conceptions of learningdeveloped towards reconstruction of meaning, the good teacher became seenas a facilitator of learning.

Taking into account these two studies and our data, it seems reasonableto infer that the responses students make to student feedback questionnairesmay well be influenced by their beliefs about learning and their perceptionsof their lecturer’s beliefs about teaching. A transmissive form of teaching isless likely to be appreciated by students’ with an active belief about learning.More disturbingly, innovative non-traditional forms of teaching may not beappreciated and rated lowly by students with passive beliefs about learning.

Dealing with the bias

The logical response to beliefs about learning and other potential sourcesof bias is to treat the student feedback ratings as indicators only and toallow for potential bias where applicable. Instructors teaching very largeclasses in mammoth lecture theatres cannot realistically be expected to attainhigh ratings for interaction and rapport. Similarly, those teaching service orcompulsory courses usually find it more difficult to attain high ratings oninterest or motivational items than those teaching speciality courses.

Students’ beliefs about learning should be treated in the same way. Allow-ances should be made for lecturers who introduce an innovatory form ofteaching to cohorts who have experienced only didactic teaching. Innova-tions invariably need adaptation or fine-tuning based upon initial feedback.It, therefore, makes no sense to make judgements about a teacher based uponratings from an initial use of a particular innovation.

It is also sensible to make use of several forms of evaluation data. Triangu-lation across multiple sources is a well respected technique in research (e.g.Kember and Kelly 1993; McKernan 1991). It is also more likely to give amore reliable result in teaching evaluation than relying upon a single form ofevaluation alone.


Conclusion

This study has questioned some of the apparently widespread assumptionsabout student feedback questionnaires and their use in rating instructor’steaching. Use of a standard questionnaires implies that students share acommon view of what distinguishes good teaching from poor. It also inti-mates that the dimensions measured by the instrument are those which thestudents will accept as appropriate criteria for judging the quality of teachingand their learning. As our study has shown considerable variation in thecharacteristics and form of the descriptions of good teaching, it suggests thatboth of these assumptions are questionable.

Our findings add credence to those who have criticised many standardquestionnaires for being based upon implicit models of didactic teaching.Imposing a model of teaching upon a questionnaire can be avoided if theitems ask about learning outcomes rather than teaching. There might also begreater improvements to teaching and learning if the questionnaires were seenmore as a source of feedback and less as a method of judgement.

Some of the implications explored in the discussion section, however, areas yet tentative. There is a need for further research into the magnitude ofthe effect on feedback responses of beliefs about learning and perceptionsof the teachers’ beliefs about teaching. There is also a need for longitudinalstudies to see how any effects change with time when more innovative formsof teaching are introduced. Finally, this study also suggests that research intothe nature of quality in teaching would benefit from greater reliance upon thestudent perspective rather than the predominant researcher-driven perspectivecommon to much of the research on this topic.

Acknowledgements

This research was supported by grants from the Evaluation of the StudentExperience Project which was funded by the University Grants Committee ofHong Kong.

References

Biggs, J. (1996). ‘Western misperceptions of the Confucian-heritage learning culture’, inWatkins, D. and Biggs, J.B. (eds.),The Chinese Learner: Cultural, Psychological andContextual Influences. Melbourne and Hong Kong: Australian Council for EducationalResearch and the Comparative Education Research Centre, University of Hong Kong.

Biggs, J. (1999). ‘What the student does: Teaching for enhanced learning’,Higher EducationResearch and Development18(1), 57–75.


Centra, J. (1993).Reflective Faculty Evaluation. San Francisco: Jossey Bass.d’Appollonia, S. and Abrami, P.C. (1997). ‘Navigating student ratings of instruction’,

American Psychologist52, 1198–1208.Entwistle, N. and Tait, H. (1990). ‘Approaches to learning, evaluations of teaching and

preferences for contrasting academic environment’,Higher Education19, 169–194.Feldman, K.A. (1976). ‘The superior college teacher from the student’s view’,Research in

Higher Education5, 243–288.Feldman, K.A. (1986). ‘The perceived instructional effectiveness of college teachers as related

to their personality and attitudinal characteristics: A review and synthesis’,Research inHigher Education24, 139–213.

Feldman, K.A. (1996a). ‘Identifying exemplary teaching: Using data from course and teacherevaluations’,New Directions for Teaching and Learning65, 41–50.

Feldman, K.A. (1996b). ‘Identifying exemplary teachers and teaching: Evidence from studentratings’, in Perry, R.P. and Smart, J.C. (eds.),Effective Teaching in Higher Education:Research and Practice. New York: Agathon Press.

Gibbs, G. (1992).Improving the Quality of Student Learning. Bristol: Technical and Educa-tional Services.

Gow, L., Kember, D. and Sivan, A. (1992). ‘Lecturers’ views of their teaching practices:Implications for staff development needs’,Higher Education Research and Development11(2), 135–149.

Hinton, H. (1993). ‘Reliability and validity of student evaluations: Testing models versussurvey research models’,Political Science and Politics26, 562–569.

Ho, D.Y.F. (1986). ‘Chinese patterns of socialisation: a critical review’, in Bond, M.H. (ed.),The Psychology of the Chinese People. London: Oxford University Press.

Ho, D.Y.F. (1981). ‘Traditional patterns of socialisation in Chinese society’,Acta Psycholo-gica Taiwanica23(2), 81–95.

Husbands, C.T. (1996). ‘Variations in students evaluations of teachers’ lecturing and smallgroup teaching: A study at the London School of Economics and Political Science’,Studies in Higher Education21, 187–206.

Husbands, C.T. (1997). ‘Variations in students evaluations of teachers’ lecturing in differentcourses on which they lecture: A study at the London School of Economics and PoliticalScience’,Higher Education33, 51–70.

Kember, D. (1996). ‘The intention to both memorise and understand: Another approach tolearning?’,Higher Education31, 341–351.

Kember, D. (1997). ‘A reconcepualisation of the research into university academics’ concep-tions of teaching’,Learning and Instruction7(3), 255–275.

Kember, D. (2000).Action Learning and Action Research: Improving the Quality of Teachingand Learning. London: Kogan Page.

Kember, D. and Gow, L. (1991). ‘A challenge to the anecdotal stereotype of the Asian student’,Studies in Higher Education16(2), 117–128.

Kember, D. and Kelly, M. (1993).Improving Teaching Through Action Research. N.S.W.:HERDSA Green Guide No. 14.

Kember, D., Lam, B.H., Yan, L., Yum, J.C.K. and Liu, S.B. (1997).Case Studies of ImprovingTeaching and Learning from the Action Learning Project. Hong Kong: Action LearningProject.

Kolitch, E. and Dean, A.V. (1999). ‘Student ratings of instruction in the USA: Hidden assump-tions and missing conceptions about ‘good’ teaching’,Studies in Higher Education24(1),27–42.


Kwan, K.P. (1999). ‘How fair are student ratings in assessing the teaching performance ofuniversity teachers?’,Assessment and Evaluation in Higher Education24(2), 181–195.

Marsh, H.W. (1987). ‘Students’ evaluations of university teaching: Research findings,methodological issues, and directions for future research’,International Journal ofEducational Research11, 253–388.

Marton, F., Dall’Alba, G. and Beaty, E. (1993). ‘Conceptions of learning’,InternationalJournal of Educational Research19(3), 277–300.

McKeachie, W. (1997). ‘Student ratings: The validity of use’,American Psychologist52,1218–1225.

McKernan, J. (1991).Curriculum Action Research. London: Kogan Page.Perry, W.G. (1970).Forms of Intellectual and Ethical Development in the College Years. New

York: Holt, Rinehart and Winston.Perry, W.G. (1988). ‘Different worlds in the same classroom’, in Ramsden, P. (ed.),Improving

Learning: New Perspectives. London: Kogan Page.Richards, T.J. and Richards, L. (1991). ‘The NUD·IST qualitative data analysis system’,

Qualitative Sociology14(4), 307–324.Salili, F. (1996). ‘Accepting personal responsibility for learning’, in Watkins, D. and Biggs,

J.B. (eds.),The Chinese Learner: Cultural, Psychological and Contextual Influences.Melbourne and Hong Kong: Australian Council for Educational Research and theComparative Education Research Centre, University of Hong Kong.

Säljö, R. (1979).Learning in the Learner’s Perspective, I – Some Commonsense Conceptions.Reports from the Institute of Education, University of Gothenburg, No. 77.

Tagamori, H. and Bishop, L. (1995). ‘Student evaluation of teaching: Flaws in the instru-ments’, Thought and Action: the National Education Association Higher EducationJournal11, 63–78.

Van Rossum and Taylor (1987). ‘The relationship between conceptions of learning and goodteaching: A scheme of cognitive development’,Paper Presented at the AERA AnnualMeeting in Washington, April.

Watkins, D. and Biggs, J.B. (eds.) (1996).The Chinese Learner: Cultural, Psychological andContextual Influences. Melbourne and Hong Kong: Australian Council for EducationalResearch and the Comparative Education Research Centre, University of Hong Kong.

Weeks, P. and Scott, D. (eds.) (1992).Exploring Tertiary Teaching: Papers from the TRAC(Teaching, Reflection and Collaboration) Project. Armidale: University of New England.

Yang, K.S. (1986). ‘Chinese personality and its change’, in Bond, M.H. (ed.),The Psychologyof the Chinese People. Hong Kong: Oxford University Press.

Innovations

Documents

Transcript of Innovations