Marsh (1982). Validity of Students' Evaluations of College Teaching - A Multi-trait-multi-method...

Journal of Educational Psychology 1982, Vol. 74, No. 2, 284-279. Copyright 1982 by the American Psychological Association, Inc. 0022-0663/82/7402-0264$00.75 Validity f Students' Evaluations of College Teaching: A Multitrait- Multimethod Analys is Herbert W. Marsh University Sydney, Australia College instructors in 329 classes evaluated their own teaching effectiveness with the same 35-item rating form that w a s used by their students. There w a s student-instructor agreement in courses taught b y teaching assistants ( r = .46), undergraduate courses taught by faculty (r = .41), an d even graduate level courses ( r = .39). Separate factor analyses of the student and instructor ratings demonstrated that the same nine evaluation factors underlay both sets of ratings. A multitrait-multimethod analysis provided support fo r both con- vergent and divergent validity of these rating factors. o t only were correlations between student a n d instructor ratings on the same factors statistically significant fo r each o f nine factors (median r = .45), bu t correlations between their ratings o n different factors were low (median r = .02). These findings demonstrate student-instructor agreement on evaluations o f teaching effectiveness, support th e validity o f student ratings for both graduate an d undergraduate courses, and emphasize th e importance of using multifactor rating scales that ar e derived through the application o f factor analysis. Students' evaluations of teaching effectiveness are often criticized as lacking validity. However, the ratings are difficult to validate because there is no universal criterion of goo d teaching. Researchers, usin g a construct validation approach, have sought to demonstrate that student ratings are re- lated to a variety of other measures assumed to be indicative of effective teaching. If two indicators of the same quality—student ratings and some other me asure of teaching effectiveness—show agreement, then there is support for th e val idity o f both. The most commonly employed criterion has been performance on a final examination in a multi section course. When different sections of the same course were taught by different instructors, the sections that did best This study w as also the basis of a paper presented a t the 1 97 9 Annual Meeting of the America n Educational Research Association. (Irateful acknowledgments a r e extended to John Schutz, Joseph Kertes, a n d Robert Linnell, adminis- trators at the University o f Southern California wh o have supported t h e student evaluation program. Thanks are also extended to J. U . Overall and X oe Cosgrove, who made many helpful suggestions o n earlier drafts o f this paper. Requests f o r reprints should b e sent to Herbert W. Marsh, Department of Education, University of Sydney, Sydney, New South Wales 2006, Australia. on the standardized examination given to all sections were also the ones who evaluated their instructors more favorably (Centra, 1977; Frey, 1973, 1978; Frey, Leonard, & Beatty, 1975; Marsh, Fleiner, & Thomas, 19 75 ; Marsh & Overa ll, 1980). Using a similar proc edur e Marsh and Overall (198 0) demonstrated that sections that evaluated teaching more favor ably were als o the ones that felt better able to apply the course ma- terials and were more inclined to pursue the subject fu rther. Other researchers have reported goo d agreement b etween st udent ratings and both the retrospective ratings of recent alumni (Centra, 1973; Marsh, 1977) and the follow-up evaluations of the same students several years after graduation (Overall & Marsh, 1980). Validity research such as that described above, although supporting the use of student ratings, has generally been li mited to a specialized setting (e.g., large multisection courses) or has employed criteria (e.g., student reports) that are unlikely to convince sce ptics . Thus, facu lty will continue to question the usefulness of student ratings validity criteria that are both con- vincing and applicable across a wide range o f courses are utilized . One criterion that meets both these requirements is instructors' evaluations o f their o w n teaching effective-

Upload
dan-simonet
Category

Documents
view
215
download
0

Embed Size (px):