Medical students self-appraisal of first-year learningoutcomes: use of the course valuing inventory

DEJANO T. SOBRALUniversity of Brasilia, Faculty of Medicine, Brasilia, Brazil

SUMMARY The aims of this paper were: to appraise how medical

students perceive the meaning and value of their first-year experi-

ences in medical studies, as measured by the Course Valuing

Inventory (CVI); and to identify the relationships between the CVI

responses and the learners attributes and expectations. The study

involved 282 second-year students of a six-year medical pro-

gramme over a four-year timeframe. Factor analysis identified five

dimensions of CVI responses: worthiness of learning experience,

emotional awareness, personal development, cognitive enhance-

ment and task drive. Higher CVI scores related positively and

significantly to female gender, stronger self-confidence as a learner,

greater motivation to learn, meaning orientation and reflection in

learning. The CVI score was the strongest independent predictor of

intention to continue the studies as a motivational consequence of

first-year learning experience. In addition, a separate test showed

significant relationships between the dimensions of CVI responses

and the DREEM score (Dundee Ready Education Environment

Measure). In conclusion, CVI responses that are more positive are

likely to be associated with a quest for meaning, reflection in

learning and autonomous motivation, given the relevance of the

educational environment for the proximal interests of the students.

Introduction

How do students judge their early experience in medical

studies? As they go through the initial terms of the medical

programme students are likely to perceive both the formal

and informal components of their learning opportunities.

These components include the explicit and implicit repre-

sentations of the curriculum manifested as the educational

environment. The importance of the educational environment

has been highlighted lately in recognition of the challenging

desiderata of professional education (Genn, 2001a). Genn

(2001b) emphasized the relevance of the concept of ambience

or climate as an expression of the curriculum and educational

environment, as well as a measure of quality. Roff et al. (1997)

developed a useful measure of the quality of the educational

ambience, the Dundee Ready Education Environment

Measure (DREEM).

It is apparent from Genns seminal papers that students

perceptions of the educational environment influence their

responses to teaching and learning processes. Moreover, the

effects of the learning experience on the learner, going

beyond cognitive achievement, have a bearing on students

self-evaluation and educational adaptation.

Nehari & Bender (1978) proposed a conceptual model for

the meaningfulness and value of a course of study and

suggested that such educational outcomes are related to

perceived learning in the cognitive-subject matter, affective-

personal, and behavioural domains. These authors developed

an instrumentthe Course Valuing Inventory (CVI)to

measure the value and meaningfulness of a learning expe-

rience from the learner viewpoint and from a humanist

outlook. In their paper, Nehari & Bender indicated that

the learners expertise or level of educational experience was

influential in his/her perceptions of course value. They pro-

posed the use of the tool as a measure of the educational

efforts and of the courses effects on the students in terms of

change and growth.

There are several previous reports bearing on the use of

the CVI in the medical education field that point to the

usefulness of this tool. A first study showed that students

CVI responses appear to be a good measure of the appeal or

worthiness of an educational experience (Sobral, 1989). In

particular, the CVI responses were shown to relate mean-

ingfully to the learners affective reactions as a measure of the

emotional climate of course experiences. Perceived worthi-

ness of a course experience strongly correlated with students

enjoyment or euphoria (Sobral, 1992). Further, it has been

suggested that the students CVI responses differentiate

between educational approaches to learning and teaching

and that they could detect subtle aspects of the quality of the

learning environment, as viewed by the learners (Sobral,

1995). From a different point of view, a recent study showed

that students CVI scores were significantly associated with a

reflective stance and meaning orientation in learning (Sobral,

2000).

To what extent do students CVI responses in judging the

early experience reflect the attitudes and preferences of the

learners, in addition to the quality of curriculum? A premise

of this study was the idea that the CVI could be sensitive in

tapping the educational effects on the learners in relation to

their own attributes and expectations. In this sense, CVI

responses to an earlier experience could relate to perceptions

of the quality of the educational environment later on.

The aims of this study were to appraise how medical

students perceive the meaning and value of their first-year

experiences in medical studies, as detected by the Course

Valuing Inventory, and to analyse the relationships between

the CVI perceptions and other measures of the learners

responses to the learning experience. This report addresses

three specific objectives:

(1) to describe the dimensions of the students CVI

responses after the first year of medical studies;

Correspondence: D.T. Sobral, University of Brasilia, Faculty of Medicine, CP

04569, Brasilia 70919-970, Brazil. Tel: 55(61) 5772268; fax: 55(61) 5773581;

email: dtsobral@unb.br or dtsobral@netscape.com

Medical Teacher, Vol. 26, No. 3, 2004, pp. 234238

234 ISSN 0142159X print/ISSN 1466187X online/03/030234-5 2004 Taylor & Francis LtdDOI: 10.1080/10.1080/0142159042000192028

(2) to determine the relationships between the CVI responses

and the learners performance and attributes such as

academic achievement, self-confidence as a learner,

motivation to learn, meaning orientation to learning

and reflection in learning;

(3) to assess the sensitivity of the CVI responses in predicting

the students overall motivation or intention to continue

medical studies.

In addition, a test of the relevance of CVI as a measure of

educational ambience involved the determination of the

relationship between this tool and the DREEM inventory

in a sample of the study subjects (a class of students).

Method

This section addresses the setting/context, subjects, measures,

procedures and data analysis.

Setting/context

The study evolved in the context of the University of Brasilia

medical programme. This is a six-year programme, including

four semesters of basic science courses, five semesters of

clinical courses/clerkships and three semesters of internship

training. Since 1988, the curricular track has required

the first-year student to take courses in anatomy and

histology, cellular biology, biochemistry and biophysics,

physiology and community health. In addition, the student

has opportunities for self-selected studies and activities in

optional courses, research apprenticeship and cross-year

tutoring of peers in courses already taken. Class size is

variable: it ranged from 32 to 44 students (average 36) during

the study period.

Subjects

The target population consisted of students who completed

the first year of medical studies. A consecutive sampling

design was used (Hulley & Cummings, 1988). This involved

taking every student who registered for the third term within

a four-year timeframe (19972000). Males made up 55.3% of

the 282 students who participated in the study. The average

age of the participants in the contact year was 20.5 (range 18

to 31 years of age).

Measures and procedures

The study data derived from three inventoriesthe Course

Valuing Inventory (CVI), the Reflection-in-Learning Scale

(RLS) and the short version of the Approaches to Study

Inventory (s-ASI)a learner profile, and a measure of

achievement.

As described by Nehari & Bender (1978) the CVI is an

instrument consisting of 36 items in four equal-sized

scales: course valuing, content learning, personal learning

and behavioural learning. The paper by Nehari & Bender

(1978) reproduces the inventory. In the current study, the

wording of some items was slightly modified to meet the

perspective of a broader curricular experience and not just

of a single course.

The learner responds to each item on a four-point scale

that gauges the level of positive or negative response to

the item statement. The sum of item scores with polarity

adjustment constitutes the students score on the inventory

(score range: 36144). The CVI has shown high internal

consistency in several studies (range of Cronbachs alpha

0.930.94) and acceptable temporal stability (testretest

r 0.63 within a six-month interval). For the present studysample Cronbachs alpha was 0.94, the adjusted item-total

correlation ranged from 0.36 to 0.64, and the coefficients for

inter-item correlation were all positive.

The RLS is a self-report questionnaire emphasizing

metacognitive behaviours for the appraisal of reflection

in learning, as reported before (Sobral, 2000). The RLS

version used has 14 items and the total score ranges

from 14 to 98. Cronbachs alpha was 0.86 in the study

sample.

The s-ASI is a short version of the instrument devised

by Entwistle (1988) that was used to measure students

perceptions of their learning orientation as they started a

new course of studies. The abbreviated 32-item instrument

focuses on two structural factors: meaning orientation and

reproductive orientation (Richardson, 1990). The score

range for the meaning orientation scale is 0 to 64 and

Cronbachs alpha for the scale was 0.71 in the study sample.

The learner profile supplied information on the following

attributes: age, gender, self-confidence as a learner, motiva-

tion to learn and intention to continue medical studies.

Self-report visual analogue scales (100mm lines anchored

at the extreme ends) provided measures for the perceived

states regarding the three attitudinal attributes.

A composite grade-point average (GPA) derived from

the assessment of the first-year courses provided the measure

of academic achievement.

The DREEM is a 50-item inventory, which is useful

in the assessment of the quality of the educational environ-

ment, particularly the learning and teaching climate (Roff &

McAleer, 2001). It consists of five subscales or domains

covering students perceptions of atmosphere, learning,

teachers, and social and academic issues. The score range

for the inventory is 0 to 200. In the current study, this

instrument was administered to a class of 41 subjects in

the last year of the study. Cronbachs alpha was 0.90 in

a combined sample of subjects (n 91).Students in each consecutive class completed the inven-

tories and the learner profile in the classroom in the first week

of the third term of the medical programme. Students in the

last class also completed the DREEM instrument five weeks

after the start of term.

Data analysis

Three main statistical procedures were used in the study:

(a) correlation coefficients for measuring the association

between pairs of variables; (b) t-tests or analyses of variance

for assessing the differences between group means; and (c)

multiple regression analyses to verify the association between

a single explanatory factor and a dependent variable, when

controlling for other factors. The level of alpha (significance)

was set at p 0.05 (two-tailed). The works of Glantz &Slinker (1990) and Norman & Streiner (1994) guided the

choice of procedures.

Medical students perceptions of learning outcomes

235

Results

Mean CVI scores among students grouped by entry

class were not significantly different (ANOVA, F 1.73,df 7, p 0.10). In addition, the scores did not showa temporal trend over the four-year timeframe of the study

(r 0.00, p 0.94). Subsequent analysis involved all thesubjects.

Table 1 provides the mean and the standard deviation of

each study measure for all students and for the students

grouped by gender. Females showed significantly higher

scores for CVI as well as higher achievement index (first-year

GPA).

Overall, the mean CVI score was 118.80 (SD 13.4).About 57% of the students showed scores above 117

(>75% in the score range) and 3.9% showed scores not

greater than 90 (50% in the score range). Higher scores

reflected a more positive learning experience overall.

In fact, the items showing highest item-total score correla-

tions were:

Item 20: The medical programme has been so far adefinitely helpful and constructive learning experience

(r 0.65). Item 16: The first-year courses were not stimulating

(r 0.65). Item 15: I feel more perceptive of others now, and more

sensitive to their needs (r 0.63).A factor analysis (principal components, equamax rotation)

extracted five dimensions of responses accounting for 49.8%

of the variance. Such dimensions identified groups of similar

items labelled as follows (in decreasing order of explained

variance): worthiness of learning experience; emotional

awareness; personal development; cognitive enhancement;

and task drive.

Analysis of variance (repeated measures) revealed very

significant differences among the mean scores of the factorial

dimensions. The highest positive responses were associated

with worthiness of learning experience and the lowest positive

responses with emotional awareness. Pair-wise comparisons

showed no significant differences between the scores for

personal development, cognitive enhancement and task

drive. Table 2 shows the mean score and representative

items for each factorial dimension.

Welchs t-tests revealed differential gender effects for the

CVI factorial dimensions. Females showed significantly

higher scores for emotional awareness (t 2.84, p 0.005)and task drive (t 2.25, p 0.025), but not for cognitiveenhancement (t 0.20, p 0.84).

A correlation analysis revealed positive relationships

of the CVI total score with five attributes: academic

achievement, self-confidence as a learner, motivation to

learn, meaning orientation to learning, and reflection in

learning, in the order of increasing strength of correlation.

This order of relationship closeness was mostly the same

for each factorial dimension. The extreme correlation

figures were r 0.00 (between emotional awarenessand academic achievement) and r 0.53 (betweentask drive and reflection in learning). Table 3 shows the

Table 2. CVI factorial dimensions, mean scores and representative items based on responses of four-year classes of medical

students after the first year of medical studies (n 282).Factorial dimensions Mean score Representative items

1. Worthiness of experience 3.55 The medical programme has been a very valuable experience for me

(7 items) (0.44) So far I would not recommend the medical programme to a friend

2. Emotional awareness 3.04 The (first-year) experience helped me realize the importance of my own feelings

(7 items) (0.52) I feel more perceptive of others now, and more sensitive to their needs

3. Personal development 3.33 The courses had no effect on my personal development

(8 items) (0.45) The courses had no effect on the way I communicate

4. Cognitive enhancement 3.30 The courses helped me achieve a deeper understanding of medicine

(7 items) (0.38) The courses helped me acquire important basic knowledge

5. Task drive 3.30 I have taken more responsibility for my own learning than I usually did

(7 items) (0.47) Somehow I have worked harder (in the medical programme) than I usually did

Table 1. Comparisons of means (SD) for each study measure between students grouped by gender (n 282).

Measures

Total (n 282)Mean (SD)

Males (n 157)Mean (SD)

Females (n 125)Mean (SD)

T-test

p

CVI 118.80 (13.37) 117.30 (14.30) 120.69 (11.89) 0.034

Meaning orientation 48.17 (5.79) 47.94 (5.94) 48.47 (5.61) 0.446

Reflection in learning 70.94 (10.83) 69.90 (10.79) 72.24 (10.79) 0.071

Self-confidence 79.99 (16.58) 81.37 (16.21) 78.26 (16.93) 0.117

Motivation to learn 89.52 (10.71) 88.57 (10.98) 90.70 (10.27) 0.097

First-year GPA 3.73 (0.47) 3.64 (0.47) 3.83 (0.46) 0.001

Intention to continue 87.72 (15.49) 87.00 (15.95) 88.67 (14.90) 0.435

D. T. Sobral

236

relationships between every CVI dimension and each

attribute measure.

How much did the course-valuing perceptions contribute

to persistence drive in the pursuit of studies? To answer

this question, the variable intention to continue medical

studies was taken as a motivational consequence of all the

individual measures, including the CVI measure. Multiple

regression analysis revealed that four variables explained

38% (adjusted R2 0.38) of the variance associated withsuch motivational consequence: CVI, motivation to learn,

self-confidence as a learner, and first-year GPA. CVI score

was the strongest independent predictor, while gender,

meaning orientation and reflection in learning had indepen-

dent effects of no significance. Further analysis (replacing

CVI by the five factorial dimensions) identified worthiness of

learning experience as the strongest independent predictor

of intention to continue the studies. Table 4 shows the

regression coefficients.

Do the dimensions of first-year course perceptions

relate to the quality of the educational environment?

A decisive test for this question used the DREEM inven-

tory in a single class of 41 students. The mean for the

DREEM total score was 123.1 (SD 18.8). Table 5 showsthe correlation of the DREEM score with each of the

CVI factorial-dimension scores. There was a clear positive

and significant relationship between the total scores of

the inventories. The CVI dimension of task drive showed

the strongest correlation with the DREEM score. Overall, the

DREEM subscales for perceptions of learning and social

self-perceptions held the strongest relationships with the CVI

dimensions.

Discussion

Almost all students felt at ease to report their overall

perceptions of learning outcomes for their first-year educa-

tional experience. This suggests a halo effect but many

students pointed out differences among the courses taken as

regards their perceived effects. The small differences among

the CVI class scores and the absence of any temporal trend in

the class scores within the study timeframe suggest that most

of the score variation reflected differences in individual

perceptions of the curriculum track and not ongoing changes

in teaching practices and course features.

As revealed by the Course Valuing Inventory, perceptions

of the educational effects on the learners resulting from the

first-year experience in the medical programme comprise a

number of interrelated but distinct outcomes relating to

personal knowledge, affect and behaviour. Such perceptions

displayed a distinct pattern of responses in the population

sampledas indicated by factor analysis. The five-dimension

pattern related differentially to gender. Female students

showed a more positive and empathic view of their early

experiences on account, perhaps, of their heightened

sensitivity to the learning ambience. The gender effect

could also relate to eventual differences in learning style

and early career preferences.

Table 3. Coefficients of correlation between the CVI dimension scores and measures of the students

learning features after the first year of medical studies (n 282).CVI: factorial

dimensions

First-year

GPA Self-confidence

Motivation

to learn

Meaning

orientation

Reflection

in learning

Worthiness of experience 0.13 0.20 0.39 0.33 0.43

Emotional awareness 0.00 0.24 0.29 0.38 0.51

Personal development 0.08 0.25 0.24 0.37 0.49

Cognitive enhancement 0.08 0.18 0.14 0.35 0.47

Task drive 0.11 0.28 0.28 0.45 0.53

Total scale 0.10 0.28 0.33 0.46 0.60

Notes: Critical values: r 0.11, p

The findings of the correlation analysis indicate

that the CVI responses reflect better a personal frame

of mind towards learning than the effect of academic

achievement. Two features of such frame of mind are

especially relevant: the levels of reflection in learning and

motivation to learn. Altogether, it seems that students with

stronger self-regulation of learning and autonomous motiva-

tion have more positive perceptions of the meaningfulness

and value of their early experiences in the medical

programme.

Consistently, CVI perceptions of the first-year experience

relate to intention to continue with studies, independently of

academic achievement and of other learner features. It is

fitting that, among the CVI responses, worthiness of learning

experience appeared as the strongest predictor of the

persistence drive in pursuit of studies at an early stage of

medical training. Unpublished data showing an association

between CVI perceptions of the first-year experience and the

number of peer-tutoring courses taken by the students over a

four-semester follow-up attest the predicting power of the CVI

dimensions.

In addition, the significant correlation between the

scores of CVI and DREEM is of great interest. The

relationship suggests that the CVI responses do reflect

broader influences of the educational environment besides

the learners frame of mind towards learning. Unreported

data show that the relationship was still true after an

interval of 18 months between the first measure (CVI) and

the second measure (DREEM), in the same group of

students.

The findings point to a significant interaction between the

curricular features and students individual features in the

perceptions of early medical studies. This broad educational

impact probably reflects a variety of contextual features in the

students course experience and is related to, but not

grounded by, academic achievement.

The results of this study and of previous reports indi-

cate that the CVI is a useful tool for appraisal of the

educational effects of students early experience in medical

studies in relation to their attributes and expectations. It

could assess differential perceptions of learning outcomes at a

curricular level. However, it is very likely that the CVI reflects

aspects of the educational experience different from other

tools, such as the DREEM. The CVI gives a more personal

appraisal of the educational experience that may reflect the

learners self-evaluation or self-regulation of learning and

self-motivation.

In conclusion, the CVI responses comprise dimensions of

variation of educational outcomes that seem sensitive to

personal-learning attributes in the context of curricular and

environmental influences. For the early medical education

experience, CVI responses that are more positive are likely

to be associated with a quest for meaning, reflection in

learning and autonomous motivation, given the relevance of

the educational environment for the proximal interests of the

students.

Practice points

Acknowledgements

The author is grateful to all students for their genuine

cooperation and participation.

Notes on contributor

DEJANO T. SOBRAL has a medical degree and is a teacher at the University

of Brasilia, Faculty of Medicine. He has broad interests in medical

education and in particular in student learning.

References

ENTWISTLE, N. (1988) Styles of Learning and Teaching (London, David

Fulton).

GENN, J.M. (2001a) Curriculum, environment, climate, quality and

change in medical education: a unifying perspective, Medical Teacher,

23(4), pp. 337344.

GENN, J.M. (2001b) Curriculum, environment, climate, quality and

change in medical education: a unifying perspective, Medical Teacher,

23(5), pp. 445454.

GLANTZ, A.S. & SLINKER, B.K. (1990) Primer of Applied Regression Analysis

and Variance (New York, McGraw-Hill).

HULLEY, S.B. & CUMMINGS, S.R. (1988) Designing Clinical Research

(Baltimore, MD, Williams & Wilkins).

NEHARI, M. & BENDER, H. (1978) Meaningfulness of a learning

experience: a measure for educational outcomes in higher education,

Higher Education, 7, pp. 111.

NORMAN, G.R. & STREINER, D.L. (1994) Biostatistics (St Louis, MO,

Mosby).

RICHARDSON, J.T.E. (1990) Reliability and replicability of the Approaches

to Studying Inventory, Studies in Higher Education, 15, pp. 155168.

ROFF, S., MCALEER, S., HARDEN, R.M. et al. (1997) Development and

validation of the Dundee Ready Education Environment Measure

(DREEM), Medical Teacher, 19, pp. 295299.

ROFF, S. & MCALEER, S. (2001) What is educational climate?, Medical

Teacher, 21, pp. 333334.

SOBRAL, D.T. (1989) Learning the educator role: a course for medical

students, Medical Education, 23, pp. 7076.

SOBRAL, D.T. (1992) Self-report visual scale of course appeal, Higher

Education, 23, pp. 321329.

SOBRAL, D.T. (1995) The problem-based learning approach as an

enhancement factor of personal meaningfulness of learning, Higher

Education, 29, pp. 93101.

SOBRAL, D.T. (2000) Medical students reflection in learning in relation

to approaches to study and academic achievement, Medical Teacher, 23,

pp. 508513.

The Course Valuing Inventory (CVI) is a useful tool tomeasure students perceptions of learning outcomes

from a humanist outlook. The CVI comprises distinct although interrelated

dimensions of perceptions that show meaningful

relationships with the learners attributes such as

motivation to learn and reflection in learning. The CVI scores relate to drive in the pursuit of studies

as well as to a measure of the quality of education

environment (the DREEM inventory).

D. T. Sobral

238