Capturing the Student Perspective: A New Instrument for Measuring Advising Satisfaction

Marilee L. Teasley, Erin M. Buchanan, Ph.D.

Why is Academic Advising Important?

•Kuh (2008): Quality of advising is a powerful predictor of campus satisfaction

•Metzner (1989): Lower attrition rates for high quality advising▫But some advising is better than none!

What Should Be Measured?

•Types of advising (Crookston, 1972):▫Prescriptive▫Developmental

•Which is better?

Measuring Advising: Reliability & Validity•Reliability: Consistency of scores•Validity: How well do you measure what

you are claiming to measure?

Measuring Advising: Factor Analysis•Exploratory Factor Analysis (EFA)•Confirmatory Factor Analysis (CFA)

Exploratory Factor Analysis

Wonderful

TERRIBLEHappiness

Joyful

Exciting

FRUSTRATING

TRAGEDYANGRY

Confirmatory Factor Analysis

Wonderful

TERRIBLEHappiness

Joyful

Exciting

FRUSTRATING

TRAGEDY

ANGRY

Existing Measures

•Very few standardized assessments exist•Validity and Reliability?•Qualitative versus Quantitative methods

Original Research Question

•How do our students feel about their past and present academic advising experiences?

Additional Research Question

•Can we extend the existing advising assessment research by creating a new scale with statistical reliability and validity?

Our Questionnaire: Version 1• Investigated:

▫Advisor Traits (patience, trustworthiness)▫Prescriptive Functions (course selection,

graduation requirements)▫Developmental Functions (campus/community

involvement, overall student development)

•Seven-point Likert scale▫1 = strongly disagree, 4 = neutral, 7 =

strongly agree

  Experiment 1 Experiment 2 Experiment 3

Age 20.8 (4.8) 19.76 (4.44) 19.31 (2.45)

Gender      

Female 62.5% 75.7% 78.1%

Male 37.5% 24.3% 21.9%

Classification      

Freshman 47.4% 69.6% 63.9%

Sophomore 28.3% 16.6% 22.4%

Junior 13.2% 9.4% 8.8%

Senior 11.2% 4.4% 4.8%

Transfer Students      

Transfer 23.8% 19.3% 19.7%

Non-Transfer 76.2% 80.7% 80.3%

Ethnicity      

Caucasian 82.9% 80.1% 83.0%

Other 17.1% 19.9% 17.0%

Major      

Decided 85.5% 80.7% 87.1%

Undecided 14.5% 19.3% 12.9%

Our Study: Experiment 1

•155 total students, 152 complete questionnaires

•Demographics + 24 advising questions

How did we do? (Version 1)

•Exploratory Factor Analysis (EFA):▫One factor model, poor overall fit

•Compound questions & clarity issues▫“My advisor encourages me to speak freely

and listens to what I have to say.”

Our Study: Take 2!

•181 total students, 177 complete questionnaires, 157 remaining after outlier analysis

•Original demographics questions + 30 revised advising questions

How Did We Do? (Version 2)

•Exploratory Factor Analysis (EFA) showed that one or two factors would be appropriate

•Best fit: two factors, 24 questions

The Two Factor Model

•Advising Functions▫“Advising appointments are worth my time”▫“I find academic advising appointments to

be a positive experience.”•Outreach Functions

▫“I learn how I can contribute to the surrounding community during my advising appointments.”

▫“My advisor lets me know about the importance of our public affairs mission.”

One More Time… (Experiment 3)•184 total students, 167 remaining after

outlier analysis•59 returning students from Experiment 2

for test-retest reliability purposes ▫24 final advising questions, no

demographics•New participants:

▫Demographics + 24 final advising questions

How Did We Do? (Version 3)

•Confirmatory Factor Analysis▫Good overall fit

•High test-retest reliability

What About Our Original Question?•Both factors had high averages (above

“neutral”)•Advising factor > Outreach factor

What’s Next?

•NACADA Journal, 33(2)▫Late Fall/Early Winter 2013

•The goal: impact advising assessment!

Thank You!

•For questions about our project or usage of our scale, please contact either one of us:

Marilee Teasley (teasley888@live.missouristate.edu)

or Dr. Erin Buchanan

(erinbuchanan@missouristate.edu)

Questions?