Student Opinion Survey

It’s required…how do we make meaning out of it? How can results be the

start of action?

This is radical:• We need to decide in advance what

we want from the SOS, not just be unhappy (or happy) with what we get

A possible first step• Find a partner, a functional office like

Student Life or Academic Services

How in the heck do we do that?

• Identify areas of concernLook at other assessments: what makes us anxious?Look at data from past SOS administrations: what do we not want to share?What in the environment/ atmosphere/ milieu keeps us up at night?

Be strategic with local questions

• Look at all the opportunities for local questions

• In a given time frame, keep the local questions largely the same

• Compare from one survey to the next

Four Phases• Prep by IR

– Decide local questions

– Decide areas of interest

• Decide on the meaning of the data– To what will you

respond?– How will you report?

• Dissemination of data– Who will see the data?– When?– In what format?

• Take action– Who gets to say

“Houston, we have a problem?”

– Who gets to say “This is what we’ll do to respond?”

Prep Respondents• Yes, have a campaign

–In 2006, you said X, Y, Z–We did A, B, C to address–Respondents want to know that what they said made a difference

–Respondents are volunteers and the best volunteers are prepared for the task

Prep Respondents• In the very few areas you are

specifically interested in, explain to respondents what the content of an item means

• Give them a rubric: what does “Very Satisfied” mean? What does “Very Dissatisfied” mean?

Now Here’s a Rubric!http://www.teachervision.fen.com/teaching-methods-and-management/rubrics/4522.html#what_is_a_rubric

Delicious Good Needs Improvement Poor Number of

Chips Chocolate chip in

every bite Chips in about 75%

of bites Chocolate in 50% of

bites Too few or too

many chips

Texture Chewy Chewy in middle, crisp on edges

Texture either crispy/crunchy or 50% uncooked

Texture resembles a dog biscuit

Color Golden brown Either light from

overcooking or light from being 25% raw

Either dark brown from overcooking or

light from undercooking

Burned

Taste Home-baked taste Quality store-bought taste Tasteless

Store-bought flavor, preservative

aftertaste – stale, hard, chalky

Richness Rich, creamy, high-fat flavor Medium fat contents Low-fat contents Nonfat contents

Prep Ourselves• To what will we pay attention? • Decide in advance• Keep it to a small number (yes,

that means everything is not #1)• Create clusters of questions

around these areas, using local questions

Prep Ourselves• It’s not just an IR job• Needs high level of involvement• To whom will you report results?

These are the people/offices you need to work with before the administration

• Urge these folks to commit to action

Build an Appetite for the Data

• Who are the natural customers of SOS data?

• What is the best way to provide the data to them?

• How do we get feedback about what they did in response to the data?

Prep Ourselves• Conduct focus groups with

students a year before the survey administration

• Focus on the areas with which you are concerned

• Or the areas in which you were unhappy with the results

Make meaning via triangulation of the Data

• Results from the specific administration

• Comparisons of means to past administrations - trends

• Sector rankings• Always display data in context

Understand the limitations of the instrument

– The SOS is a once-over lightly survey of many items

– Most content areas have one question– Broad administration in a semi-

controlled environment– Points us to more research or to action

where general perceptions are validated– Is neither everything nor nothing

Everything is Relative (thanks Mr. Einstein)

• If we are doing ‘badly’ in areas that are of top priority to us, then we need action

• If the results are mixed, then we need to find out more information

• If the area is not in our top 10, then ‘who cares’?

• A plan to improve would involve changes in areas (see #1), and also a metaplan

Conversation based on results of SUNY New Paltz 2006 data

• A few 2006 SOS positives:– We are #1 in “Contribution to

understanding/appreciating ethnic/cultural differences”

– We do well in helping students develop an open mind to the opinions of others

– What do we do to respond to these findings?

• A few New Paltz not-so-positives:– We rank #12 out of 12 in the

comprehensive sector in ‘helping students get a part-time job’

– Plus, our average mean is the lowest in 4 SOS administrations

– Under what conditions would we do nothing about this finding?

• SUNY NP ranked #12 out of 12 in satisfaction with advising

• There is no clear trend in comparing means over time

• What should we do?

• SUNY NP is #12 out of 12 in the “Purposes for which the student activity fee is used”

• What should we do?• What was going on at the time?

Meaning & Action• You can’t take action without meaning• You can’t find meaning without a plan• You can just administer the survey,

and then be faced with a mess of responses that don’t yield much meaning

• The result: no action