Addressing Assessment Fatigue: Generating Student Interest, Participation, and Ownership in...
-
Upload
cordelia-dixon -
Category
Documents
-
view
214 -
download
1
Transcript of Addressing Assessment Fatigue: Generating Student Interest, Participation, and Ownership in...
Addressing Assessment Fatigue:Generating Student Interest,
Participation, and Ownership in Assessment
Marissa CopeStudentVoice
Nathan LindsayUniversity of North Carolina Wilmington
Overview of Session
•General information on applicable research•Techniques for generating student participation•Coordination of assessment activity and student
ownership•Generating interest: Sharing results with students •Examples from UNCW and other campuses•Discussion questions
Audience participation will be requested throughout the session.
Assessment fatigue among students has been a challenge at our university.
17% 17% 17%17%17%17%
A. Strongly AgreeB. AgreeC. Neither Agree nor DisagreeD. DisagreeE. Strongly DisagreeF. Don’t Know/Not Applicable
General Information• Survey response rates have been falling • Difficult to contact people• Refusals to participate increasing
• National survey response rates fell from about 60% in the 1960s to just above 20% in the 1980s
• Nonresponse may not be random
• Two strategies for correcting low response rates:1.Weight the data for nonresponse2.Implement strategies to increase response rates
Research on Survey Fatigue
• Multiple surveys are seen as a burden; would expect response rates to decrease as requests increase
• Time is a major issue
• Example - 1998 study at Air Force Academy:▫ Anecdotal evidence – survey about surveys▫ 97 percent: somewhat over surveyed▫ Should be surveyed only 3-4 times a year▫ Over surveyed: “combination of frequent surveys
perceived as irrelevant”
Implications from Research
• Potential of multiple surveys can reduce response rates
• Non-respondents cite time concerns as reason
• Effects of survey fatigue may be moderated by salience of survey
content
• Number of previous surveys may have an impact on current survey
response
• Survey fatigue may have biggest impact on survey administered
back-to-back
• Feeling of “I have done enough” (reciprocity)
pg. 66
Theories of Survey Response
1. Reasoned action approach• Calculation of costs and benefits• Social exchange: rewards, cost, and trust• Increase rewards• Reduce costs• Establish trust
2. Psychological approach▫Informal decision rules▫Norm of reciprocity▫Norm of social responsibility
Theories of Survey Response
•What this means in action:▫More likely to comply when request appears to be from
a legitimate authority▫Requests for help should be clear and straightforward in
survey invitation▫Emphasize that recipients are part of a select group
Techniques for Generating Student Participation
pro.corbis.com
Technique 1: Multiple Contacts
Pre-notifications:Provides information about the project including data
collection times, incentives, purpose, contact information
Invitation with survey information:Repeat of pre-notification information
Follow-up reminders:Ideally only to non-respondentsRange of 2-5 day intervals
We usually use pre-notifications for our surveys.
17% 17% 17%17%17%17%
A. Strongly AgreeB. AgreeC. Neither Agree nor DisagreeD. DisagreeE. Strongly DisagreeF. Don’t Know/Not Applicable
Technique 2: Length
• The longer the survey the higher the perceived cost
• Moderate correlation between length and non-response
• Studies recommend no more than the following:• 22 questions• 13 minutes
• Also consider type of questions: text fields vs. answer choices
Our surveys are generally shorter than 22 questions.
17% 17% 17%17%17%17%
A. Strongly AgreeB. AgreeC. Neither Agree nor DisagreeD. DisagreeE. Strongly DisagreeF. Don’t Know/Not Applicable
Technique 3: Incentives
• Research on the effect of incentives on college students is lacking
• Prepaid incentives consistently raise levels of response
• Effect of postpaid incentives is minimal• Payment contingent upon completion =
compensation• 2 effects:• Getting paid removes reciprocity aspect• Expectations for future participation
Incentives for assessments improve response rates at our institution.
17% 17% 17%17%17%17%
A. Strongly AgreeB. AgreeC. Neither Agree nor DisagreeD. DisagreeE. Strongly DisagreeF. Don’t Know/Not Applicable
Technique 4: Salience
Salience: How important or relevant a survey topic is to the
survey recipient
• 12-14% increase in response rates for salient surveys
• Difficult to control
• Highlight salience in survey invitations
Technique 5: Statement of Confidentiality
Anonymous vs. confidentialAnonymous: responses are unidentifiableConfidential: can identify but will not share
• Voluntary statement assuring participants of confidence of information ascertained through survey (may be required)
• Reduces perceived cost
• Establishes trust
• Heightens awareness of what may be asked
• Misuse can lead to decrease in response rates
Other Techniques
Technique 6: Request for Help• Follows social responsibility theory• Be careful of wording in contact, especially with
subject lines
Technique 7: Sponsorship• Who is coordinating the survey• Collaborations with external entity may need
elaboration
Technique 8: Deadlines• Scarce/limited opportunity: more valuable• Results are mixed: won’t hurt, may not help
Contact/Invitation Checklist
Should convey the importance of the study
Should include a request for help
Should guarantee confidentiality (as appropriate)
Should provide information on how long it will take to complete the assessment
Should indicate the time period for which the survey will be accepting responses
Should include information on incentives
Timing of Contact/Administration
• Avoid busy times (e.g., finals) or holidays
• Send email/pre-announcement 2-3 days prior to survey mailing
• First half of semester/term may be better if you are surveying in an academic environment
• Mid-week invitations are typically recommended
Web Survey Practices•Possible ways of motivating respondents to
continue:
• Keep design simple
• Make directions clear
• Include questions on the first page
• Break the survey up into multiple pages
• Utilize a progress tool
• Utilize skip patterns so that respondents do not have to see questions that do not apply to them
Generating Interest, Participation, and Ownership in Qualitative
Research•Many of the same principles outlined for
quantitative assessment apply•For sampling, a purposeful sample is often the best
strategy, but captive audiences can also be used • Incentives (t-shirts, food, drinks) are helpful in
recruiting participants•Provide opportunities to discuss positive/negative
feedback•Students can help in data collection, analysis, and
reporting
Ownership: Include Students
When developing and administering the project, reviewing the data, and discussing potential changes/action, think about:
•Resident assistants•Peer mentors•Orientation leaders•Student club/organization leaders•Tutors/Supplemental Instruction leaders•Student employees
Coordination of Assessment Activity
• Overseeing body: person or committee that monitors activity for department, division, or institution
• Assessment calendar for division and institution• Think about what other areas on campus are
doing and collaborate with them• Partnerships between units with related missions and
objectives• Find opportunities to share assessment activities and results
(e.g., “Assessments of the Month”)• Share results widely (internally and externally): Publish an
Assessment Update
Sharing Results • Know your audience• Use drawings/other visuals to represent your assessment
plan • Use graphs/charts/other visuals to represent your findings
- Size/biggest or smallest- How things change over time- What is typical - What is exceptional- How one piece of data is related to another
• Share results back with assessment participants/ publicize assessment results in meaningful ways
Sharing Results with Students at UNCW • Reports posted on the Assessment Website, Other Websites
• See University Learning Center website: http://www.uncw.edu/stuaff/uls/si_leaders.htm
• Results shared with leadership groups (Student Government, Resident Assistants, Campus Entertainment Executive Board, Peer Mentors)
• Normative data on healthy behaviors shared in poster campaign
• “Your Voice Has Been Heard” Marketing Campaign• Space specific flyers within each department (e.g.,
Housing, Union)• Campus Television Station/Student Newspaper• Parent Newsletter
Our institution has effectively shared assessment results with students.
17% 17% 17%17%17%17%
A. Strongly AgreeB. AgreeC. Neither Agree nor DisagreeD. DisagreeE. Strongly DisagreeF. Don’t Know/Not Applicable
Other Examples from UNCW• Career Center Counseling Evaluations and Tutor Evaluations
in the University Learning Center• Web link on a laptop used instead of emailed survey
• Health Center and Housing and Residence Life• Bubble forms and paper/pencil surveys used to capture responses
from the captive audience• Health Survey
• Acronyms removed and incentives bolded in email invitation• Campus Life Study
• High salience and $100 incentives• Sustainability Film Series
• PDA’s used directly after the film
Additional Campus Examples
•University of Michigan - Spring Commencement▫New, temporary location for graduation▫Initial PDA survey followed by online surveys to graduating
students▫High publicity: emails, newspaper, website
•University of Richmond - First-Year Student Intervention▫Email from Vice President of Student Development▫6 questions about experience at UR▫Direct follow-up by UR staff if requested
Discussion: Incentives
•What incentives have you used at your institution?
•How have these affected participation rates?
•What recommendations do you have for using incentives?
Discussion: Sharing Results
•How have you shared results with your students?
•Have you noticed any effects on future participation?
•Are there other “best practices” for sharing results that you can pass along?
Discussion: Other Best Practices
•What other recommendations do you have for…▫generating student interest?▫ increasing student participation?▫enhancing student ownership in assessment?
www.chiuni.ac.uk
Thank you!
Please feel free to contact us with any questions or further ideas regarding these topics!
Marissa CopeAssociate Director of Assessment [email protected]
Nathan Lindsay, Ph.D.Director of Student Life AssessmentUniversity of North Carolina [email protected]