Mission Possible : Assessing graduate and professional Programs
description
Transcript of Mission Possible : Assessing graduate and professional Programs
![Page 1: Mission Possible : Assessing graduate and professional Programs](https://reader031.fdocuments.in/reader031/viewer/2022013004/56816426550346895dd5e445/html5/thumbnails/1.jpg)
Dr. Timothy S. BrophyDirector of Institutional Assessment
University of FloridaGainesville, FL
MISSION POSSIBLE: ASSESSING
GRADUATE AND PROFESSIONAL
PROGRAMS
![Page 2: Mission Possible : Assessing graduate and professional Programs](https://reader031.fdocuments.in/reader031/viewer/2022013004/56816426550346895dd5e445/html5/thumbnails/2.jpg)
Part 1: To introduce, describe, and explain the basic elements of student learning outcomes and program goals, their development, and measurement
Part 2: To share a structure for assessment planning and reporting for graduate and professional programs and review an example
Part 3: Review and discuss graduate sample academic assessment data reports
TODAY’S GOALS
![Page 3: Mission Possible : Assessing graduate and professional Programs](https://reader031.fdocuments.in/reader031/viewer/2022013004/56816426550346895dd5e445/html5/thumbnails/3.jpg)
Size and scope• Multiple colleges/units• Undergraduate, graduate, professional, and certificate programs
Institutional consistency• Outcomes• Assessment reporting• Cycles
Institutional CultureManagement and ToolsHonoring unit autonomy, disciplinary distinctions, and institutional requirementsFaculty comportment
COMMON CHALLENGES
![Page 4: Mission Possible : Assessing graduate and professional Programs](https://reader031.fdocuments.in/reader031/viewer/2022013004/56816426550346895dd5e445/html5/thumbnails/4.jpg)
An educational program is a coherent set of courses leading to a credential (degree, diploma, or certificate) awarded by the institution. (SACSCOC, 2011)
HOW ACCREDITORS DEFINE EDUCATIONAL PROGRAMS
![Page 5: Mission Possible : Assessing graduate and professional Programs](https://reader031.fdocuments.in/reader031/viewer/2022013004/56816426550346895dd5e445/html5/thumbnails/5.jpg)
There is a clear expectation that an institution be able to demonstrate institutional effectiveness for all its diplomas, certificates, and undergraduate and graduate educational degree programs.The expectation is that the institution will engage in ongoing planning and assessment to ensure that for each academic program, the institution develops and assesses student learning outcomes.
Program and learning outcomes specify the knowledge, skills, values, and attitudes students are expected to attain in courses or in a program.
EXPECTATIONS FOR ACADEMIC ASSESSMENT FOR ACCREDITATION
![Page 6: Mission Possible : Assessing graduate and professional Programs](https://reader031.fdocuments.in/reader031/viewer/2022013004/56816426550346895dd5e445/html5/thumbnails/6.jpg)
Methods for assessing the extent to which students achieve these outcomes are appropriate to the nature of the discipline, and consistent over time to enable the institution to evaluate cohorts of students who complete courses or a program. Shared widely within and across programs, the results of this assessment can affirm the institution’s success at achieving its mission and can be used to inform decisions about curricular and programmatic revisions.
At appropriate intervals, program and learning outcomes and assessment methods are evaluated and revised.
EXPECTATIONS FOR ACADEMIC ASSESSMENT
![Page 7: Mission Possible : Assessing graduate and professional Programs](https://reader031.fdocuments.in/reader031/viewer/2022013004/56816426550346895dd5e445/html5/thumbnails/7.jpg)
PART 1: STUDENT LEARNING
OUTCOMES, PROGRAM GOALS,
AND OUTPUTS
![Page 8: Mission Possible : Assessing graduate and professional Programs](https://reader031.fdocuments.in/reader031/viewer/2022013004/56816426550346895dd5e445/html5/thumbnails/8.jpg)
Student Learning Outcomes (SLOs) are defined generally as “what students are expected to know and be able to do by completion of their degree program”Define this for your faculty and ensure that this definition is consistent across campus and clearly posted
DEFINE AND DISSEMINATE THE TERMS
![Page 9: Mission Possible : Assessing graduate and professional Programs](https://reader031.fdocuments.in/reader031/viewer/2022013004/56816426550346895dd5e445/html5/thumbnails/9.jpg)
Undergraduate
Content Knowledge
Critical thinking
Communication
Graduate
Content Knowledge
Professional Behavior Skills
CONSIDER A CATEGORICAL ORGANIZING FRAMEWORK FOR
SLOS
![Page 10: Mission Possible : Assessing graduate and professional Programs](https://reader031.fdocuments.in/reader031/viewer/2022013004/56816426550346895dd5e445/html5/thumbnails/10.jpg)
Student Learning Outcomes reflect the curriculum the discipline, and faculty
expectations; as these elements evolve, learning outcomes change.
Recency has to do with the degree to which the outcome reflects current
knowledge and practice in the discipline.Relevance is the degree to which the
outcome relates logically and significantly to the discipline and the degree. Rigor has to do with the degree of
academic precision and thoroughness that the outcome requires to be met
successfully.
THEE CHARACTERISTICS OF SLOS:
RECENCY, RELEVANCE, AND RIGOR
![Page 11: Mission Possible : Assessing graduate and professional Programs](https://reader031.fdocuments.in/reader031/viewer/2022013004/56816426550346895dd5e445/html5/thumbnails/11.jpg)
Outputs describe and count what we do and whom we reach, and represent products or services we produce. Processes deliver outputs; what is produced at the end of a process is an output.
An outcome is a level of performance or achievement. It may be associated with a process or its output. Outcomes imply measurement - quantification - of performance.
DISTINGUISH OUTPUTS FROM OUTCOMES
![Page 12: Mission Possible : Assessing graduate and professional Programs](https://reader031.fdocuments.in/reader031/viewer/2022013004/56816426550346895dd5e445/html5/thumbnails/12.jpg)
We seek to measure outcomes as well as their associated outputs; however, SLOs focus on outcomes.
For example, while we produce a number of new graduates (the output), it is critical that we have a measure of the quality of the graduates as defined by the college or discipline (the outcome).
Outcomes describe, in measurable terms, these quality characteristics by defining our expectations for students.
OUTCOMES AND OUTPUTS: WHAT IS THE DIFFERENCE?
![Page 13: Mission Possible : Assessing graduate and professional Programs](https://reader031.fdocuments.in/reader031/viewer/2022013004/56816426550346895dd5e445/html5/thumbnails/13.jpg)
EXERCISE 1: ARE THESE RESULTS
STATEMENTS OUTPUTS OR OUTCOMES?
![Page 14: Mission Possible : Assessing graduate and professional Programs](https://reader031.fdocuments.in/reader031/viewer/2022013004/56816426550346895dd5e445/html5/thumbnails/14.jpg)
OUR PROGRAM GRADUATED 25 STUDENTS IN SPRING 2013.
A. OutputB. Outcome
Output
Outcome
0%0%
![Page 15: Mission Possible : Assessing graduate and professional Programs](https://reader031.fdocuments.in/reader031/viewer/2022013004/56816426550346895dd5e445/html5/thumbnails/15.jpg)
75% OF OUR STUDENTS ACHIEVED LEVEL 4 (OUT OF 5) ON OUR PRESENTATION ASSESSMENT RUBRIC.
A. OutputB. Outcome
Output
Outcome
0%0%
![Page 16: Mission Possible : Assessing graduate and professional Programs](https://reader031.fdocuments.in/reader031/viewer/2022013004/56816426550346895dd5e445/html5/thumbnails/16.jpg)
WE RECRUITED 10 ADDITIONAL STUDENTS IN 2013-14.
A.OutputB.Outcome
Output
Outcome
0%0%
![Page 17: Mission Possible : Assessing graduate and professional Programs](https://reader031.fdocuments.in/reader031/viewer/2022013004/56816426550346895dd5e445/html5/thumbnails/17.jpg)
IN 2013, OUR DOCTORAL STUDENTS PUBLISHED 10 PAPERS IN THE INTERNATIONAL JOURNAL OF PSYCHOLOGY .
A. OutputB. Outcome
Output
Outcome
0%0%
![Page 18: Mission Possible : Assessing graduate and professional Programs](https://reader031.fdocuments.in/reader031/viewer/2022013004/56816426550346895dd5e445/html5/thumbnails/18.jpg)
Student Learning Outcomes (SLOs) describe what students should know and be able to do as a result of completing an academic program.
Program faculty set targets for their SLOs
Program Goals describe the unit’s expectations for programmatic elements, such as admission criteria, acceptance and graduation rates, etc.
DISTINGUISH SLOS AND PROGRAM GOALS
![Page 19: Mission Possible : Assessing graduate and professional Programs](https://reader031.fdocuments.in/reader031/viewer/2022013004/56816426550346895dd5e445/html5/thumbnails/19.jpg)
EXERCISE 2: STUDENT LEARNING
OUTCOMES OR PROGRAM GOALS?
![Page 20: Mission Possible : Assessing graduate and professional Programs](https://reader031.fdocuments.in/reader031/viewer/2022013004/56816426550346895dd5e445/html5/thumbnails/20.jpg)
WE WILL LOWER OUR ATTRITION RATE TO 10%.
A. Program GoalB. Student
Learning Outcome
Program Goal
Student Learning O
utcome
0%0%
![Page 21: Mission Possible : Assessing graduate and professional Programs](https://reader031.fdocuments.in/reader031/viewer/2022013004/56816426550346895dd5e445/html5/thumbnails/21.jpg)
MUSIC EDUCATION STUDENTS DISCRIMINATE MUSICAL QUALITY BASED ON SOUND MUSICAL REASONING.
A. Program GoalB. Student
Learning Outcome
Program Goal
Student Learning O
utcome
0%0%
![Page 22: Mission Possible : Assessing graduate and professional Programs](https://reader031.fdocuments.in/reader031/viewer/2022013004/56816426550346895dd5e445/html5/thumbnails/22.jpg)
WE WILL REDUCE THE AVERAGE TIME TO DEGREE IN OUR GRADUATE PROGRAM FROM THE 2012-13 RATE OF 6.5 YEARS TO 5 YEARS IN 2014-15.
A. Program GoalB. Student
Learning Outcome
Program Goal
Student Learning O
utcome
0%0%
![Page 23: Mission Possible : Assessing graduate and professional Programs](https://reader031.fdocuments.in/reader031/viewer/2022013004/56816426550346895dd5e445/html5/thumbnails/23.jpg)
STUDENTS ANALYZE EXPERIMENTAL DATA AND INTERPRET RESULTS IN THE CELLULAR AND MOLECULAR SCIENCES.
A. Program GoalB. Student
Learning Outcome
Program Goal
Student Learning O
utcome
0%0%
![Page 24: Mission Possible : Assessing graduate and professional Programs](https://reader031.fdocuments.in/reader031/viewer/2022013004/56816426550346895dd5e445/html5/thumbnails/24.jpg)
EFFECTIVE SLOS:Focus on what students will know and be able to do.
All disciplines have a body of core knowledge that students must learn to be successful as well as a core set of applications of that knowledge in professional settings.
Describe observable and measurable actions or behaviors. Effective SLOs present a core set of observable, measureable
behaviors. Measurement tools vary from exams to complex tasks graded by rubrics.
The key to measurability: an active verb that describes a observable behavior, process, or product
A framework for developing SLOs: Bloom’s Taxonomy
ENSURE THE OUTCOME IS MEASURABLE
![Page 25: Mission Possible : Assessing graduate and professional Programs](https://reader031.fdocuments.in/reader031/viewer/2022013004/56816426550346895dd5e445/html5/thumbnails/25.jpg)
• An internal process that is indicated by demonstrated behaviors – OK for learning goals but not recommended for program or course SLOs
Understand
• Internal processes that are indicated by demonstrated behaviors closely tied to personal choice or preference; OK if the appreciation or valuing is supported by discipline-specific knowledge
Appreciate; value
• Focuses assessment on “becoming familiar,” not familiarity
Become familiar with
• Not observable; demonstrable through communication or other demonstration of learning
Learn about, think about
• Focuses assessment on becoming and/or gaining – not actual awareness
Become aware of, gain an awareness of
• Focuses assessment on ability, not achievement or demonstration of a skill
Demonstrate the ability to
VERBS AND PHRASES THAT COMPLICATE MEASURABILITY
![Page 26: Mission Possible : Assessing graduate and professional Programs](https://reader031.fdocuments.in/reader031/viewer/2022013004/56816426550346895dd5e445/html5/thumbnails/26.jpg)
This model connects course-level and program-level SLOs directly to the program learning goals
Course-level Student Learning Outcome these are determined by the faculty and specify course-level, observable products or
demonstrations
Program-level – Student Learning Outcomethese describe what students will do to demonstrate they have met the learning
goals
Program Learning Goal Level – programs establish learning goals for the degree these goals require multiple actions over time to measure
DEVELOPING MEASURABLE SLOS: A THREE-LEVEL MODEL (CARRIVEAU,
2010)
![Page 27: Mission Possible : Assessing graduate and professional Programs](https://reader031.fdocuments.in/reader031/viewer/2022013004/56816426550346895dd5e445/html5/thumbnails/27.jpg)
EXERCISE 3: ARE THE FOLLOWING
OUTCOME STATEMENTS
MEASURABLE?
![Page 28: Mission Possible : Assessing graduate and professional Programs](https://reader031.fdocuments.in/reader031/viewer/2022013004/56816426550346895dd5e445/html5/thumbnails/28.jpg)
STUDENTS UNDERSTAND GOOD WRITING STYLE.
A. YesB. No
Yes No
0%0%
![Page 29: Mission Possible : Assessing graduate and professional Programs](https://reader031.fdocuments.in/reader031/viewer/2022013004/56816426550346895dd5e445/html5/thumbnails/29.jpg)
STUDENTS SIGHT-SING A 16-MEASURE MELODY WITH NO ERRORS.
A. YesB. No
Yes No
0%0%
![Page 30: Mission Possible : Assessing graduate and professional Programs](https://reader031.fdocuments.in/reader031/viewer/2022013004/56816426550346895dd5e445/html5/thumbnails/30.jpg)
STUDENTS EXPLORE AND LEARN ABOUT GEOLOGICAL FORMATIONS.
A. YesB. No
Yes No
0%0%
![Page 31: Mission Possible : Assessing graduate and professional Programs](https://reader031.fdocuments.in/reader031/viewer/2022013004/56816426550346895dd5e445/html5/thumbnails/31.jpg)
STUDENTS DEFINE THE ETHICAL RESPONSIBILITIES OF BUSINESS ORGANIZATIONS AND IDENTIFY RELEVANT ETHICAL ISSUES.
A. YesB. No
Yes No
0%0%
![Page 32: Mission Possible : Assessing graduate and professional Programs](https://reader031.fdocuments.in/reader031/viewer/2022013004/56816426550346895dd5e445/html5/thumbnails/32.jpg)
Direct assessments of student learning are those that provide for direct examination or observation of student knowledge or skills against measurable performance indicators.
Indirect assessments are those that ascertain the opinion or self-report of the extent or value of learning experiences (Rogers, 2011)
BALANCE DIRECT AND INDIRECT ASSESSMENTS
![Page 33: Mission Possible : Assessing graduate and professional Programs](https://reader031.fdocuments.in/reader031/viewer/2022013004/56816426550346895dd5e445/html5/thumbnails/33.jpg)
EXERCISE 4: DIRECT OR INDIRECT
ASSESSMENTS?
![Page 34: Mission Possible : Assessing graduate and professional Programs](https://reader031.fdocuments.in/reader031/viewer/2022013004/56816426550346895dd5e445/html5/thumbnails/34.jpg)
COURSE FINAL EXAM
A. DirectB. Indirect
Direct
Indirect
0%0%
![Page 35: Mission Possible : Assessing graduate and professional Programs](https://reader031.fdocuments.in/reader031/viewer/2022013004/56816426550346895dd5e445/html5/thumbnails/35.jpg)
SERU OR NSSE SURVEY DATA.
A. DirectB. Indirect
Direct
Indirect
0%0%
![Page 36: Mission Possible : Assessing graduate and professional Programs](https://reader031.fdocuments.in/reader031/viewer/2022013004/56816426550346895dd5e445/html5/thumbnails/36.jpg)
FINAL PAPER, PERFORMANCE, OR PRESENTATION GRADED BY A FACULTY DEVELOPED RUBRIC.
A. DirectB. Indirect
Direct
Indirect
0%0%
![Page 37: Mission Possible : Assessing graduate and professional Programs](https://reader031.fdocuments.in/reader031/viewer/2022013004/56816426550346895dd5e445/html5/thumbnails/37.jpg)
SENIOR EXIT INTERVIEW.
A. DirectB. Indirect
Direct
Indirect
0%0%
![Page 38: Mission Possible : Assessing graduate and professional Programs](https://reader031.fdocuments.in/reader031/viewer/2022013004/56816426550346895dd5e445/html5/thumbnails/38.jpg)
PART 2: PLANNING AND REPORTING
![Page 39: Mission Possible : Assessing graduate and professional Programs](https://reader031.fdocuments.in/reader031/viewer/2022013004/56816426550346895dd5e445/html5/thumbnails/39.jpg)
Academic Assessment Plans provide a common framework for units to plan how they assess and measure student achievement of the SLOs Plans also present the process for how the data from these assessments are used to enhance the quality of student learning
ACADEMIC ASSESSMENT PLANNING
![Page 40: Mission Possible : Assessing graduate and professional Programs](https://reader031.fdocuments.in/reader031/viewer/2022013004/56816426550346895dd5e445/html5/thumbnails/40.jpg)
Provides faculty a focal point for the discussion of the assessment of student learning in the degree programs.
Planning discussions provide an opportunity to revisit the curriculum and its relationship to the SLOs.
Provides a consistent reference resource when faculty and leadership change.
WHY PLAN?
![Page 41: Mission Possible : Assessing graduate and professional Programs](https://reader031.fdocuments.in/reader031/viewer/2022013004/56816426550346895dd5e445/html5/thumbnails/41.jpg)
Graduate /Professio
nal program
Assessment Plan
Mission Alignment
Student
Learning
Outcomes
Research
Assessment Timeline
Assessment Cycle
Methods and
Procedures
Assessment Oversight
GRADUATE AND PROFESSIONAL PROGRAM ASSESSMENT PLAN
![Page 42: Mission Possible : Assessing graduate and professional Programs](https://reader031.fdocuments.in/reader031/viewer/2022013004/56816426550346895dd5e445/html5/thumbnails/42.jpg)
Call issued by the Office of Institutional Assessment
SACS coordinators collect and
review plans
Plans are submitted to
the online approval system
Director of Institutional Assessment
reviews plans
Approved plans go to the Academic
Assessment Committee
(AAC)
Plans approved by the AAC go
to the University Curriculum Committee
(UCC)
UCC approves plans and
forwards to the Student
Academic Support System
or Graduate School
These are uploaded to the
catalog
ACADEMIC ASSESSMENT PLAN APPROVAL PROCESS
![Page 43: Mission Possible : Assessing graduate and professional Programs](https://reader031.fdocuments.in/reader031/viewer/2022013004/56816426550346895dd5e445/html5/thumbnails/43.jpg)
Template: Figure 4 – pp. 3-4 in your handoutRubric: Figure 1, p. 2
SECTIONS OF THE PLAN
![Page 44: Mission Possible : Assessing graduate and professional Programs](https://reader031.fdocuments.in/reader031/viewer/2022013004/56816426550346895dd5e445/html5/thumbnails/44.jpg)
Describe briefly the program’s mission, and how the program meets the department, college, and university missions. For example:
The mission of the<enter name> program is to <enter text>. This aligns with the department mission by <enter text.> It also supports the college mission to <enter text>. It supports the university mission by <enter text>.
MISSION
![Page 45: Mission Possible : Assessing graduate and professional Programs](https://reader031.fdocuments.in/reader031/viewer/2022013004/56816426550346895dd5e445/html5/thumbnails/45.jpg)
The complete file of graduate and professional program SLOs is on the Institutional Assessment website – http://assessment.aa.ufl.edu
UF’s Graduate SLO categories are Content, Skills, and Professional Behaviors
Online resources at the UF Institutional Assessment website:
“Writing Measurable Student Learning Outcomes” PowerPoint
“Guide to Writing Student Learning Outcomes”
STUDENT LEARNING OUTCOMES
![Page 46: Mission Possible : Assessing graduate and professional Programs](https://reader031.fdocuments.in/reader031/viewer/2022013004/56816426550346895dd5e445/html5/thumbnails/46.jpg)
What are the research expectations for students in your program?
Briefly describe the research expectations for students in the degree program.
How does your program prepare the students to become researchers in the discipline?
If the degree is NOT a research degree, briefly state this, and include a brief description of any research-related activities that students complete in the program.
RESEARCH
![Page 47: Mission Possible : Assessing graduate and professional Programs](https://reader031.fdocuments.in/reader031/viewer/2022013004/56816426550346895dd5e445/html5/thumbnails/47.jpg)
The Assessment Timeline is a matrix that shows when the SLOs assessed and measured in the program
It should be clear to a student when an assessment occurs, and the type of assessment that is planned (assignment, project, paper, performance, presentation, etc.)
ASSESSMENT TIMELINE
![Page 48: Mission Possible : Assessing graduate and professional Programs](https://reader031.fdocuments.in/reader031/viewer/2022013004/56816426550346895dd5e445/html5/thumbnails/48.jpg)
Assessment SLOs
Assessment 1
Assessment 2
Assessment 3
Enter more as needed
Content Knowledge
#1
#2
Skills
#3
#4 Professional Behavior
#5
#6
ASSESSMENT TIMELINEProgram_____________________________________ College ____________________________________
![Page 49: Mission Possible : Assessing graduate and professional Programs](https://reader031.fdocuments.in/reader031/viewer/2022013004/56816426550346895dd5e445/html5/thumbnails/49.jpg)
The Assessment Cycle is a matrix that graphically organizes the frequency of SLO assessment
The Assessment Cycle is a multi-year process that is completed in three years
It should be clear to a student when an assessment occurs, and the type of assessment that is planned (assignment, project, paper, performance, presentation, etc.)
ASSESSMENT CYCLE
![Page 50: Mission Possible : Assessing graduate and professional Programs](https://reader031.fdocuments.in/reader031/viewer/2022013004/56816426550346895dd5e445/html5/thumbnails/50.jpg)
YearSLOs
11-12 12-13 13-14 14-15 15-16
Content Knowledge #1 #2
Critical Thinking (Undergrad)Skills (Grad/Prof)
#3 #4
Communication (Undergrad)Professional Behavior (Grad/Prof)
#5 #6
ASSESSMENT CYCLEProgram College _Analysis and Interpretation: [Enter date or time frame here]Improvement Actions: Completed by [Enter date here]Dissemination: Completed by [Enter date here]
![Page 51: Mission Possible : Assessing graduate and professional Programs](https://reader031.fdocuments.in/reader031/viewer/2022013004/56816426550346895dd5e445/html5/thumbnails/51.jpg)
Each unit employs various methods and procedures to assess and collect data on student learning.
In this section of the plan, units provide information on their specific methods and procedures for the SLO assessments they identify in Assessment Timelines.
They must include a sample grading rubric.
It’s OK to sample the total student population – see the recommended sample size chart on the Institutional Assessment website
METHODS AND PROCEDURES
![Page 52: Mission Possible : Assessing graduate and professional Programs](https://reader031.fdocuments.in/reader031/viewer/2022013004/56816426550346895dd5e445/html5/thumbnails/52.jpg)
These are the
individuals who are
responsible for
managing the
assessment work in
your unit
List everyone and their contact
information
The first person on
the list should be the lead contact
Purpose: ongoing
communication
ASSESSMENT OVERSIGHT
![Page 53: Mission Possible : Assessing graduate and professional Programs](https://reader031.fdocuments.in/reader031/viewer/2022013004/56816426550346895dd5e445/html5/thumbnails/53.jpg)
DEVELOP A SYSTEM OR CYCLE OF PLANNING AND REPORTING
Sp
ring
A
ssessmen
t P
lans
sub
mitted
for th
e next
AY
Fall Assessment Data, results,
and use of results for
previous AY reported
Planning Reporting
Establish Mission,
Goals, and Outcomes
Assessment Planning
Implement the Plan
and Gather Data
Interpret and
Evaluate the Data
Modify and Improve
![Page 54: Mission Possible : Assessing graduate and professional Programs](https://reader031.fdocuments.in/reader031/viewer/2022013004/56816426550346895dd5e445/html5/thumbnails/54.jpg)
The review of Academic Assessment Plans is a constructive exercise designed to guide faculty to consider their assessment work thoughtfully.The 2012-13 PhD in English Academic Assessment Plan is on pp. 8-11 of your handout. Using the rubric on p. 2, discuss in your group the degree to which this plan meets the guidelines.
Does the plan meet or not meet the criteria?
What comments would you provide?
ASSESSMENT PLANNING EXERCISE: PHD IN ENGLISH
![Page 55: Mission Possible : Assessing graduate and professional Programs](https://reader031.fdocuments.in/reader031/viewer/2022013004/56816426550346895dd5e445/html5/thumbnails/55.jpg)
REPORTING ASSESSMENT RESULTS – STUDENT LEARNING OUTCOMES
(P. 5)Assessment Method:• List the assignment, exam, project, etc.• If this is a sample, describe the sampling procedure used
Results:• Enter the criterion for success. The “criterion for success” is the minimum
percentage of students who pass the assessment that you consider to be acceptable for your program. If the criterion is less than 70%, provide a rationale.
• State: “X number of students passed the assessment out of a total of Y students, for a percentage of Z%”.
• State: This meets/does not meet the criterion for success.• Attach the data you shared with your faculty (student names redacted). Use of Results:• State who reviewed the results.• Refer to the results that were reviewed.• State actions taken in past tense. For example:
• “Based on our review, we decided to…”• “We modified our SLO #1 because the data indicated that…”• “We changed the course content for ABCXXXX effective fall 20XX because
the data revealed that…”
![Page 56: Mission Possible : Assessing graduate and professional Programs](https://reader031.fdocuments.in/reader031/viewer/2022013004/56816426550346895dd5e445/html5/thumbnails/56.jpg)
REPORTING ASSESSMENT RESULTS – ACADEMIC PROGRAM GOALS
Assessment Method:• State the measurement method
Results:• Briefly state your results• Include or attach the data you collected in summary formUse of Results:• State who reviewed the results.• Refer to the results that were reviewed.• State actions taken in past tense. For example:
• “Based on our review, we decided to…”• “Our results led us to modify our goal to…”• “We developed a new measure for this long term goal
based on our review…”
![Page 57: Mission Possible : Assessing graduate and professional Programs](https://reader031.fdocuments.in/reader031/viewer/2022013004/56816426550346895dd5e445/html5/thumbnails/57.jpg)
DEVELOP A QUALITY ASSURANCE
PROCESS
![Page 58: Mission Possible : Assessing graduate and professional Programs](https://reader031.fdocuments.in/reader031/viewer/2022013004/56816426550346895dd5e445/html5/thumbnails/58.jpg)
Multi-step, institutional review and
approval process
Templates and rubrics for guiding
faculty through the process
Review and evaluate faculty
submissions
Cross-reference plans with data
reported annually
Develop and provide
professional development
Model the process: Modify
and improve quality assurance processes based on the data you
collect
ELEMENTS OF QUALITY ASSURANCE
![Page 59: Mission Possible : Assessing graduate and professional Programs](https://reader031.fdocuments.in/reader031/viewer/2022013004/56816426550346895dd5e445/html5/thumbnails/59.jpg)
2012-13 ACADEMIC ASSESSMENT DATA REPORTING REVIEW RESULTSIn early 2014 all of the 2012-13 graduate and professional academic assessment data reports were reviewed.76% were returned for revisions due to failure to follow our guidelinesThe most requested revision was in the Use of Results field. Specifically, programs did not state who reviewed the results. Programs were most likely cited for Program Goal Use of Results statements.Many programs were cited for not stating results and use of results in the past tense
![Page 60: Mission Possible : Assessing graduate and professional Programs](https://reader031.fdocuments.in/reader031/viewer/2022013004/56816426550346895dd5e445/html5/thumbnails/60.jpg)
PART 3: EXAMPLES
![Page 61: Mission Possible : Assessing graduate and professional Programs](https://reader031.fdocuments.in/reader031/viewer/2022013004/56816426550346895dd5e445/html5/thumbnails/61.jpg)
The 2011-12 Data Analysis Report for the PhD in English is on pp. 6-7 of your handout (Figures 7 and 8).
The 2012-13 Data Analysis Report for this program is on pp. 12-13 of your handout (Figure 10). This report is based on the 2012-13 AAP you have just reviewed. Compare the two data reports. What differences and similarities do you find? Using the Data Entry Guides for Faculty in Figure 5 (p. 5 in your handout) , discuss the degree to which the 2012-13 data report complies with these guidelines. Note: the guidelines were provided beginning with the 2012-13 report; the 2011-12 report was completed without these guidelines.
DATA REPORTING REVIEW EXERCISE – PHD IN ENGLISH
![Page 62: Mission Possible : Assessing graduate and professional Programs](https://reader031.fdocuments.in/reader031/viewer/2022013004/56816426550346895dd5e445/html5/thumbnails/62.jpg)
We will review Figure 11 together, on pp. 14-16 of your handout
OTHER EXAMPLES OF DATA REPORTS
![Page 63: Mission Possible : Assessing graduate and professional Programs](https://reader031.fdocuments.in/reader031/viewer/2022013004/56816426550346895dd5e445/html5/thumbnails/63.jpg)
1. Define the terms and disseminate them2. Consider an institutional categorical
organizing framework for SLOs3. Recency, Relevance, and Rigor4. Distinguish Outputs from Outcomes 5. Distinguish SLOs from Program Goals6. Ensure the outcome is measurable7. Balance direct and indirect assessments8. Planning Timeline/Cycle9. Templates and Rubrics10.Approval and Management Process11.A system or cycle of planning and
reporting12.Quality Assurance Process
A SUMMARY
![Page 64: Mission Possible : Assessing graduate and professional Programs](https://reader031.fdocuments.in/reader031/viewer/2022013004/56816426550346895dd5e445/html5/thumbnails/64.jpg)
Timothy S. Brophy, Ph.D.Director, Institutional Assessment
235 Tigert HallOffice of the Provost
Email: [email protected]: 352-273-4476
QUESTIONS