Applying the QM Rubric 2011-13 Rubric ©2014 MarylandOnline.
-
Upload
clifford-thistlethwaite -
Category
Documents
-
view
230 -
download
4
Transcript of Applying the QM Rubric 2011-13 Rubric ©2014 MarylandOnline.
![Page 1: Applying the QM Rubric 2011-13 Rubric ©2014 MarylandOnline.](https://reader036.fdocuments.in/reader036/viewer/2022062312/551c471c550346b1458b4826/html5/thumbnails/1.jpg)
Applying the QM Rubric
2011-13 Rubric
©2014 MarylandOnline
![Page 2: Applying the QM Rubric 2011-13 Rubric ©2014 MarylandOnline.](https://reader036.fdocuments.in/reader036/viewer/2022062312/551c471c550346b1458b4826/html5/thumbnails/2.jpg)
Workshop Overview
• Today’s Agenda• Folder Overview• QM Website
www.qmprogram.org• QM Contact Information
©2014 MarylandOnline
![Page 3: Applying the QM Rubric 2011-13 Rubric ©2014 MarylandOnline.](https://reader036.fdocuments.in/reader036/viewer/2022062312/551c471c550346b1458b4826/html5/thumbnails/3.jpg)
Login to MyQM
1. Go to MyQMhttp://www.qmprogram.org/myQM/
2. Click on Need an Account? link (lower part of page) or login if you already have an account
3. Review the MyQM site: My Account, My Activity, My Tools, and Workshops
4. Certificates
©2014 MarylandOnline
![Page 4: Applying the QM Rubric 2011-13 Rubric ©2014 MarylandOnline.](https://reader036.fdocuments.in/reader036/viewer/2022062312/551c471c550346b1458b4826/html5/thumbnails/4.jpg)
After this workshop, you will be able to:
1. Identify the underlying principles of QM. (Recognize key QM underlying principles and concepts.)
2. Identify the critical elements of the QM quality assurance program, including the QM Rubric, materials, processes, and administrative components.
3. Apply the 2011-2013 QM Rubric to review online courses. 4. Decide if the SPCH 1113 course meets selected QM Rubric
standards. 5. Apply the concept of alignment. 6. Write helpful recommendations for course improvement by
citing annotations from the QM Rubric and evidence from the course.
©2014 MarylandOnline
![Page 5: Applying the QM Rubric 2011-13 Rubric ©2014 MarylandOnline.](https://reader036.fdocuments.in/reader036/viewer/2022062312/551c471c550346b1458b4826/html5/thumbnails/5.jpg)
And in addition, we expect that you will:
1. Collaborate and network with colleagues through substantive and timely interactions.
2. Reflect on key QM concepts and whether you wish to serve on a peer review team.
3. Explore the challenges of online teaching by familiarizing yourself with the QM rubric and the SPCH 1113 course.
4. Relate the QM rubric (standards and annotations) to your own course and consider changes that might benefit your online students.
5. Evaluate the QM APPQMR workshop.
©2014 MarylandOnline
![Page 6: Applying the QM Rubric 2011-13 Rubric ©2014 MarylandOnline.](https://reader036.fdocuments.in/reader036/viewer/2022062312/551c471c550346b1458b4826/html5/thumbnails/6.jpg)
Activity 1: Introductions
• Share name, institution, job, and best distance learning practice.
• Briefly (in one sentence) describe your best distance learning practice.
©2014 MarylandOnline
![Page 7: Applying the QM Rubric 2011-13 Rubric ©2014 MarylandOnline.](https://reader036.fdocuments.in/reader036/viewer/2022062312/551c471c550346b1458b4826/html5/thumbnails/7.jpg)
Activity 2: Trying it out with Standard 1.2
Standard 1.2: Students are introduced to the purpose and structure of the course.
©2014 MarylandOnline
![Page 8: Applying the QM Rubric 2011-13 Rubric ©2014 MarylandOnline.](https://reader036.fdocuments.in/reader036/viewer/2022062312/551c471c550346b1458b4826/html5/thumbnails/8.jpg)
Review Scenario
This course reflects a shift in the importance that the world outside the schools increasingly places on thinking and problem solving. Procedural skills alone do not prepare students for that world. Therefore, students deserve a curriculum that develops their mathematical power and an assessment system that enables them to show it. Assessments that match the current vision of school mathematics involve activities that are based on significant and correct mathematics. These activities provide all students with opportunities to formulate problems, reason mathematically, and make connections among mathematical ideas. Students engage in solving realistic problems using information and the technological tools available in real life. Moreover, skills, procedural knowledge, and factual knowledge are assessed as part of the doing of real life mathematics.
1. 5-Tests (Must be taken on campus) 4002. 15-Quizzes 753. Project (Optional) 50Total number of points possible 475 (no project)
525 (if you do the project)Grading Scale:90% to 100% = A 80% to 89% = B 70% to 79% = C 60% to 69% = D 0% to 59% = F
A student who earns a final average of 70% or more on all assessments of objectives and intended learning outcomes for the College Algebra course has successfully fulfilled the general education and other essential core skill goals. Any extra credit problems are given to the entire class. There are no extra credit problems for any one individual during any part of the semester.
You are reviewing a College Algebra course and find the following statement:
©2014 MarylandOnline
![Page 9: Applying the QM Rubric 2011-13 Rubric ©2014 MarylandOnline.](https://reader036.fdocuments.in/reader036/viewer/2022062312/551c471c550346b1458b4826/html5/thumbnails/9.jpg)
About Quality Matters
©2014 MarylandOnline
![Page 10: Applying the QM Rubric 2011-13 Rubric ©2014 MarylandOnline.](https://reader036.fdocuments.in/reader036/viewer/2022062312/551c471c550346b1458b4826/html5/thumbnails/10.jpg)
Underlying Principles of QM
QM is a faculty-driven, peer review process that is…
CollaborativeCollegialContinuousCentered - in academic foundation
- around student learning
©2014 MarylandOnline
![Page 11: Applying the QM Rubric 2011-13 Rubric ©2014 MarylandOnline.](https://reader036.fdocuments.in/reader036/viewer/2022062312/551c471c550346b1458b4826/html5/thumbnails/11.jpg)
Peer Course Review Process
©2014 MarylandOnline
![Page 12: Applying the QM Rubric 2011-13 Rubric ©2014 MarylandOnline.](https://reader036.fdocuments.in/reader036/viewer/2022062312/551c471c550346b1458b4826/html5/thumbnails/12.jpg)
For Our Purposes, Quality Is…
• More than average; more than “good enough”
• Attempt to capture what’s expected in an effective online course at about an 85% level
• Based on research and widely accepted standards
©2014 MarylandOnline
![Page 13: Applying the QM Rubric 2011-13 Rubric ©2014 MarylandOnline.](https://reader036.fdocuments.in/reader036/viewer/2022062312/551c471c550346b1458b4826/html5/thumbnails/13.jpg)
Factors Affecting Course Quality
QM Reviews Course Design ONLY©2014 MarylandOnline
![Page 14: Applying the QM Rubric 2011-13 Rubric ©2014 MarylandOnline.](https://reader036.fdocuments.in/reader036/viewer/2022062312/551c471c550346b1458b4826/html5/thumbnails/14.jpg)
Design vs. Delivery
The faculty member is integral to both design and delivery.
QM is about DESIGN - not delivery or faculty performance
Course Design … is the forethought and planning that a faculty member puts into the course.
Course Delivery …is the actual teaching of the course, the implementation of the design.
©2014 MarylandOnline
![Page 15: Applying the QM Rubric 2011-13 Rubric ©2014 MarylandOnline.](https://reader036.fdocuments.in/reader036/viewer/2022062312/551c471c550346b1458b4826/html5/thumbnails/15.jpg)
Distinguish between design and delivery…
Example: Discussion Board
Design:
Discussion board planned in course; students told how they should participate and how they can expect the faculty to participate.
Delivery:
How often the faculty member actually participates in the discussion; what the faculty member actually says to students.
©2014 MarylandOnline
![Page 16: Applying the QM Rubric 2011-13 Rubric ©2014 MarylandOnline.](https://reader036.fdocuments.in/reader036/viewer/2022062312/551c471c550346b1458b4826/html5/thumbnails/16.jpg)
Activity 3: Design vs. Delivery
• Find a partner (or two)• Share stories about your own
experiences with design vs. delivery (professional or personal)
• Decide on an experience that you’d like to share with the group.
©2014 MarylandOnline
![Page 17: Applying the QM Rubric 2011-13 Rubric ©2014 MarylandOnline.](https://reader036.fdocuments.in/reader036/viewer/2022062312/551c471c550346b1458b4826/html5/thumbnails/17.jpg)
What QM is NOT About…
• Not about an individual instructor (it’s about the course)
• Not about faculty evaluation (it’s about course quality)
• Not about judgment(it’s about diagnosis and improvement)
• Not about “win/lose” or “pass/fail” (it’s about continuous improvement in a supportive environment)
©2014 MarylandOnline
![Page 18: Applying the QM Rubric 2011-13 Rubric ©2014 MarylandOnline.](https://reader036.fdocuments.in/reader036/viewer/2022062312/551c471c550346b1458b4826/html5/thumbnails/18.jpg)
The QM Rubric
©2014 MarylandOnline
![Page 19: Applying the QM Rubric 2011-13 Rubric ©2014 MarylandOnline.](https://reader036.fdocuments.in/reader036/viewer/2022062312/551c471c550346b1458b4826/html5/thumbnails/19.jpg)
About The QM Rubric 2011-2013
Eight General Standards:
1. Course Overview and Introduction2. Learning Objectives (Competencies)3. Assessment and Measurement4. Resources and Materials5. Learner Engagement6. Course Technology7. Learner Support8. Accessibility
Key components must align.
Alignment: Critical course elements work together to ensure that students achieve the desired learning outcomes.
©2014 MarylandOnline
![Page 20: Applying the QM Rubric 2011-13 Rubric ©2014 MarylandOnline.](https://reader036.fdocuments.in/reader036/viewer/2022062312/551c471c550346b1458b4826/html5/thumbnails/20.jpg)
Key Sections that Must Align
©2014 MarylandOnline
Submitted by James Fowlkes & Brenda Boyd
![Page 21: Applying the QM Rubric 2011-13 Rubric ©2014 MarylandOnline.](https://reader036.fdocuments.in/reader036/viewer/2022062312/551c471c550346b1458b4826/html5/thumbnails/21.jpg)
QM Rubric – Standard Point Values
8 General Standards– 21 Standards are “Essential”– 12 Standards are “Very Important”– 8 Standards are “Important”
# Standards Points Relative Value
21 3 Essential
12 2 Very Important
8 1 Important
©2014 MarylandOnline
![Page 22: Applying the QM Rubric 2011-13 Rubric ©2014 MarylandOnline.](https://reader036.fdocuments.in/reader036/viewer/2022062312/551c471c550346b1458b4826/html5/thumbnails/22.jpg)
Activity 4: QM Rubric Puzzle
• Divide into groups of 3• You will have 10 minutes to:
– Decide which general standard each specific review standard supports
– Decide whether your standard is essential, very important or important.
– Note: your group does not have come to agreement on where to place the specific standard or its relative value; majority rules
©2014 MarylandOnline
![Page 23: Applying the QM Rubric 2011-13 Rubric ©2014 MarylandOnline.](https://reader036.fdocuments.in/reader036/viewer/2022062312/551c471c550346b1458b4826/html5/thumbnails/23.jpg)
The Faculty Developer (Instructor)Worksheet
©2014 MarylandOnline
![Page 24: Applying the QM Rubric 2011-13 Rubric ©2014 MarylandOnline.](https://reader036.fdocuments.in/reader036/viewer/2022062312/551c471c550346b1458b4826/html5/thumbnails/24.jpg)
Faculty Developer (Instructor) Worksheet
• The Voice of the Instructor• Key Piece in the review• Includes information about
– Institutionally mandated objectives, materials, practices, and policies
– Materials outside the course site– Types of interaction used and instructor’s statement on
the appropriateness of interaction in the course
• Read it before you begin the review• Refer to it during the review and in team discussions
©2014 MarylandOnline
![Page 25: Applying the QM Rubric 2011-13 Rubric ©2014 MarylandOnline.](https://reader036.fdocuments.in/reader036/viewer/2022062312/551c471c550346b1458b4826/html5/thumbnails/25.jpg)
Activity 5: Faculty Developer Worksheet
• Locate the Faculty Developer (Instructor) Worksheet in your packet
• Read the Worksheet, be sure to pay close attention to the instructor’s answers to:– Course- and Module-Level Objectives– Student Engagement– Technology– Accessibility– Assessment
©2014 MarylandOnline
![Page 26: Applying the QM Rubric 2011-13 Rubric ©2014 MarylandOnline.](https://reader036.fdocuments.in/reader036/viewer/2022062312/551c471c550346b1458b4826/html5/thumbnails/26.jpg)
Hands-On Practice
©2014 MarylandOnline
![Page 27: Applying the QM Rubric 2011-13 Rubric ©2014 MarylandOnline.](https://reader036.fdocuments.in/reader036/viewer/2022062312/551c471c550346b1458b4826/html5/thumbnails/27.jpg)
Goals for Hands-on Practice
1. Introduce the General Standards2. Use the Faculty Developer Worksheet3. Take an in-depth look at Alignment4. Make decisions on specific review standards
by looking at an online course5. Practice writing useful recommendations6. See how the recommendations affect a
course
©2014 MarylandOnline
![Page 28: Applying the QM Rubric 2011-13 Rubric ©2014 MarylandOnline.](https://reader036.fdocuments.in/reader036/viewer/2022062312/551c471c550346b1458b4826/html5/thumbnails/28.jpg)
Your Point of View …
As a QM Peer Reviewer, you should:• Take the students’ point of view • Advocate for the student• Support your decisions with
– Citations from the standards and annotations– Evidence from the course
©2014 MarylandOnline
![Page 29: Applying the QM Rubric 2011-13 Rubric ©2014 MarylandOnline.](https://reader036.fdocuments.in/reader036/viewer/2022062312/551c471c550346b1458b4826/html5/thumbnails/29.jpg)
Accessing the SPCH 1113 Course
1. Find a Partner
2. Read Faculty Developer (Instructor) Worksheet for course
3. Login to Principles of Speech (SPCH 1113)
• URL: https://sautech.blackboard.com/• Username: qmstudent• Password: qmstudent
©2014 MarylandOnline
![Page 30: Applying the QM Rubric 2011-13 Rubric ©2014 MarylandOnline.](https://reader036.fdocuments.in/reader036/viewer/2022062312/551c471c550346b1458b4826/html5/thumbnails/30.jpg)
Strategies for Applying the QM Rubric
General Standard 1: Course Overview and Introduction
©2014 MarylandOnline
![Page 31: Applying the QM Rubric 2011-13 Rubric ©2014 MarylandOnline.](https://reader036.fdocuments.in/reader036/viewer/2022062312/551c471c550346b1458b4826/html5/thumbnails/31.jpg)
General Standard 1
About the General Standard
Brief Description from the Rubric
To meet the General Standard
General Standard 1,Course Overview and Introduction:
The overall design of the course is made clear to the student at the beginning of the course.
The course introduction sets the tone for the course, tells students what to expect, and provides guidance to ensure they get off to a good start.
You must share your deep knowledge of course1. Organization2. Navigation3. Expectations (Policies)
©2014 MarylandOnline
![Page 32: Applying the QM Rubric 2011-13 Rubric ©2014 MarylandOnline.](https://reader036.fdocuments.in/reader036/viewer/2022062312/551c471c550346b1458b4826/html5/thumbnails/32.jpg)
How To Decide… For EACH Standard
1. Read the specific review standard and the annotation; review the examples.
a. Identify key components: ideas, directions, examples, etc.b. Ask relevant questions
2. Look for evidence that the standard is met in this course.
3. Ask yourself: Does this course meet the standard at an 85% or better level?
4. Decide Met or Not Met.
5. Write Helpful Recommendations.
©2014 MarylandOnline
![Page 33: Applying the QM Rubric 2011-13 Rubric ©2014 MarylandOnline.](https://reader036.fdocuments.in/reader036/viewer/2022062312/551c471c550346b1458b4826/html5/thumbnails/33.jpg)
Look at a Helpful Recommendation
©2014 MarylandOnline
![Page 34: Applying the QM Rubric 2011-13 Rubric ©2014 MarylandOnline.](https://reader036.fdocuments.in/reader036/viewer/2022062312/551c471c550346b1458b4826/html5/thumbnails/34.jpg)
Review Standard 1.1
1.1 Instructions make clear how to get started and where to find various course components.
As a Reviewer:What do you do first? Where do you start?
(Refer to the “How to Decide” handout)
©2014 MarylandOnline
![Page 35: Applying the QM Rubric 2011-13 Rubric ©2014 MarylandOnline.](https://reader036.fdocuments.in/reader036/viewer/2022062312/551c471c550346b1458b4826/html5/thumbnails/35.jpg)
Read the Annotation
Instructions provide a general course overview, present the schedule of activities, guide the new student to explore the course website, and indicate what to do first, in addition to listing detailed navigational instructions for the whole course.
Instructors may choose to incorporate some of this information in the course syllabus. In this case, students should be directed to the syllabus at the beginning of the course. A useful feature is a “Read Me First” or “Start Here” button or icon on the course home page, linking students to start-up information.
Examples: 1. A course “tour” 2. Clear statements about how to get started in the course 3. A “scavenger hunt” assignment that leads students through an exploration of the different areas of the
course 4. A graphical table or diagram that depicts the relationship between the online and face-to-face portions of
a blended course
Blended Courses: Instructions in the online classroom make it clear to students that the course is a blended course, with both online and face-to-face components and activities. Instructions specify the requirements for participation in both the online and face-to-face portions of the course. The introductory information clearly states when and where students should participate each week, and a structured set of topics and a schedule are provided for each face-to-face meeting.
©2014 MarylandOnline
![Page 36: Applying the QM Rubric 2011-13 Rubric ©2014 MarylandOnline.](https://reader036.fdocuments.in/reader036/viewer/2022062312/551c471c550346b1458b4826/html5/thumbnails/36.jpg)
Questions to ask:1. Does the instructor
provide instructions about key course components?
2. Do the students know what to do first and how to start the course?
Annotations specify: 1. What to look for:
• Instructions: a general course overview, schedule, website
• Indication of what to do first• Detailed navigation instructions
2. Where to look: • Course syllabus • “Read Me First” or “Start Here” button
or icon
3. Examples
Standard 1.1 – How to use the Annotations
©2014 MarylandOnline
![Page 37: Applying the QM Rubric 2011-13 Rubric ©2014 MarylandOnline.](https://reader036.fdocuments.in/reader036/viewer/2022062312/551c471c550346b1458b4826/html5/thumbnails/37.jpg)
Apply Standard 1.1
• Apply Standard 1.1 to the SPCH 1113 course– Imagine that you are a new student (new to the course, the
discipline, and/or the Learning Management System)
• Consider the following questions: – Do you know what to do first?– Do you know how to start the course? – Can you easily find clear instructions?– Note your initial reaction to the course:
• Do you feel comfortable with the navigation?• What are your first impressions about how this course works?
©2014 MarylandOnline
![Page 38: Applying the QM Rubric 2011-13 Rubric ©2014 MarylandOnline.](https://reader036.fdocuments.in/reader036/viewer/2022062312/551c471c550346b1458b4826/html5/thumbnails/38.jpg)
Activity 6: Your Decision
MET NOT MET
Standard 1.1
©2014 MarylandOnline
![Page 39: Applying the QM Rubric 2011-13 Rubric ©2014 MarylandOnline.](https://reader036.fdocuments.in/reader036/viewer/2022062312/551c471c550346b1458b4826/html5/thumbnails/39.jpg)
Consequences of Your Decision?
In a formal QM review:
— What does a MET decision mean?
— What does a NOT MET decision mean?
©2014 MarylandOnline
![Page 40: Applying the QM Rubric 2011-13 Rubric ©2014 MarylandOnline.](https://reader036.fdocuments.in/reader036/viewer/2022062312/551c471c550346b1458b4826/html5/thumbnails/40.jpg)
Importance of Learning Objectives
General Standard 2:Learning Objectives (Competencies)
©2014 MarylandOnline
![Page 41: Applying the QM Rubric 2011-13 Rubric ©2014 MarylandOnline.](https://reader036.fdocuments.in/reader036/viewer/2022062312/551c471c550346b1458b4826/html5/thumbnails/41.jpg)
General Standard 2
About the General Standard
Brief Description from the Rubric
To meet the General Standard
General Standard 2,Learning Objectives:
Learning objectives are measurable and are stated clearly.
The learning objectives establish a foundation upon which the rest of the course is based.
You must provide measurable, precise learning objectives at the course- and module-level objectives so your students will know what is expected of them.
©2014 MarylandOnline
![Page 42: Applying the QM Rubric 2011-13 Rubric ©2014 MarylandOnline.](https://reader036.fdocuments.in/reader036/viewer/2022062312/551c471c550346b1458b4826/html5/thumbnails/42.jpg)
Standard 2 as a Sentence
Quality is measurable objectives (2.1) and consistent module-level objectives (2.2), that are written from the student perspective (2.3), with instructions on how to meet them (2.4), that are appropriate for the level of the course (2.5).
Submitted by Sasha Thackaberry, Cuyahoga Community College©2014 MarylandOnline
![Page 43: Applying the QM Rubric 2011-13 Rubric ©2014 MarylandOnline.](https://reader036.fdocuments.in/reader036/viewer/2022062312/551c471c550346b1458b4826/html5/thumbnails/43.jpg)
Review Standard 2.1
2.1 The course learning objectives describe outcomes that are measurable.
As a Reviewer:What do you do first? Where do you start?
(Refer to the “How to Decide” handout)
©2014 MarylandOnline
![Page 44: Applying the QM Rubric 2011-13 Rubric ©2014 MarylandOnline.](https://reader036.fdocuments.in/reader036/viewer/2022062312/551c471c550346b1458b4826/html5/thumbnails/44.jpg)
Review Standard 2.2
2.2 The module/unit learning objectives describe outcomes that are measurable and consistent with the course-level objectives (alignment).
As a Reviewer:What do you do first? Where do you start?
(Refer to the “How to Decide” handout)
©2014 MarylandOnline
![Page 45: Applying the QM Rubric 2011-13 Rubric ©2014 MarylandOnline.](https://reader036.fdocuments.in/reader036/viewer/2022062312/551c471c550346b1458b4826/html5/thumbnails/45.jpg)
Standard 2.1 and 2.2How to apply the Annotations
Questions to ask:1. Are there course level
objectives?2. Are there module level
objectives?3. Are they measurable?4. Are they precise?5. Do they provide criteria for
assessment?6. Do they describe a behavior
that can be observed or evaluated?
Annotations specify: 1. What to look for:
– Measurable course and module learning objectives precisely describe what students are to gain from instruction and provide the criteria instructors need to accurately assess student accomplishment.
– Objectives describe student performance in specific, observable terms.
2. Examples3. Special situations4. Alignment
©2014 MarylandOnline
![Page 46: Applying the QM Rubric 2011-13 Rubric ©2014 MarylandOnline.](https://reader036.fdocuments.in/reader036/viewer/2022062312/551c471c550346b1458b4826/html5/thumbnails/46.jpg)
Standards 2.1 & 2.2 – Your Job
Write measurable course and module learning objectives that• precisely describe what students will gain from
instruction • guide instructors to accurately assess student
accomplishment.• are consistent
©2014 MarylandOnline
![Page 47: Applying the QM Rubric 2011-13 Rubric ©2014 MarylandOnline.](https://reader036.fdocuments.in/reader036/viewer/2022062312/551c471c550346b1458b4826/html5/thumbnails/47.jpg)
Measurable Learning Objective
Component Example
Completes this sentence: Upon completion of this course/module, students will be able to (DO SOMETHING).
At the end of this course, you will be able to:
Begins with an action verb:See: Clemson’s Bloom’s Taxonomy Action Verbs
deliver
Precisely describes behavior that can be observed or evaluated
a carefully constructed persuasive speech to an audience
©2014 MarylandOnline
![Page 48: Applying the QM Rubric 2011-13 Rubric ©2014 MarylandOnline.](https://reader036.fdocuments.in/reader036/viewer/2022062312/551c471c550346b1458b4826/html5/thumbnails/48.jpg)
Activity 7: Measurable? Precise?
1. Realize the significance of ethical behavior in the business environment.
2. Document the critical events leading to the beginning of World War I.
3. Identify the seven most serious risks associated with cigarette smoking.
4. Understand the continuing impact of World War II on the European Union.
5. Demonstrate the effects of improper blood handling in a critical care environment.
6. Describe how to create a financial statement using MS Excel.
©2014 MarylandOnline
![Page 49: Applying the QM Rubric 2011-13 Rubric ©2014 MarylandOnline.](https://reader036.fdocuments.in/reader036/viewer/2022062312/551c471c550346b1458b4826/html5/thumbnails/49.jpg)
Cassinelli, C. (2008). Bloom's Taxonomy: Original and revised. In edTEch Vision, Updated Bloom’s Taxonomy. Retrieved September 21, 2012, from http://edtechvision.org/?p=123
Bloom’s Taxonomy
©2014 MarylandOnline
![Page 50: Applying the QM Rubric 2011-13 Rubric ©2014 MarylandOnline.](https://reader036.fdocuments.in/reader036/viewer/2022062312/551c471c550346b1458b4826/html5/thumbnails/50.jpg)
Action Verbs
generate, plan, produce
judge, check, critique
differentiate, organize, attribute
execute, implement
interpret, exemplify, classify, summarize, infer, compare,
and explain.
retrieve, recognize, recall
©2014 MarylandOnline
Cassinelli, C. (2008). Bloom's Taxonomy: Original and revised. In edTEch Vision, Updated Bloom’s Taxonomy. Retrieved September 21, 2012, from http://edtechvision.org/?p=123
![Page 51: Applying the QM Rubric 2011-13 Rubric ©2014 MarylandOnline.](https://reader036.fdocuments.in/reader036/viewer/2022062312/551c471c550346b1458b4826/html5/thumbnails/51.jpg)
Mnemonic Device: Bloom’s Taxonomy
R U At An Excellent College?
©2014 MarylandOnline
![Page 52: Applying the QM Rubric 2011-13 Rubric ©2014 MarylandOnline.](https://reader036.fdocuments.in/reader036/viewer/2022062312/551c471c550346b1458b4826/html5/thumbnails/52.jpg)
Bloom’s For Students
University of Victoria Counseling Services http://www.coun.uvic.ca/learning/exams/blooms-taxonomy.html
©2014 MarylandOnline
![Page 53: Applying the QM Rubric 2011-13 Rubric ©2014 MarylandOnline.](https://reader036.fdocuments.in/reader036/viewer/2022062312/551c471c550346b1458b4826/html5/thumbnails/53.jpg)
In search of … a “precise” verb
Verbs from Bloom’s TaxonomyOld and New Version
A list of verbs that focus on performance (what the students will do)
©2014 MarylandOnline
![Page 54: Applying the QM Rubric 2011-13 Rubric ©2014 MarylandOnline.](https://reader036.fdocuments.in/reader036/viewer/2022062312/551c471c550346b1458b4826/html5/thumbnails/54.jpg)
Activity 8: Learning Objectives Worksheet
For this activity:• Find a partner• Locate the Learning Objectives Worksheet in your
packet• Review the SPCH 1113 course site• Complete the worksheet by applying Standards 2.1
through 2.5 to the SPCH 1113 course site
©2014 MarylandOnline
![Page 55: Applying the QM Rubric 2011-13 Rubric ©2014 MarylandOnline.](https://reader036.fdocuments.in/reader036/viewer/2022062312/551c471c550346b1458b4826/html5/thumbnails/55.jpg)
Practice Alignment
General Standards 2 – 6Objectives, Assessments, Materials,
Engagement, and Tools/Media
©2014 MarylandOnline
![Page 56: Applying the QM Rubric 2011-13 Rubric ©2014 MarylandOnline.](https://reader036.fdocuments.in/reader036/viewer/2022062312/551c471c550346b1458b4826/html5/thumbnails/56.jpg)
Key Sections That Must Align
©2014 MarylandOnline
![Page 57: Applying the QM Rubric 2011-13 Rubric ©2014 MarylandOnline.](https://reader036.fdocuments.in/reader036/viewer/2022062312/551c471c550346b1458b4826/html5/thumbnails/57.jpg)
Think About “Alignment”
Objectives From Intro Psychology:
Course : Summarize the relationships that exist between biology and human behavior
Module 1. Identify and define
7 major biological areas.
2. Recognize examples of how each area affect behavior.
Assignment: Read Chapter 4 (assume it deals with relationships between biology and human behavior) in text; review study guide, objective 2, page 4.
Assignment: View Chapter 4 PPT; listen to Chapter 4 podcast
Graded Assignment:Prepare table that lists the 7 major biological areas; ask students to define/describe each area and list 3 specific ways this area affects human behavior.
Graded Discussion Forum Question: Psychology is the study of behavior and mental processes. Why, then, are psychologists interested in biological matters such as the nervous system, the endocrine system and heredity?Read and respond to classmates’ postings.
Exam: 50-item multiple choice exam taken in campus Testing Center.
©2014 MarylandOnline
![Page 58: Applying the QM Rubric 2011-13 Rubric ©2014 MarylandOnline.](https://reader036.fdocuments.in/reader036/viewer/2022062312/551c471c550346b1458b4826/html5/thumbnails/58.jpg)
General Standard 3
About the General Standard
Brief Description from the Rubric
To meet the General Standard
General Standard 3,Assessment and Measurement:
Assessment strategies are designed to evaluate student progress by reference to stated learning objectives; to measure the effectiveness of student learning; and to be integral to the learning process.
Assessment is implemented in a manner that not only allows the instructor a broad perspective on the students’ mastery of the content, but also allows students to measure their own learning throughout the course.
Your assessments must align with your stated learning objectives and guide students to measure their own learning progress.
©2014 MarylandOnline
![Page 59: Applying the QM Rubric 2011-13 Rubric ©2014 MarylandOnline.](https://reader036.fdocuments.in/reader036/viewer/2022062312/551c471c550346b1458b4826/html5/thumbnails/59.jpg)
Review Standard 3.1
3.1 The types of assessments selected measure the stated learning objectives and are consistent with course activities and resources.
As a Reviewer:What do you do first? Where do you start?
(Refer to the “How to Decide” handout)
©2014 MarylandOnline
![Page 60: Applying the QM Rubric 2011-13 Rubric ©2014 MarylandOnline.](https://reader036.fdocuments.in/reader036/viewer/2022062312/551c471c550346b1458b4826/html5/thumbnails/60.jpg)
Questions to ask:1. Do the assessments
flow from the course and module/unit objectives?
2. Are they consistent with these objectives?
3. Do you (and will the students) recognize alignment?
Annotations specify: 1. Key Ideas (What to look for):
• Consistent with course and module level objectives.
• Alignment: Easy to see relationship between learning objectives and assessments. Clear that learning objectives guide students to mastery
2. Examples• Alignment • Lack of Alignment
3. Special situations: Course objectives are mandated and do not align with assessments
Standard 3.1: How to use the Annotations
©2014 MarylandOnline
![Page 61: Applying the QM Rubric 2011-13 Rubric ©2014 MarylandOnline.](https://reader036.fdocuments.in/reader036/viewer/2022062312/551c471c550346b1458b4826/html5/thumbnails/61.jpg)
Activity 9: Apply Standard 3.1
Standard 3.1 Evidence from the Course
Met or Not Met
Aligned?
The types of assessments selected measure the stated learning objectives and are consistent with course activities and resources.
©2014 MarylandOnline
![Page 62: Applying the QM Rubric 2011-13 Rubric ©2014 MarylandOnline.](https://reader036.fdocuments.in/reader036/viewer/2022062312/551c471c550346b1458b4826/html5/thumbnails/62.jpg)
General Standard 4
About the General Standard
Brief Description from the Rubric
To meet the General Standard
General Standard 4,Instructional Materials:
Instructional materials are sufficiently comprehensive to achieve stated course objectives and learning outcomes.
The instructional materials form the core of the course, and these standards respect the instructor’s prerogative in selecting them. The focus of this standard is on supporting the course objectives and competencies, rather than on qualitative judgments about the materials.
You must provide rich, robust and appropriate resources and materials that support your stated learning objectives.
©2014 MarylandOnline
![Page 63: Applying the QM Rubric 2011-13 Rubric ©2014 MarylandOnline.](https://reader036.fdocuments.in/reader036/viewer/2022062312/551c471c550346b1458b4826/html5/thumbnails/63.jpg)
Review Standard 4.1
4.1 The instructional materials contribute to the achievement of the stated course and module/unit learning objectives.
As a Reviewer:What do you do first? Where do you start?
(Refer to the “How to Decide” handout)
©2014 MarylandOnline
![Page 64: Applying the QM Rubric 2011-13 Rubric ©2014 MarylandOnline.](https://reader036.fdocuments.in/reader036/viewer/2022062312/551c471c550346b1458b4826/html5/thumbnails/64.jpg)
Questions to ask:1. What instructional
materials are used in this course?
2. Do these materials enable students to meet learning objectives?
3. Do I need to consult with the SME?
Annotations specify: 1. Key Ideas (What to look for):
The course materials and resources enable students to achieve the stated learning objectives. Consult with the team SME (subject matter expert) and use common sense. Focus only on the alignment of the instructional materials with the learning objectives rather than attempt to evaluate the content.
2. Special situations: Use module level objectives if course-level objectives are not measurable
3. Alignment: Part of alignment concept
Standard 4.1: How to use the Annotations
©2014 MarylandOnline
![Page 65: Applying the QM Rubric 2011-13 Rubric ©2014 MarylandOnline.](https://reader036.fdocuments.in/reader036/viewer/2022062312/551c471c550346b1458b4826/html5/thumbnails/65.jpg)
Activity 10: Apply Standard 4.1
Standard 4.1 Evidence from the Course
Met or Not Met
Aligned?
The instructional materials contribute to the achievement of the stated course and module/unit learning objectives.
©2014 MarylandOnline
![Page 66: Applying the QM Rubric 2011-13 Rubric ©2014 MarylandOnline.](https://reader036.fdocuments.in/reader036/viewer/2022062312/551c471c550346b1458b4826/html5/thumbnails/66.jpg)
General Standard 5
About the General Standard
Brief Description from the Rubric
To meet the General Standard
General Standard 5,Learner Interaction & Engagement:
Forms of interaction incorporated in the course motivate students and promote learning.
Engaging students to become active learners contributes to the learning process and to student persistence.
You must provide meaningful and productive activities that prompt your students to actively practice their learning.
©2014 MarylandOnline
![Page 67: Applying the QM Rubric 2011-13 Rubric ©2014 MarylandOnline.](https://reader036.fdocuments.in/reader036/viewer/2022062312/551c471c550346b1458b4826/html5/thumbnails/67.jpg)
Review Standard 5.1
5.1 The learning activities promote the achievement of the stated learning objectives.
As a Reviewer:What do you do first? Where do you start?
(Refer to the “How to Decide” handout)
©2014 MarylandOnline
![Page 68: Applying the QM Rubric 2011-13 Rubric ©2014 MarylandOnline.](https://reader036.fdocuments.in/reader036/viewer/2022062312/551c471c550346b1458b4826/html5/thumbnails/68.jpg)
Questions to ask:1. What learning activities are
used in this course?2. Do these activities enable
students to meet learning objectives?
3. Are the learning activities engaging and varied?
Annotations specify: 1. Key Ideas (What to look for):
Engage students in activities that directly contribute to the achievement of learning objectives.
2. Examples: readings, student presentations, science labs, class discussions, case studies, role playing, simulations, practice quizzes, tests, etc.
3. Special situations: blended courses4. Alignment: Part of Alignment Concept
Standard 5.1: Analyze the Standard
©2014 MarylandOnline
![Page 69: Applying the QM Rubric 2011-13 Rubric ©2014 MarylandOnline.](https://reader036.fdocuments.in/reader036/viewer/2022062312/551c471c550346b1458b4826/html5/thumbnails/69.jpg)
Activity 11: Standard 5.1
©2014 MarylandOnline
Standard 5.1 Evidence from the Course
Met or Not Met
Aligned?
The learning activities promote the achievement of the stated learning objectives.
![Page 70: Applying the QM Rubric 2011-13 Rubric ©2014 MarylandOnline.](https://reader036.fdocuments.in/reader036/viewer/2022062312/551c471c550346b1458b4826/html5/thumbnails/70.jpg)
General Standard 6
About the General Standard
Brief Description from the Rubric
To meet the General Standard
General Standard 6,Course Tools and Technology:
Course navigation and technology support student engagement and ensure access to course components.
The technology enabling the various course components facilitates the student’s learning experience and is easy to use, rather than impeding the student’s progress.
You must select the tools and media that best support your learning objectives and provide opportunities for your students to actively practice their learning.
©2014 MarylandOnline
![Page 71: Applying the QM Rubric 2011-13 Rubric ©2014 MarylandOnline.](https://reader036.fdocuments.in/reader036/viewer/2022062312/551c471c550346b1458b4826/html5/thumbnails/71.jpg)
Review Standard 6.1
6.1 The tools and media support the course learning objectives.
As a Reviewer:What do you do first? Where do you start?
(Refer to the “How to Decide” handout)
©2014 MarylandOnline
![Page 72: Applying the QM Rubric 2011-13 Rubric ©2014 MarylandOnline.](https://reader036.fdocuments.in/reader036/viewer/2022062312/551c471c550346b1458b4826/html5/thumbnails/72.jpg)
StandardQuestions to ask:1. What tools and media are
used in this course (if any)?2. Do the tools and media
selected enable students to meet objectives?
3. Is the purpose of the tools and media clear?
Annotations specify: 1. Key Ideas (What to look for): The
tools and media align with objectives of the course by supporting assessments, instructional materials, activities.
2. Examples: tools and media; not required.
3. Special situations: use module objectives if course-level not measurable
4. Alignment: Part of Alignment Concept
Standard 6.1: How to use the Annotations
©2014 MarylandOnline
![Page 73: Applying the QM Rubric 2011-13 Rubric ©2014 MarylandOnline.](https://reader036.fdocuments.in/reader036/viewer/2022062312/551c471c550346b1458b4826/html5/thumbnails/73.jpg)
Activity 12: Standard 6.1
Technology Used
Evidence From the Course
Met or Not Met
Aligned?
Course Tools
Media
©2014 MarylandOnline
![Page 74: Applying the QM Rubric 2011-13 Rubric ©2014 MarylandOnline.](https://reader036.fdocuments.in/reader036/viewer/2022062312/551c471c550346b1458b4826/html5/thumbnails/74.jpg)
Activity 13: Alignment Worksheet
• Complete the Alignment Worksheet if you haven’t already done so.
• Share and compare your alignment worksheets and any “epiphanies” you had with the group.
©2014 MarylandOnline
![Page 75: Applying the QM Rubric 2011-13 Rubric ©2014 MarylandOnline.](https://reader036.fdocuments.in/reader036/viewer/2022062312/551c471c550346b1458b4826/html5/thumbnails/75.jpg)
Write Helpful Recommendations
General Standards 7 – 8Learner Support and Accessibility
©2014 MarylandOnline
![Page 76: Applying the QM Rubric 2011-13 Rubric ©2014 MarylandOnline.](https://reader036.fdocuments.in/reader036/viewer/2022062312/551c471c550346b1458b4826/html5/thumbnails/76.jpg)
No Choice!
You MUST write a Helpful Recommendation
if you decide that the course does not meet
the specific standard.
©2014 MarylandOnline
![Page 77: Applying the QM Rubric 2011-13 Rubric ©2014 MarylandOnline.](https://reader036.fdocuments.in/reader036/viewer/2022062312/551c471c550346b1458b4826/html5/thumbnails/77.jpg)
Look at a Helpful Recommendation
©2014 MarylandOnline
![Page 78: Applying the QM Rubric 2011-13 Rubric ©2014 MarylandOnline.](https://reader036.fdocuments.in/reader036/viewer/2022062312/551c471c550346b1458b4826/html5/thumbnails/78.jpg)
Write a Useful Recommendation
Constructive Try to offer solutions, not just identify problems.
Specific Include a specific example of what is being recommended.
Measurable How will you or the ID/instructor know when the recommendation has been implemented?
Sensitive Avoid negative language. Keep recommendations and comments on a positive note.
Balanced Point out strengths as well as weaknesses.
©2014 MarylandOnline
![Page 79: Applying the QM Rubric 2011-13 Rubric ©2014 MarylandOnline.](https://reader036.fdocuments.in/reader036/viewer/2022062312/551c471c550346b1458b4826/html5/thumbnails/79.jpg)
Mnemonic Device
Can Someone Make Some Brownies?
©2014 MarylandOnline
![Page 80: Applying the QM Rubric 2011-13 Rubric ©2014 MarylandOnline.](https://reader036.fdocuments.in/reader036/viewer/2022062312/551c471c550346b1458b4826/html5/thumbnails/80.jpg)
Analyze A Recommendation
The Start Here button was a great idea, but when I read the information it contained, I still couldn’t tell exactly how to begin the course. It might be very helpful to include a prominent link and directions about what students should do next to actually begin the first lesson of the course.
Constructive
Specific
Measurable
Sensitive
Balanced
©2014 MarylandOnline
![Page 81: Applying the QM Rubric 2011-13 Rubric ©2014 MarylandOnline.](https://reader036.fdocuments.in/reader036/viewer/2022062312/551c471c550346b1458b4826/html5/thumbnails/81.jpg)
Activity 14: Improve Recommendations
For this activity you will work in groups. Each group will be assigned one of the recommendations listed below and will rewrite it using the “components” of an effective recommendation. Each group will share their recommendation.
1. Assignment instructions weren’t clear.2. You didn't tell the students how to find the
additional resources.3. The text on the page was too hard to read. 4. The learning objectives aren’t measurable.5. Your assessments are weak.©2014 MarylandOnline
![Page 82: Applying the QM Rubric 2011-13 Rubric ©2014 MarylandOnline.](https://reader036.fdocuments.in/reader036/viewer/2022062312/551c471c550346b1458b4826/html5/thumbnails/82.jpg)
General Standard 7
About the General Standard
Brief Description from the Rubric
To meet the General Standard
General Standard 7,Learner Support:
The course facilitates student access to institutional support services essential to student success.
In the learner support standard, four different kinds of support services are addressed: technical support, accessibility support, academic services support, and student services support.
You must direct your online students to the support services they need to be successful and would be available to them if they attended campus-based courses.
©2014 MarylandOnline
![Page 83: Applying the QM Rubric 2011-13 Rubric ©2014 MarylandOnline.](https://reader036.fdocuments.in/reader036/viewer/2022062312/551c471c550346b1458b4826/html5/thumbnails/83.jpg)
Review Standard 7.2
7.2 Course instructions articulate or link to the institution’s accessibility policies and services.
As a Reviewer:What do you do first? Where do you start?
Refer to the “How to Decide” handout
©2014 MarylandOnline
![Page 84: Applying the QM Rubric 2011-13 Rubric ©2014 MarylandOnline.](https://reader036.fdocuments.in/reader036/viewer/2022062312/551c471c550346b1458b4826/html5/thumbnails/84.jpg)
Questions to ask:1. Does the institution
have a stated accessibility or accommodation policy?
2. Where are they? Can a student find them?
3. What if there is no stated policy?
Annotations specify: 1. Key Ideas (What to look
for): accessibility policies, accommodations; types of and how to access services
2. Examples: link to institution’s policy; access statement
3. Special situations: no institutional policy
Standard 7.2: How to use the Annotations
©2014 MarylandOnline
![Page 85: Applying the QM Rubric 2011-13 Rubric ©2014 MarylandOnline.](https://reader036.fdocuments.in/reader036/viewer/2022062312/551c471c550346b1458b4826/html5/thumbnails/85.jpg)
Activity 15: Standard 7.2
Standard Link to the Institution’s formal policy included (evidence from the course)?
Statement to the students included?
Your Decision
Recommendations
7.2
©2014 MarylandOnline
![Page 86: Applying the QM Rubric 2011-13 Rubric ©2014 MarylandOnline.](https://reader036.fdocuments.in/reader036/viewer/2022062312/551c471c550346b1458b4826/html5/thumbnails/86.jpg)
Activity 16: Write a Recommendation
• Use the evidence you discovered in the SPCH 1113 course to write a recommendation for improving Standard 7.2
• Each team will share their recommendation and point out each component of an effective recommendation.
©2014 MarylandOnline
![Page 87: Applying the QM Rubric 2011-13 Rubric ©2014 MarylandOnline.](https://reader036.fdocuments.in/reader036/viewer/2022062312/551c471c550346b1458b4826/html5/thumbnails/87.jpg)
General Standard 8
About the General Standard
Brief Description from the Rubric
To meet the General Standard
General Standard 8,Accessibility:
The course demonstrates a commitment to accessibility for all students.
The accessibility standard incorporates the principles of Universal Design for Learning (UDL) and is consistent with Web Content Accessibility Guidelines (WCAG).
You must design your course so students with special needs can access course materials and be successful in your course.
©2014 MarylandOnline
![Page 88: Applying the QM Rubric 2011-13 Rubric ©2014 MarylandOnline.](https://reader036.fdocuments.in/reader036/viewer/2022062312/551c471c550346b1458b4826/html5/thumbnails/88.jpg)
Sites to Explore
Universal Design for Learning (UDL)Site provides “a framework for:
• designing educational environments that enable all learners to gain knowledge, skills, and enthusiasm for learning
• reducing barriers to the curriculum • providing rich supports for learning.”
Web Content Accessibility Guidelines (WCAG)Site provides documents that
• “explain how to make Web content more accessible to people with disabilities.”
©2014 MarylandOnline
![Page 89: Applying the QM Rubric 2011-13 Rubric ©2014 MarylandOnline.](https://reader036.fdocuments.in/reader036/viewer/2022062312/551c471c550346b1458b4826/html5/thumbnails/89.jpg)
Review Standard 8.1
8.1 The course employs accessible technologies and provides guidance on how to obtain accommodation.
As a Reviewer:What do you do first? Where do you start?
(Refer to the “How to Decide” handout)
©2014 MarylandOnline
![Page 90: Applying the QM Rubric 2011-13 Rubric ©2014 MarylandOnline.](https://reader036.fdocuments.in/reader036/viewer/2022062312/551c471c550346b1458b4826/html5/thumbnails/90.jpg)
Standard
Questions to ask:1. Does the course contain
a link to the LMS statement?
2. Are the other tools, content, or software accessible?
3. Do the students know how to obtain accommodation if needed?
Annotations specify: 1. Key Ideas (What to look for):
LMS accessibility statement, documentation on any content, tools, and software, information on how to obtain accommodation
2. Examples: link to the LMS accessibility statement, link to other tools, content, or software accessibility, statement from instructor
Standard 8.1: How to use the Annotations
©2014 MarylandOnline
![Page 91: Applying the QM Rubric 2011-13 Rubric ©2014 MarylandOnline.](https://reader036.fdocuments.in/reader036/viewer/2022062312/551c471c550346b1458b4826/html5/thumbnails/91.jpg)
©2014 MarylandOnline
Standard 8.1 - What to Look For
• Link to the Learning Management System Accessibility Statement• Students will be using Microsoft PowerPoint to create a
presentation:‒ Microsoft PowerPoint® will be used to create your e-portfolio. The
Microsoft Corporation has developed this software product with accessibility in mind. Product accessibility information is available on their website: http://www.microsoft.com/enable/default.aspx
• Students are asked to watch videos created by their classmates that are not closed captioned.– If you are unable to view the videos created by your classmates please
contact the Disability Services Center. The Disability Services personnel will work with you to ensure you have access to this content. The center can be reached by calling (555-555-1234) or email ([email protected]). In addition, you can review their website at: http://www.somecollege.edu/disabilityservices.
![Page 92: Applying the QM Rubric 2011-13 Rubric ©2014 MarylandOnline.](https://reader036.fdocuments.in/reader036/viewer/2022062312/551c471c550346b1458b4826/html5/thumbnails/92.jpg)
Activity 17: Standard 8.1
Standard Evidence from the Course
Decision Recommendations
8.1
©2014 MarylandOnline
![Page 93: Applying the QM Rubric 2011-13 Rubric ©2014 MarylandOnline.](https://reader036.fdocuments.in/reader036/viewer/2022062312/551c471c550346b1458b4826/html5/thumbnails/93.jpg)
Activity 18: Write a Recommendation
• Use the evidence you discovered in the SPCH 1113 course to write a recommendation for improving Standard 8.1
• Each team will share their recommendation and point out each component of an effective recommendation.
©2014 MarylandOnline
![Page 94: Applying the QM Rubric 2011-13 Rubric ©2014 MarylandOnline.](https://reader036.fdocuments.in/reader036/viewer/2022062312/551c471c550346b1458b4826/html5/thumbnails/94.jpg)
Effects of Helpful Recommendations
SPCH 1113 courseStandards 7.2 and 8.1
©2014 MarylandOnline
![Page 95: Applying the QM Rubric 2011-13 Rubric ©2014 MarylandOnline.](https://reader036.fdocuments.in/reader036/viewer/2022062312/551c471c550346b1458b4826/html5/thumbnails/95.jpg)
Activity 19: Course Improvements
• Read the decision and the recommendations from the review team for standards 7.2 and 8.1 (located in your packet)– Did you make the same decision?– What additional information did you include in your recommendations?
• Log in to the revised version of the SPCH 1113 course (login information located in your packet)– Did the recommendations from the review team help the instructor
improve the course?– What impact will the revisions have on the student’s experience?– Do you feel the revised course meets these standards?
©2014 MarylandOnline
![Page 96: Applying the QM Rubric 2011-13 Rubric ©2014 MarylandOnline.](https://reader036.fdocuments.in/reader036/viewer/2022062312/551c471c550346b1458b4826/html5/thumbnails/96.jpg)
The Peer Review Process
©2014 MarylandOnline
![Page 97: Applying the QM Rubric 2011-13 Rubric ©2014 MarylandOnline.](https://reader036.fdocuments.in/reader036/viewer/2022062312/551c471c550346b1458b4826/html5/thumbnails/97.jpg)
QM = Process and Rubric
Process Rubric
OFFICIAL Outcome: Earn QM recognitionMust follow official QM guidelines and procedures
Outcome: Improve courses, meet institutional goals, demonstrate commitment to quality
Outcome: Earn QM recognitionTool to assess online courses during formal QM review
Outcome: Improve courses, meet institutional goals
INFORMAL Outcome: Improve courses, meet institutional goals, demonstrate commitment to quality
Institutions determine use and procedures
Outcome: Improve courses, meet institutional goals, demonstrate commitment to quality
Guide to develop new online courses AND review and update online courses
©2014 MarylandOnline
![Page 98: Applying the QM Rubric 2011-13 Rubric ©2014 MarylandOnline.](https://reader036.fdocuments.in/reader036/viewer/2022062312/551c471c550346b1458b4826/html5/thumbnails/98.jpg)
Peer Course Review Process
©2014 MarylandOnline
![Page 99: Applying the QM Rubric 2011-13 Rubric ©2014 MarylandOnline.](https://reader036.fdocuments.in/reader036/viewer/2022062312/551c471c550346b1458b4826/html5/thumbnails/99.jpg)
About the Course
• QM is designed to review “mature” courses (taught at least two semesters)
• QM logo indicates year course met expectations
• Triggers for subsequent reviews:– Faculty request– More than 5 years since original review– New textbook or instructor– Professional or accreditation review pending
©2014 MarylandOnline
![Page 100: Applying the QM Rubric 2011-13 Rubric ©2014 MarylandOnline.](https://reader036.fdocuments.in/reader036/viewer/2022062312/551c471c550346b1458b4826/html5/thumbnails/100.jpg)
The Peer Review Team
• 3 faculty peer reviewers:• must be experienced online instructors• must attend QM training• one MUST be external to the course’s originating institution• there must be a subject matter expert (SME)
on the team. Note: The SME could also be the external reviewer.
AND • Faculty developer:
• access to rubric prior to review• involved in pre-review discussions• consulted during review
©2014 MarylandOnline
![Page 101: Applying the QM Rubric 2011-13 Rubric ©2014 MarylandOnline.](https://reader036.fdocuments.in/reader036/viewer/2022062312/551c471c550346b1458b4826/html5/thumbnails/101.jpg)
In an Official QM Review
• Team of 3 reviewers initially score individually
– One score per standard based on team majority
– Pre-assigned point value
– Met/Not Met decision; All/None points
– Consensus is NOT required
• To Meet Expectations…Two Thresholds– “Met” on all 21 of the 3-point “essential” standards.
– A minimum of 81 out of 95 points (81/95 = 85%)©2014 MarylandOnline
![Page 102: Applying the QM Rubric 2011-13 Rubric ©2014 MarylandOnline.](https://reader036.fdocuments.in/reader036/viewer/2022062312/551c471c550346b1458b4826/html5/thumbnails/102.jpg)
Points are NOT assigned on a sliding scale…
Standards If the standard is met …
If the standard is not met …
Essential 3 points 0 points
Very Important
2 points 0 points
Important 1 point 0 points
©2014 MarylandOnline
![Page 103: Applying the QM Rubric 2011-13 Rubric ©2014 MarylandOnline.](https://reader036.fdocuments.in/reader036/viewer/2022062312/551c471c550346b1458b4826/html5/thumbnails/103.jpg)
The 85% Rule
Two 85% Uses:
1. A minimum of 81 out of 95 points (81/95 = 85%)
2. Reviewers use their own gauge of 85% as to if a standard is met or not. Standards do not have to be 100% to be marked “Met.”
©2014 MarylandOnline
![Page 104: Applying the QM Rubric 2011-13 Rubric ©2014 MarylandOnline.](https://reader036.fdocuments.in/reader036/viewer/2022062312/551c471c550346b1458b4826/html5/thumbnails/104.jpg)
About the Review
• On average, a course review takes 7-10 hours
• Factors affecting review time include• Reviewer familiarity with the discipline• Reviewer familiarity with the LMS• Reviewer familiarity with the QM review
process• Organization of the course
©2014 MarylandOnline
![Page 105: Applying the QM Rubric 2011-13 Rubric ©2014 MarylandOnline.](https://reader036.fdocuments.in/reader036/viewer/2022062312/551c471c550346b1458b4826/html5/thumbnails/105.jpg)
Timeline for a QM Review
• Pre-Review: (1-2 weeks)– Instructor Worksheet– Conference Call– Set Team Calendar– Faculty Developer
• Active Review: (3 weeks)– Reviewers actively review course
• Post-Review: (1 week)– Have post-review discussion, if applicable
• Revisions: (14 weeks)– Course improvements made to meet standards
©2014 MarylandOnline
![Page 106: Applying the QM Rubric 2011-13 Rubric ©2014 MarylandOnline.](https://reader036.fdocuments.in/reader036/viewer/2022062312/551c471c550346b1458b4826/html5/thumbnails/106.jpg)
Formal Review Outcome
• If meets expectations:– Recognized by Quality Matters– Notifications distributed– Recognized on QM website
• If does not yet meet expectations:– Instructor (and/or ID) make changes– Team Chair/Master Reviewer approve revisions– Course meets expectations and is recognized
©2014 MarylandOnline
![Page 107: Applying the QM Rubric 2011-13 Rubric ©2014 MarylandOnline.](https://reader036.fdocuments.in/reader036/viewer/2022062312/551c471c550346b1458b4826/html5/thumbnails/107.jpg)
Self-Review Tool
• Used to do a self-review of online or blended courses• Available through MyQM >> HE tab >> My Tools >>
Course Review System• Course Review System
– Start a self-review• Lists each specific standard with links to the annotations• Allows you to make a decision and add a recommendation• Can save and return• Once all the standards are completed and saved you can “View” or
“Email” the results• Can do as many self-reviews as you want (one for every course you teach)• No one else has access to the self-review
©2014 MarylandOnline
![Page 108: Applying the QM Rubric 2011-13 Rubric ©2014 MarylandOnline.](https://reader036.fdocuments.in/reader036/viewer/2022062312/551c471c550346b1458b4826/html5/thumbnails/108.jpg)
Serving as a QM Peer Reviewer
©2014 MarylandOnline
![Page 109: Applying the QM Rubric 2011-13 Rubric ©2014 MarylandOnline.](https://reader036.fdocuments.in/reader036/viewer/2022062312/551c471c550346b1458b4826/html5/thumbnails/109.jpg)
Peer Reviewer Eligibility
You must be ELIGIBLE:• Must complete the Applying the QM
Rubric workshop.• Must complete all assessments in the
online Peer Reviewer Course– Two weeks online– Objective and written assessments– Practice Review
• Must have recent (within last 18 months) online teaching experience in a for-credit course.
How Do I Become a QM Peer Reviewer?
©2014 MarylandOnline
![Page 110: Applying the QM Rubric 2011-13 Rubric ©2014 MarylandOnline.](https://reader036.fdocuments.in/reader036/viewer/2022062312/551c471c550346b1458b4826/html5/thumbnails/110.jpg)
Reflections
©2014 MarylandOnline
![Page 111: Applying the QM Rubric 2011-13 Rubric ©2014 MarylandOnline.](https://reader036.fdocuments.in/reader036/viewer/2022062312/551c471c550346b1458b4826/html5/thumbnails/111.jpg)
Reflections
• What are the three most significant (or surprising) things you learned by participating in our APP course?
• What do you intend to do next in regards to beginning to actively use QM and the QM Rubric with your course and/or at your institution?
• Do you have plans to become a QM peer reviewer? Explain, please.
• Do you have any general questions about the course review process, the rubric, or Quality Matters?
©2014 MarylandOnline
![Page 112: Applying the QM Rubric 2011-13 Rubric ©2014 MarylandOnline.](https://reader036.fdocuments.in/reader036/viewer/2022062312/551c471c550346b1458b4826/html5/thumbnails/112.jpg)
Wrap Up and EvaluationLink to the online evaluation:
©2014 MarylandOnline
![Page 113: Applying the QM Rubric 2011-13 Rubric ©2014 MarylandOnline.](https://reader036.fdocuments.in/reader036/viewer/2022062312/551c471c550346b1458b4826/html5/thumbnails/113.jpg)
Thanks to you… Quality Matters!
For more informationvisit the QM website at:
www.qmprogram.org
©2014 MarylandOnline