onlinetechnologyEDUC5103G

in education7th Adobe Connect Session Agenda:Wednesday, March 30, 2016

• Check-In• Types of Surveys Most Used by Educators• Break-Out 1: Student Satisfaction Surveys• Data Analysis in Instructional Design• Break• Pre-Test / Post-Test Analyses• Competency / Aptitude Profiles• Break-Out 2: Best Practices & Pitfalls• Ethical Standards• Common Survey Analysis Tools• Check-Out• To Do List and Course Wrap-Up

checkin

surveyresultsin instructional design

Three types of survey data most likely to be used by educators and instructional designers:

1. Student Satisfaction Surveys2. Pre-Test / Post-Test Results3. Competency / Aptitude Profiles

flippedclassstudent satisfaction surveys

activity

Six Critical Steps in Conducting Student Satisfaction Assessmentshttps://youtu.be/d14AZE61_EE

Ways to Guide Strategic Planning https://youtu.be/ns7ekUDVe1Q

Ways to Impact Student Retentionhttps://youtu.be/HBtuYScGEcQ

Ways to Use Student Satisfaction Results for Accreditationhttps://youtu.be/9aECPrIDaDw

openingmindmup

openingmindmup

Group Instructions:Watch the Video (can be opened directly from MindMup)

Choose a group recorder and reporter (can be the same person)

As a group, use MindMup to:

• Create a summary of the key points

• How are learner satisfaction surveys being used at your institution?

o Institutional Level?

o Teacher / Classroom Level?

o Course Level?

• How would you use the video, and the key points, to inform your own practice as an educator / instructional designer?

• Describe how you would use the video, and the key points, to frame and plan a professional development session for teachers / trainers at your institution

• Report your summaries to the class

flippedclassstudent satisfaction surveys

activity

mindmuplinksGroup 1:https://drive.google.com/file/d/0B9ufnp3wjEnbSlpJZGloc3Q0Rms/view?usp=sharing

Group 2:https://drive.google.com/file/d/0B9ufnp3wjEnbUFhrTjFGOTRHaXM/view?usp=sharing

Group 3:https://drive.google.com/file/d/0B9ufnp3wjEnbOEtyTGl3azQ0MEk/view?usp=sharing

Group 4:https://drive.google.com/file/d/0B9ufnp3wjEnbVHB0aS1lZ2ZMS0E/view?usp=sharing

surveyresultsin instructional design

February 1st

March 1st

February 1st

March 1st

surveyresultsin instructional design

URGENT:Things that NEED to be

Addressed NOW

NICE:Things that CAN be Addressed

NOW

FUTURE:Things that CANNOT be

Addressed NOW

NOT POSSIBLE:Things that CANNOT, SHOULD

NOT, or DO NOT NEED to be

changed

More clarity on

instructions / due

dates, etc

Add video intros

and tips for

specific

assignments and

activities

Offer learners

opportunity to

change groups and

/ or topics for

presentations

Add a video on

using the Zotero

Tool

Adding

gamification or

games based

learning to the

topics list

Add a course

calendar to the

website

Utilize features of

Blackboard?

Adding wearables,

AR and VR to topics

list

Adding data

analytics to topics

list

Utilize features of

Blackboard?

surveyresultsqualitative analysis

http://dfreelon.org/utils/recalfront/recal2/

breaktime

return at 7:45 pm

surveyresultscause & effect

Correlation ≠ Causation

surveyresultspre-test / post-test analysis

surveyresultspre-test / post-test analysis

Maturation Effect

surveyresultspre-test / post-test analysis

Comparison to a Control GroupNecessary to demonstrate a causal relationship

Experimental Design and Survey Data

surveyresultspre-test / post-test analysis

What are the chances that these changes

were random?

Less than 5%= Statistically Significant

surveyresultspre-test / post-test analysis

(mTSES2 – mTSES1) – (TSES2 – TSES1) = Net Change(Intervention Effect)

Pre-Test:Self-Efficacy with mLearning

Post-Test:Self-Efficacy with mLearning

Pre-Test:Self-Efficacy with Teaching

Post-Test:Self-Efficacy with Teaching

Setting a control within the same group is possible – with careful instrument design

(Power, 2015)

surveyresultspre-test / post-test analysis

0.19

0.31

0.01

0.57

0.68

0.11

0.57 0.68

0.10

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

Student Engagement Instructional Strategies Classroom Management

Overall Trends in Self-Efficacy Scores(Power, 2015)

TSES mTSES Net Change

(mTSES2 – mTSES1) – (TSES2 – TSES1) = Net Change(Intervention Effect)

surveyresultscompetency profiles

Visual Auditory

Reading & Writing

Kinesthetic

Learning Styles Preferences

IQ

0.19

0.31

0.01

0.570.68

0.11

0.57 0.68

0.10

0

0.2

0.4

0.6

0.8

Student Engagement InstructionalStrategies

ClassroomManagement

Overall Trends in Self-Efficacy Scores

(Power, 2015)

TSES mTSES Net Change

0.51

0.64

0.49

0.64 0.62

0.45

0.12-0.01

-0.04-0.10

0.00

0.10

0.20

0.30

0.40

0.50

0.60

0.70

Student Engagement InstructionalStrategies

ClassroomManagement

Overall Trends in Self-Efficacy Scores

(ID4ML)

TSES (ID4ML) mTSES (ID4ML) Net Change (ID4ML)

More info:https://youtu.be/JN4K8XevefM

flippedclassbest practices & pitfalls

activity

7 tips for good survey questionshttps://youtu.be/Iq_fhTuY1hw

Confirmation Biashttps://youtu.be/B_YkdMwEO5U

How Ice Cream Kills! Correlation vs. Causation https://youtu.be/VMUQSMFGBDo

Types of Survey Sampling Techniqueshttps://youtu.be/rASK8PpqakM

Group Instructions:Watch the Video (can be opened directly from MindMup)

Choose a group recorder and reporter (can be the same person)

As a group, use MindMup to:

• Create a summary of the key points

• How would you use the video, and the key points, to inform your own practice as an educator / instructional designer?

• Describe how you would use the video, and the key points, to frame and plan a professional development session for teachers / trainers at your institution

• Report your summaries to the class

flippedclassbest practices & pitfalls

activity

mindmuplinksGroup 1:https://drive.google.com/file/d/0B9ufnp3wjEnbbUoxOThZYk0xQU0/view?usp=sharing

Group 2:https://drive.google.com/file/d/0B9ufnp3wjEnbSkdhaWNlUThjVFE/view?usp=sharing

Group 3:https://drive.google.com/file/d/0B9ufnp3wjEnbbjlvd1ZMWGVrM28/view?usp=sharing

Group 4:https://drive.google.com/file/d/0B9ufnp3wjEnbMFM3RkNtLVdEcFE/view?usp=sharing

https://youtu.be/9pCfq5QH_v4

surveyresultsresearch ethics

Canadian Tri-Council Policy Statement (TPS-2)

http://www.pre.ethics.gc.ca/eng/policy-politique/initiatives/tcps2-eptc2/Default/

UOIT Research Serviceshttps://research.uoit.ca/faculty/ethics-

safety-research-conduct/index.php

surveyresultscommon tools

Quantitative Tools: Statistics Package for the Social Sciences (SPSS)http://www-01.ibm.com/software/analytics/spss/

surveyresultscommon tools

Qualitative Tools: NVivo

surveyresultscommon tools

Quantitative & Qualitative Tools: Excel (Spreadsheet Applications)

moreresourcesVideos:

Confirmation Bias - https://youtu.be/B_YkdMwEO5U

How Ice Cream Kills! Correlation vs. Causation - https://youtu.be/VMUQSMFGBDo

Research Ethics - https://youtu.be/9pCfq5QH_v4

Six Critical Steps in Conducting Student Satisfaction Assessments - https://youtu.be/d14AZE61_EE

7 tips for good survey questions - https://youtu.be/Iq_fhTuY1hw

Types of Survey Sampling Techniques - https://youtu.be/rASK8PpqakM

Using the mTSES to Evaluate and Optimize mLearning Professional Development -https://youtu.be/JN4K8XevefM

Ways to Guide Strategic Planning with Student Satisfaction Survey Results -https://youtu.be/ns7ekUDVe1Q

Ways to Impact Student Retention with Satisfaction Survey Results - https://youtu.be/HBtuYScGEcQ

Ways to Use Student Satisfaction Results for Accreditation - https://youtu.be/9aECPrIDaDw

moreresourcesOther Resources:Freelon, D. (2011). ReCal2: Reliability for 2 coders. Retrieved from http://dfreelon.org/utils/recalfront/recal2/

Government of Canada (2015). Tri-Council Policy Statement: Ethical Conduct for Research Involving Humans (TPS-2 2014). Available from http://www.pre.ethics.gc.ca/eng/policy-politique/initiatives/tcps2-eptc2/Default/

IMB (n.d.). SPSS Software. Available from http://www-01.ibm.com/software/analytics/spss/

Power, R., Cristol, D., Gimbert, B., Bartoletti, R, & Kilgore, W. (in press). Using the mTSES to evaluate and optimize mLearning professional development. The International Review of Research in Open and Distributed Learning.

Power, R. (2015). A framework for promoting teacher self-efficacy with mobile reusable learning objects (Doctoral dissertation, Athabasca University). Available from http://hdl.handle.net/10791/63

QSR International (n.d.). Nvivo: The #1 software for qualitative data analysis. Available from http://www.qsrinternational.com/product

University of Ontario Institute of Technology (2016). Ethics, Safety and Research Conduct. Available from https://research.uoit.ca/faculty/ethics-safety-research-conduct/index.php

checkout

wrappingupTo Do List:

• Submit Assignment 3: Final Reflection Paper to robpower@hotmail.com by April 1, 2016

• Have a great break!