Using Clicker Technology in a Clinically-based Occupational Therapy Course: Comparing Student Perceptions of Participation and Learning

Thomas Mernar, PhD, OTR

Acknowledgements

I would like to thank

• The Office of the Provost, the Teaching, Learning, Technology Center, and Drs. Michael Vigorito and Theresa Bartolotta for the support of this program assessment proposal

• Research assistants Danielle Dorsi, OTS; Michelle Merciadez, OTS, Jennifer Ostaszewski, OTS, Janine Petito, and OTS; Rachel Sellers, OTS

Background

• Student–teacher interaction is ranked highly among the factors influencing learning performance (Bullock et al., 2002; Hake, 1998)

• Clicker technology has been changing how students and their instructor interact within the classroom by providing new opportunities to enhance in-class participation and learning in traditional, lecture hall classroom environments (Blasco-Arcas, et al., 2013).

Purpose

• Less is known about how clicker technology improves in-class participation and learning in more clinically content based courses

• The purpose of this program assessment was to examine how learning and in-class participation is effected by the use of clicker technology within a clinically based occupational therapy course

Research Design

• Nonequivalent pretest posttest control group design

Study Sample and Group Assignment

• Study sample

• 32 first year, first semester occupational therapy students

• 15 control group

• 17 experimental group

• All students enrolled into Occupational Therapy Practice Skills course; 2 sections (Monday and Thursday)

• Group assignment:

• Monday= Experimental group

• Thursday= Control group

Measurements

Pretest and Posttest ordinal data

• Student Assessment of Learning Goals (SALG)

• Self Consciousness Scale revised (SCSR)

Posttest only ordinal data

• Perceived Importance of Clicker Technology (PICT)

• Only given to experimental group at end of semester

Student Assessment of Learning Goals

• SALG instrument focuses exclusively on the degree to which a course has enabled student learning

• SALG asks students to assess and report on their own learning, and on the degree to which specific aspects of the course have contributed to that learning

• Instructor can customize questions

• The following 32 questions have been used in this program assessment: Understanding (6), Participation (5), Proficiency (8), Feelings about learning (4), Willingness to seek help (6), and Habits (3)

• Students rate questions as:

• “strongly disagree” (-2)

• “disagree” (-1)

• “agree” (1)

• “strongly agree” (2)

Self Consciousness Scale Revised

• SCSR is a 23- item questionnaire developed for the general public, which measures individual private self-consciousness, public self-consciousness, and social anxiety

• Students rate questions as:

• 3 = a lot like me

• 2= somewhat like me

• 1= a little like me

• 0= not like me at all

Perceived Importance of Clicker Technology

• Utilized SALG and reformatted all 32 questions to ask about how clickers have enhanced their understanding, participation, proficiency, feelings about learning, willingness to seek help, and habits of the content and skills learned throughout the course to create the PICT

• For example: • Under the SALG, a question would read as:

• Presently, I am… Able to demonstrate proficiency in exercising correct infection control techniques

• Under PICT, the same question would read as:

• Since using the clickers over the past few weeks, the use of clickers increased my… Proficiency in exercising correct infection control techniques

• Students rate questions as:

• “strongly disagree” (-2) • “disagree” (-1) • “agree” (1) • “strongly agree” (2)

Application of Independent Variable

• IV= use of clicker technology during class over 5 weeks during Fall 2013 semester to obtain polling of questions related to: • Assigned readings

• Content of lecture

• Past experiences related to weekly topic

• Confidence in performing lab skills

• Feelings about the class

• Classes included: • Transfer training

• Infection control

• Vital signs

• ADL Topics: Dressing, Toileting, and Bowel & Bladder

• ADL Topics: Eating & Feeding, Personal Hygiene, Bathing, and Sexuality

Which is the most common mode of infectious disease contamination?

1 2 3 4

58%

5%

32%

5%

For example

1. Contact

2. Droplet

3. Airborne

4. Common vehicle

Statistical Analyses

• All pretest and posttest questionnaires scored twice for accuracy

• Independent samples t-test used to distinguish differences in outcome scores at baseline

• Independent samples t-test used to distinguish differences in outcome scores from posttest to pretest (change scores)

Dependent Variables

Dependent Variable

Learning Gains Total Learning Gains in Understanding Learning Gains in Participation Learning Gains in Proficiency Learning Gains in Feelings Towards Learning Learning Gains in Willingness to Seek Help Learning in Habits Self-consciousness Total Private self-consciousness Public self-consciousness Social Anxiety Total Perceived importance of clicker technology *

DVs have pretest, posttest, and change score values, except for Total PICT*, as it was given only to experimental group at posttest

Descriptive Comparison of Groups at Baseline

Learning

Gains

Total

Learning

Gains in

Understan

ding

Learning

Gains in

Participation

Learning

Gains in

Proficiency

Learning

Gains in

Feelings

Learning

Gains in

Willingness

Learning

Gains in

Habits

Self-

conscious

ness Total

Private

Self-

conscious

ness

Public Self-

conscious

ness

Social

Anxiety

Mean 31.1333 10.1333 6.8667 -6.0000 6.0667 10.4667 3.6000 39.0667 19.2667 13.3333 6.4667

N 15 15 15 15 15 15 15 15 15 15 15

Std.

Deviation

8.45042 1.24595 1.50555 4.86973 2.25093 1.95911 1.29835 7.48777 1.98086 3.22195 4.98378

Mean 30.2353 9.9412 6.9412 -6.8824 6.5882 9.8824 3.7647 39.1176 18.2941 12.7647 8.0588

N 17 17 17 17 17 17 17 17 17 17 17

Std.

Deviation

9.25695 1.67595 1.81902 4.99853 1.62245 2.28808 1.75105 6.33327 3.53137 2.79574 3.94447

Mean 30.6563 10.0313 6.9063 -6.4688 6.3438 10.1563 3.6875 39.0938 18.7500 13.0313 7.3125

N 32 32 32 32 32 32 32 32 32 32 32

Std.

Deviation

8.75697 1.46979 1.65314 4.87908 1.92788 2.12678 1.53323 6.78404 2.90717 2.96740 4.46085

Descriptive Statistics of Control and Experimental Group

type of group

control group

experimental

group

Total

Inferential Comparison of Groups at Baseline

. Lower Upper

Equal variances

assumed

1.231 .276 .285 30 .777 .89804 3.14913 -5.53335 7.32943

Equal variances

not assumed

.287 29.956 .776 .89804 3.13070 -5.49610 7.29218

Equal variances

assumed

1.830 .186 .364 30 .719 .19216 .52811 -.88639 1.27070

Equal variances

not assumed

.371 29.220 .714 .19216 .51838 -.86770 1.25202

Equal variances

assumed

1.190 .284 -.125 30 .901 -.07451 .59514 -1.28995 1.14093

Equal variances

not assumed

-.127 29.895 .900 -.07451 .58800 -1.27555 1.12653

Equal variances

assumed

.002 .966 .504 30 .618 .88235 1.74956 -2.69073 4.45544

Equal variances

not assumed

.505 29.683 .617 .88235 1.74662 -2.68632 4.45102

Equal variances

assumed

1.235 .275 -.758 30 .454 -.52157 .68767 -1.92598 .88285

Equal variances

not assumed

-.743 25.153 .464 -.52157 .70187 -1.96665 .92352

Equal variances

assumed

1.104 .302 .770 30 .447 .58431 .75839 -.96453 2.13315

Equal variances

not assumed

.778 29.980 .443 .58431 .75089 -.94925 2.11787

Independent Samples Test

Levene's Test for

Equality of Variances t-test for Equality of Means

F Sig. t df

Sig. (2-

tailed)

Mean

Difference

Std. Error

Difference

95% Confidence

Interval of the

Learning Gains

Total

Learning Gains

in

Understanding

Learning Gains

in Participation

Learning Gains

in Proficiency

Learning Gains

in Feelings

Learning Gains

in Willingness

Inferential Comparison of Groups at Baseline

Equal variances

assumed

.393 .535 -.299 30 .767 -.16471 .55130 -1.29061 .96120

Equal variances

not assumed

-.304 29.196 .763 -.16471 .54106 -1.27097 .94156

Equal variances

assumed

.442 .511 -.021 30 .983 -.05098 2.44293 -5.04010 4.93814

Equal variances

not assumed

-.021 27.622 .984 -.05098 2.46925 -5.11213 5.01017

Equal variances

assumed

4.916 .034 .943 30 .353 .97255 1.03171 -1.13448 3.07958

Equal variances

not assumed

.975 25.709 .339 .97255 .99757 -1.07912 3.02422

Equal variances

assumed

1.344 .255 .535 30 .597 .56863 1.06351 -1.60335 2.74060

Equal variances

not assumed

.530 27.977 .600 .56863 1.07324 -1.62988 2.76714

Equal variances

assumed

1.468 .235 -1.008 30 .322 -1.59216 1.57984 -4.81862 1.63431

Equal variances

not assumed

-.993 26.634 .330 -1.59216 1.60346 -4.88431 1.70000

Learning Gains

in Habits

Self-

consciousness

Total

Private Self-

consciousness

Public Self-

consciousness

Social Anxiety

Discussion of Groups at Baseline

• A statistically significant difference was found between control group and experimental group at baseline for one dependent variable

• Private self-consciousness (p=0.034)

Inferential Comparison of Groups from Posttest - Pretest

.

Inferential Comparison of Groups from Posttest - Pretest Equal

variances

assumed

3.164 .085 1.316 30 .198 .86667 .65872 -.47863 2.21196

Equal

variances

not

assumed

1.355 26.968 .187 .86667 .63977 -.44610 2.17944

Equal

variances

assumed

.680 .416 -1.070 30 .293 -2.21569 2.07075 -6.44472 2.01334

Equal

variances

not

assumed

-1.060 27.860 .298 -2.21569 2.09082 -6.49950 2.06813

Equal

variances

assumed

1.273 .268 .429 30 .671 .44314 1.03361 -1.66778 2.55406

Equal

variances

not

assumed

.420 25.329 .678 .44314 1.05428 -1.72675 2.61303

Equal

variances

assumed

1.398 .246 -1.325 30 .195 -.92549 .69861 -2.35223 .50125

Equal

variances

not

assumed

-1.306 26.705 .203 -.92549 .70885 -2.38069 .52971

Equal

variances

assumed

.005 .944 -1.520 30 .139 -1.73333 1.14045 -4.06245 .59579

Equal

variances

not

assumed

-1.492 25.607 .148 -1.73333 1.16206 -4.12377 .65710

SCSRprivateSC

change

Public Self-

consciousness

Social Anxiety

Learning Gains

in Habits

Self-

consciousness

Total

Discussion of DV Group Comparisons

• A marginally statistically significant difference was found between control group and experimental group in change scores for one dependent variable

• Learning gains in habits (p=0.085)

Presently, I am in the habit of…

Connecting key ideas I learn in class with

other topics covered in this class

Strongly

Disagree

Disagree Agree Strongly

Agree

Connecting key skills I learn in class with

other skills covered in this class

Strongly

Disagree

Disagree Agree Strongly

Agree

Connecting key ideas I learn in class with

other classes in the curriculum

Strongly

Disagree

Disagree Agree Strongly

Agree

Descriptive Statistics for Perceived Importance of Clicker Technology

17 participants in experimental group scored the PICT at the end of the semester

32 questions were scored on an ordinal scale of

• “strongly disagree” (-2)

• “disagree” (-1)

• “agree” (1)

• “strongly agree” (2)

Descriptive Statistics

N Minimum Maximum Mean Std. Deviation

Perceived Importance of Clicker Technology Total Score

17 -14.00 48.00 28.4118 17.12841

Valid N (listwise) 17

Discussion of Perceived Importance of Clicker Technology

• A mean score of 28.41 was obtained from this DV

• A mean score of 31 would suggest average agreement (1=agree) on items, conversely, a mean score of -31 would suggest average disagreement (-1=disagree)

Limitations

• Effect of IV was 5 weeks, a longer duration may have yielded different results

• Some classes were out of classroom, such as the community where clickers could not be used

• Participants themselves were not randomized into groups but class section instead

• Small sample size

Conclusion

• Clicker technology was trialed in a clinical (lab) occupational therapy course

• Although statistical significance (p≤0.05) was not obtained for DVs between control and experimental groups, the experimental group rated clicker technology has having agreeable importance

• Clicker technology may be better suited for lecture style instruction compared to laboratory style instruction

• Qualitative inquiry may be useful

• Utilization of clickers was reliable due to radio frequency

• Students using clickers typically display enthusiasm during in-class polling

References

Blasco-Arcas, L., Buil, I., Hernández-Ortega, B., & Sese, F. J. (2013). Using clickers in

class: The role of interactivity, active collaborative learning and engagement in

learning performance. Computers and Education, 62, 102-110.

Bullock, D.W., LaBella, V. P., Clingan, T., Ding, Z., Stewart, G., & Thibado, P. M. (2002).

Enhancing the student-instructor interaction frequency. Physics Teacher, 40, 535–541.

Hake, R. R. (1998). Interactive-engagement versus traditional methods: A six-

thousand student survey of mechanics test data for introductory physics courses. American Journal of Physics, 66, 64–74.