California Educational Research Association Annual Meeting Rancho Mirage, CA – December 5, 2008...
-
Upload
julie-washington -
Category
Documents
-
view
213 -
download
0
Transcript of California Educational Research Association Annual Meeting Rancho Mirage, CA – December 5, 2008...
![Page 1: California Educational Research Association Annual Meeting Rancho Mirage, CA – December 5, 2008 Hoky Min, Gregory K. W. K. Chung, Rebecca Buschang, Lianna.](https://reader036.fdocuments.in/reader036/viewer/2022070400/56649f145503460f94c29287/html5/thumbnails/1.jpg)
California Educational Research AssociationAnnual Meeting
Rancho Mirage, CA – December 5, 2008
Hoky Min, Gregory K. W. K. Chung, Rebecca Buschang, Lianna Johnson, William Kaiser
The Influence of the Use of an Open-Ended Classroom Response System on Student Outcomes
![Page 2: California Educational Research Association Annual Meeting Rancho Mirage, CA – December 5, 2008 Hoky Min, Gregory K. W. K. Chung, Rebecca Buschang, Lianna.](https://reader036.fdocuments.in/reader036/viewer/2022070400/56649f145503460f94c29287/html5/thumbnails/2.jpg)
2 / 29
Overview of Talk
• Background of research (Greg)
• Survey constructs (Greg)
• Analyses (Hoky)
• Results (Hoky)
• Implications and next step (Greg)
![Page 3: California Educational Research Association Annual Meeting Rancho Mirage, CA – December 5, 2008 Hoky Min, Gregory K. W. K. Chung, Rebecca Buschang, Lianna.](https://reader036.fdocuments.in/reader036/viewer/2022070400/56649f145503460f94c29287/html5/thumbnails/3.jpg)
3 / 29 3
Background
• Develop a classroom response system
UCLA developed (Elec. engineering by Bill Kaiser) – 3I: Individualized, interactive instruction
Different from clickers—focus on the process of problem solving, not just the final answer
• Test the machinery
To what extent can teachers make use of real-time student responses?
How do students perceive the experience?
How does the mode affect student learning?
![Page 4: California Educational Research Association Annual Meeting Rancho Mirage, CA – December 5, 2008 Hoky Min, Gregory K. W. K. Chung, Rebecca Buschang, Lianna.](https://reader036.fdocuments.in/reader036/viewer/2022070400/56649f145503460f94c29287/html5/thumbnails/4.jpg)
4 / 29 4
Overview of Research
• Develop and validate a survey measure of students’ perceptions of processes experience with a classroom response system
Examine technical quality of measure
Examine relation between perception and outcomes
![Page 5: California Educational Research Association Annual Meeting Rancho Mirage, CA – December 5, 2008 Hoky Min, Gregory K. W. K. Chung, Rebecca Buschang, Lianna.](https://reader036.fdocuments.in/reader036/viewer/2022070400/56649f145503460f94c29287/html5/thumbnails/5.jpg)
5 / 29 5
3I—Individualized, InteractiveInstruction
• Use computers to help with immediate feedback and formative assessment
• Typical lesson
Present problem / question / prompt …
Students type their response
Teacher interprets student responses and adjusts instruction immediately—moves on, reviews, elaborates, discusses, …
![Page 6: California Educational Research Association Annual Meeting Rancho Mirage, CA – December 5, 2008 Hoky Min, Gregory K. W. K. Chung, Rebecca Buschang, Lianna.](https://reader036.fdocuments.in/reader036/viewer/2022070400/56649f145503460f94c29287/html5/thumbnails/6.jpg)
6 / 29 6
![Page 7: California Educational Research Association Annual Meeting Rancho Mirage, CA – December 5, 2008 Hoky Min, Gregory K. W. K. Chung, Rebecca Buschang, Lianna.](https://reader036.fdocuments.in/reader036/viewer/2022070400/56649f145503460f94c29287/html5/thumbnails/7.jpg)
7 / 29 7
![Page 8: California Educational Research Association Annual Meeting Rancho Mirage, CA – December 5, 2008 Hoky Min, Gregory K. W. K. Chung, Rebecca Buschang, Lianna.](https://reader036.fdocuments.in/reader036/viewer/2022070400/56649f145503460f94c29287/html5/thumbnails/8.jpg)
8 / 29
Student’s View
![Page 9: California Educational Research Association Annual Meeting Rancho Mirage, CA – December 5, 2008 Hoky Min, Gregory K. W. K. Chung, Rebecca Buschang, Lianna.](https://reader036.fdocuments.in/reader036/viewer/2022070400/56649f145503460f94c29287/html5/thumbnails/9.jpg)
9 / 29
Instructor’s View
![Page 10: California Educational Research Association Annual Meeting Rancho Mirage, CA – December 5, 2008 Hoky Min, Gregory K. W. K. Chung, Rebecca Buschang, Lianna.](https://reader036.fdocuments.in/reader036/viewer/2022070400/56649f145503460f94c29287/html5/thumbnails/10.jpg)
10 / 29
• All students in session participated, drastically improved interaction
• Clear and immediate feedback
• Rate of receiving questions and observing responses to problems is much higher than conventional sessions
• Method exceeds interactivity of one-on-one from instructor perspective
Instructor Perceptions
![Page 11: California Educational Research Association Annual Meeting Rancho Mirage, CA – December 5, 2008 Hoky Min, Gregory K. W. K. Chung, Rebecca Buschang, Lianna.](https://reader036.fdocuments.in/reader036/viewer/2022070400/56649f145503460f94c29287/html5/thumbnails/11.jpg)
11 / 29
• Interviewed students and gathered written responses during pilot tests
Learning, interaction, interest
Comfort participating
Engagement
• Developed survey items based on qualitative data
• Examine technical quality of measure and relation to student outcomes (this study)
Student Perceptions
![Page 12: California Educational Research Association Annual Meeting Rancho Mirage, CA – December 5, 2008 Hoky Min, Gregory K. W. K. Chung, Rebecca Buschang, Lianna.](https://reader036.fdocuments.in/reader036/viewer/2022070400/56649f145503460f94c29287/html5/thumbnails/12.jpg)
12 / 29
• Learning
• Interaction
• Interest
• Comfort participating
• Engagement
Scales
![Page 13: California Educational Research Association Annual Meeting Rancho Mirage, CA – December 5, 2008 Hoky Min, Gregory K. W. K. Chung, Rebecca Buschang, Lianna.](https://reader036.fdocuments.in/reader036/viewer/2022070400/56649f145503460f94c29287/html5/thumbnails/13.jpg)
13 / 29
• Undergraduate—genetics
59 students
3I used for weekly discussion sessions (9 weeks)
• Middle school—summer school remedial math
104 students (6th, 7th, 8th grade)
3I used for guided practice sessions (twice over 4 weeks)
• Minimal instructor training
Method
![Page 14: California Educational Research Association Annual Meeting Rancho Mirage, CA – December 5, 2008 Hoky Min, Gregory K. W. K. Chung, Rebecca Buschang, Lianna.](https://reader036.fdocuments.in/reader036/viewer/2022070400/56649f145503460f94c29287/html5/thumbnails/14.jpg)
Analyses and Results
![Page 15: California Educational Research Association Annual Meeting Rancho Mirage, CA – December 5, 2008 Hoky Min, Gregory K. W. K. Chung, Rebecca Buschang, Lianna.](https://reader036.fdocuments.in/reader036/viewer/2022070400/56649f145503460f94c29287/html5/thumbnails/15.jpg)
15 / 29
What We Did…
Perception on technology
Post test
Pre- test
![Page 16: California Educational Research Association Annual Meeting Rancho Mirage, CA – December 5, 2008 Hoky Min, Gregory K. W. K. Chung, Rebecca Buschang, Lianna.](https://reader036.fdocuments.in/reader036/viewer/2022070400/56649f145503460f94c29287/html5/thumbnails/16.jpg)
16 / 29
Research Questions
• To what extent does the survey measure students’ perceptions on the use of the technology in class?
• To what extent do students’ perceptions influence their class achievement?
![Page 17: California Educational Research Association Annual Meeting Rancho Mirage, CA – December 5, 2008 Hoky Min, Gregory K. W. K. Chung, Rebecca Buschang, Lianna.](https://reader036.fdocuments.in/reader036/viewer/2022070400/56649f145503460f94c29287/html5/thumbnails/17.jpg)
17 / 29
Structural Equation Modeling (SEM)
• A statistical technique that tests hypotheses, theories, and models as to relationships among variables
• Latent variables: Theoretical constructs underlying performance or scores on measures
• Observed variables: Scores or performance on measures
![Page 18: California Educational Research Association Annual Meeting Rancho Mirage, CA – December 5, 2008 Hoky Min, Gregory K. W. K. Chung, Rebecca Buschang, Lianna.](https://reader036.fdocuments.in/reader036/viewer/2022070400/56649f145503460f94c29287/html5/thumbnails/18.jpg)
18 / 29
Structural Equation Modeling (SEM)
• Measurement model (confirmatory factor analysis)
![Page 19: California Educational Research Association Annual Meeting Rancho Mirage, CA – December 5, 2008 Hoky Min, Gregory K. W. K. Chung, Rebecca Buschang, Lianna.](https://reader036.fdocuments.in/reader036/viewer/2022070400/56649f145503460f94c29287/html5/thumbnails/19.jpg)
19 / 29
Structural Equation Modeling (SEM)
• Structural regression model
![Page 20: California Educational Research Association Annual Meeting Rancho Mirage, CA – December 5, 2008 Hoky Min, Gregory K. W. K. Chung, Rebecca Buschang, Lianna.](https://reader036.fdocuments.in/reader036/viewer/2022070400/56649f145503460f94c29287/html5/thumbnails/20.jpg)
20 / 29
Measurement Model (College)
![Page 21: California Educational Research Association Annual Meeting Rancho Mirage, CA – December 5, 2008 Hoky Min, Gregory K. W. K. Chung, Rebecca Buschang, Lianna.](https://reader036.fdocuments.in/reader036/viewer/2022070400/56649f145503460f94c29287/html5/thumbnails/21.jpg)
21 / 29
Structural Regression Model(College)
• Model 1
![Page 22: California Educational Research Association Annual Meeting Rancho Mirage, CA – December 5, 2008 Hoky Min, Gregory K. W. K. Chung, Rebecca Buschang, Lianna.](https://reader036.fdocuments.in/reader036/viewer/2022070400/56649f145503460f94c29287/html5/thumbnails/22.jpg)
22 / 29
Structural Regression Model (College)
• Model 2
![Page 23: California Educational Research Association Annual Meeting Rancho Mirage, CA – December 5, 2008 Hoky Min, Gregory K. W. K. Chung, Rebecca Buschang, Lianna.](https://reader036.fdocuments.in/reader036/viewer/2022070400/56649f145503460f94c29287/html5/thumbnails/23.jpg)
23 / 29
Measurement Model (Middle School)
![Page 24: California Educational Research Association Annual Meeting Rancho Mirage, CA – December 5, 2008 Hoky Min, Gregory K. W. K. Chung, Rebecca Buschang, Lianna.](https://reader036.fdocuments.in/reader036/viewer/2022070400/56649f145503460f94c29287/html5/thumbnails/24.jpg)
24 / 29
Structural Regression Model (Middle School)
• Model 1
![Page 25: California Educational Research Association Annual Meeting Rancho Mirage, CA – December 5, 2008 Hoky Min, Gregory K. W. K. Chung, Rebecca Buschang, Lianna.](https://reader036.fdocuments.in/reader036/viewer/2022070400/56649f145503460f94c29287/html5/thumbnails/25.jpg)
25 / 29
Structural Regression Model (Middle School)
• Model 2
![Page 26: California Educational Research Association Annual Meeting Rancho Mirage, CA – December 5, 2008 Hoky Min, Gregory K. W. K. Chung, Rebecca Buschang, Lianna.](https://reader036.fdocuments.in/reader036/viewer/2022070400/56649f145503460f94c29287/html5/thumbnails/26.jpg)
26 / 29
Summary of Findings
• For college and middle-school levels, the survey measures are valid indicators of students’ perceptions of the learning processes evoked from the use of 3I
• Students’ perception does not predict class achievement
• Students’ perception and class achievement are both affected by their existing knowledge on the subjects
![Page 27: California Educational Research Association Annual Meeting Rancho Mirage, CA – December 5, 2008 Hoky Min, Gregory K. W. K. Chung, Rebecca Buschang, Lianna.](https://reader036.fdocuments.in/reader036/viewer/2022070400/56649f145503460f94c29287/html5/thumbnails/27.jpg)
27 / 29
Implications
• Why was there no relation between students’ perception of classroom processes and outcomes?
Classroom interaction doesn’t matter
Poor measure
Duration of use too short
Ceiling effect with university students
Relative coverage of content (with respect to outcome) in 3I sessions was much less than lectures
Instructor training
![Page 28: California Educational Research Association Annual Meeting Rancho Mirage, CA – December 5, 2008 Hoky Min, Gregory K. W. K. Chung, Rebecca Buschang, Lianna.](https://reader036.fdocuments.in/reader036/viewer/2022070400/56649f145503460f94c29287/html5/thumbnails/28.jpg)
28 / 29
Next Steps
• Improve instructor support
Develop structured problem sets
a priori -- Common errors , possible knowledge gaps behind errors, instructional strategies
• Experimental design
With 3I vs. without 3I (business as usual), control for content
Challenging
![Page 30: California Educational Research Association Annual Meeting Rancho Mirage, CA – December 5, 2008 Hoky Min, Gregory K. W. K. Chung, Rebecca Buschang, Lianna.](https://reader036.fdocuments.in/reader036/viewer/2022070400/56649f145503460f94c29287/html5/thumbnails/30.jpg)
30 / 29
Perceived Learning
The sessions helped to reinforce what I had learned from lectures and
the book. It was a good way to solidify any potential questions I may have had regarding specific circuits. Using the computer based tools was a nice alternative to pencil and paper or white-boarding.
I think that the answer to this question is based on the type of individual.
From my perspective, it is easier for me to take notes on problems and go over it at a later time, individually. I felt some pressure when solving the problems in a group setting.
![Page 31: California Educational Research Association Annual Meeting Rancho Mirage, CA – December 5, 2008 Hoky Min, Gregory K. W. K. Chung, Rebecca Buschang, Lianna.](https://reader036.fdocuments.in/reader036/viewer/2022070400/56649f145503460f94c29287/html5/thumbnails/31.jpg)
31 / 29
Perceived Comfort
I think that maintaining anonymity is very crucial in the interaction aspect of the
discussion. Many, including myself, may feel a little embarrassed asking a "dumb" question but w/ this method, I don't feel that people will hesitate to ask those questions.
The whole "instant messaging" system was cool, but seemed impersonal. Also, it felt
intimidating to message the professor. It seemed to make more sense if we just asked the questions in person rather than messaging.
![Page 32: California Educational Research Association Annual Meeting Rancho Mirage, CA – December 5, 2008 Hoky Min, Gregory K. W. K. Chung, Rebecca Buschang, Lianna.](https://reader036.fdocuments.in/reader036/viewer/2022070400/56649f145503460f94c29287/html5/thumbnails/32.jpg)
32 / 29
Perceived Engagement
I was definitely more prone to sit and give my full attention in this section than I am normally in any discussion. I did not fall asleep, where normally I will doze off during
normal discussion. I think it's a lot easier to pay attention because I feel like I actually have to do the problem myself, rather than sit back and let some brainy kid figure it out for me, like I will tend to do when I feel lazy normally.
Whenever we were assigned a problem to do, I always ended up taking out a piece of
paper and pencil to write out the problem. Having the problem on the computer made it harder to see the whole problem because the screen was too small to fit the problem into the screen.
![Page 33: California Educational Research Association Annual Meeting Rancho Mirage, CA – December 5, 2008 Hoky Min, Gregory K. W. K. Chung, Rebecca Buschang, Lianna.](https://reader036.fdocuments.in/reader036/viewer/2022070400/56649f145503460f94c29287/html5/thumbnails/33.jpg)
33 / 29 33
Typical Approach
• Whole-group instruction
Difficult to get immediate feedback from students
Feedback is usually only from a few students
Not all students may be engaged