CSCL Research on Online Discussions to Inform DCLA (LAK13 DCLA Workshop)
description
Transcript of CSCL Research on Online Discussions to Inform DCLA (LAK13 DCLA Workshop)
LEVERAGING CSCL RESEARCH | DCLA’13 | ALYSSA WISE | SIMON FRASER UNIVERSITY
LEVERAGING CSCL RESEARCH ANALYZING ONLINE DISCUSSIONS TO
INFORM DCLAALYSSA WISE
SIMON FRASER UNIVERSITYFACULTY OF EDUCATION
LEARNING ENVIRONMENT – ONLINE DISCUSSIONS
LEVERAGING CSCL RESEARCH | DCLA’13 | ALYSSA WISE | SIMON FRASER UNIVERSITY
ONE CENTRAL CONCERN
CONNECTING THE DEVELOPMENT AND USE OF DISCOURSE-CENTRIC LEARNING
ANALYTICS WITH THEORETICAL UNDERSTANDINGS OF THE QUALITIES OF
PRODUCTIVE ONLINE DISCOURSE
LEVERAGING CSCL RESEARCH | DCLA’13 | ALYSSA WISE | SIMON FRASER UNIVERSITY
ONE CENTRAL CONCERN
THEORETICAL MODELS OF ONLINE DISCOURSE
LEVERAGING CSCL RESEARCH | DCLA’13 | ALYSSA WISE | SIMON FRASER UNIVERSITY
TWO KEY POINTS
THERE IS AN ADDITIONAL, NOVEL KNOWLEDGE BASE NEEDED TO LEVERAGE CSCL MODELS OF DISCOURSE TO INFORM
DCLA
CREATING A CLEAR THREAD BETWEEN DISCOURSE GOALS, ACTIONS AND
ANALYTICS IS CRITICAL FOR LEARNERS (AND TEACHERS) TO USE DCLA AS A TOOL FOR SUPPORTING PRODUCTIVE DISCOURSE
LEVERAGING CSCL RESEARCH | DCLA’13 | ALYSSA WISE | SIMON FRASER UNIVERSITY
TWO KEY POINTS
DCLA > LAK + CSCL
TRANSPARENCY & INTEGRATION
LEVERAGING CSCL RESEARCH | DCLA’13 | ALYSSA WISE | SIMON FRASER UNIVERSITY
CSCL HAS MANY WELL-DEVELOPED MODELS FOR THEORIZING AND ANALYZING THE QUALITIES OF ONLINE
DISCOURSE
MOST ARE PROCESS MODELS THEORETICALLY (THOUGH ANALYSES HAVE OFTEN DISREGARDED THIS)
MOST ARE GROUP, NOT INDIVIDUAL, MODELS
THERE IS ADDITIONAL, NOVEL KNOWLEDGE NEEDED TO LEVERAGE CSCL MODELS OF DISCOURSE TO INFORM
DCLA
DCLA > LAK + CSCL
LEVERAGING CSCL RESEARCH | DCLA’13 | ALYSSA WISE | SIMON FRASER UNIVERSITY
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 201
2
3
4
5
THEORETICAL PATTERN
Sharing Information
Negotiating Meaning
Testing & Modifying
Exploring Dissonance
Agreeing & Applying
KNOWLEDGE CONSTRUCTION MODELFROM GUNAWARDENA ET AL., 1997….. TO WISE & CHIU, 2011
LEVERAGING CSCL RESEARCH | DCLA’13 | ALYSSA WISE | SIMON FRASER UNIVERSITY
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 201
2
3
4
5
ACTUAL PATTERN
(PRETTY GOOD)
Sharing Information
Negotiating Meaning
Testing & Modifying
Exploring Dissonance
Agreeing & Applying
KNOWLEDGE CONSTRUCTION MODELFROM GUNAWARDENA ET AL., 1997….. TO WISE & CHIU, 2011
LEVERAGING CSCL RESEARCH | DCLA’13 | ALYSSA WISE | SIMON FRASER UNIVERSITY
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 201
2
3
4
5
ACTUAL PATTERN
(FAILURE TO LAUNCH)
Sharing Information
Negotiating Meaning
Testing & Modifying
Exploring Dissonance
Agreeing & Applying
KNOWLEDGE CONSTRUCTION MODELFROM GUNAWARDENA ET AL., 1997….. TO WISE & CHIU, 2011
LEVERAGING CSCL RESEARCH | DCLA’13 | ALYSSA WISE | SIMON FRASER UNIVERSITY
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 201
2
3
4
5
ACTUAL PATTERN
(QUICK CONSENSUS)
Sharing Information
Negotiating Meaning
Testing & Modifying
Exploring Dissonance
Agreeing & Applying
KNOWLEDGE CONSTRUCTION MODELFROM GUNAWARDENA ET AL., 1997….. TO WISE & CHIU, 2011
LEVERAGING CSCL RESEARCH | DCLA’13 | ALYSSA WISE | SIMON FRASER UNIVERSITY
AUTOMATING POST CODING, IN REAL TIME
EVALUATING “APPROXIMATIONS” OF GOOD DISCOURSE PATTERNS (DESIRABLE ENDSTATE VS PATHS FOR CHANGE)
EVALUATING DISCOURSE PATTERNS IN PROGRESS (IS “QUALITY” OF A POST PATTERN PARTIALLY DETERMINED BY THOSE YET TO COME)
CREATING ACTIONABLE ANALYTICS (GROUP / INDIVIDUAL LEVELS, INTERPRETABILITY)
ASSESSING INDIVIDUAL’S POSTS IN CONTEXT (SEQUENTIAL ANALYSIS)
ISSUES IN DRAWING ON KC MODEL TO DEVELOP DCLA
LEVERAGING CSCL RESEARCH | DCLA’13 | ALYSSA WISE | SIMON FRASER UNIVERSITY
TRANSPARENCY & INTEGRATIONLEARNERS HAVE A VARIETY OF PERCEPTIONS ABOUT THE
PURPOSE OF ONLINE DISCOURSE (THAT OFTEN DIFFER FROM THE INTENDED PURPOSE) - HOW THEY PARTICIPATE
IN ONLINE DISCUSSIONS RELATE TO THESE PERCEIVED PURPOSES
(KNOWLTON, 2005; WISE ET AL. 2012; 2013)
LEARNERS (AND TEACHERS) NEED TO SEE THE CONNECTIONS BETWEEN THE GOALS OF DISCOURSE,
RELATED ONLINE ACTIVITY, AND ANALYTIC FEEDBACK IN ORDER TO USE DCLA AS A TOOL FOR SUPPORTING
PRODUCTIVE DIALOGUE
LEVERAGING CSCL RESEARCH | DCLA’13 | ALYSSA WISE | SIMON FRASER UNIVERSITY
TRANSPARENCY & INTEGRATION
THEORETICAL MODEL
DISCOURSE GUIDELINES
RELATED ANALYTICS
GOAL SETTING
DATA
REFLECTION
LEVERAGING CSCL RESEARCH | DCLA’13 | ALYSSA WISE | SIMON FRASER UNIVERSITY
SOCIAL CONSTRUCTIVIST PERSPECTIVE - LEARNERS BUILD UNDERSTANDING THROUGH DIALOGING WITH OTHERS
VARYING IMPORTANCE OF SHARING AND SUPPORTING ONE’S OWN IDEAS, BEING EXPOSED TO MULTIPLE VIEWPOINTS,
EXPERIENCING PERSONAL COGNITIVE CONFLICT, NEGOTIATION OF GROUP UNDERSTANDINGS
IN ALL, TWO BASIC (AND RELATED) UNDERLYING PROCESSES:
SPEAKING EXTERNALIZING ONE’S IDEAS BY CONTRIBUTING POSTS TO THE
DISCUSSION
LISTENING TAKING IN THE EXTERNALIZATIONS OF OTHERS BY ACCESSING
EXISTING POSTS
THEORETICAL MODEL
LEVERAGING CSCL RESEARCH | DCLA’13 | ALYSSA WISE | SIMON FRASER UNIVERSITY
IN ONLINE DISCUSSIONS LEARNERS HAVE A HIGH DEGREE OF CONTROL OVER THE TIMELINE AND PACE OF THEIR
ENGAGEMENT
OPPORTUNITIES FOR THOUGHTFUL LISTENING AND REFLECTIVE SPEAKING
CHALLENGES OF TIME MANAGEMENT, ESPECIALLY IN PROLIFIC DISCUSSIONS
HELPING LEARNERS TO ACTIVELY MONITOR AND REGULATE HOW THEY SPEAK AND LISTEN IN ONLINE DISCUSSIONS IS AN
IMPORTANT TOOL FOR SUPPORTING PRODUCTIVE ENGAGEMENT IN DISCUSSIONS
THEORETICAL MODEL
LEVERAGING CSCL RESEARCH | DCLA’13 | ALYSSA WISE | SIMON FRASER UNIVERSITY
SPEAKING
MECHANISM FOR SHARING IDEAS WITH OTHERS
VALUE IN SPEAKING THAT IS
- RECURRING, RESPONSIVE AND RATIONALED
- TEMPORALLY DISTRIBUTED
- MODERATE PORTIONED
WHILE “SPEAKING” IS VISIBLE, NOT ALL QUALITIES ARE SALIENT IN THE SYSTEM (ESP. AS RELATED TO TIME)
POST QUALITY INFO VALUABLE, COMPLEX TO ASSES
LISTENING
ATTENDING TO THE IDEAS OF OTHERS IS A CRITICAL, THOUGH “INVISIBLE” PART OF LEARNING THROUGH ONLINE DISCUSSIONS
VALUE IN LISTENING THAT IS
- BROAD (TO CONSIDER MULTIPLE IDEAS)
- INTEGRATED (SO REPLIES ARE INFORMED BY READS)
- REFLECTIVE (TO PROVIDE CONTEXT FOR DISCUSSION FLOW)
EARLY RESEARCH SUGGESTED UNIVERSALLY POOR BEHAVIORS, BUT OUR RECENT WORK SHOWS STUDENTS INTERACT WITH PRIOR MESSAGES IN VERY DISTINCT WAYS
- E.G. COVERAGE, INTERACTIVE, SELF-FOCUSED, TARGETED
LEVERAGING CSCL RESEARCH | DCLA’13 | ALYSSA WISE | SIMON FRASER UNIVERSITY
RELATED ANALYTICS - EMBEDDED
LEVERAGING CSCL RESEARCH | DCLA’13 | ALYSSA WISE | SIMON FRASER UNIVERSITY
Metric Definition Criteria
Range Span of days a student logged in to the discussion
Temporal Distribution
# of sessions # of times a student logged in to the discussion
Av session length Total time a student spent in the discussions divided by his / her # of sessions
% of sessions with posts
# of sessions in which a student made a post, divided by his/her total of # sessions
Posts Total # of posts a student contributed to the discussionSpeaking Quantity
Av post length Total # of words posted by a student divided by the # of posts he/she made to the discussion
% of posts read # of unique posts that a student read divided by the total # of posts made by others to the discussion
Listening Breadth
# of reviews (own posts)
# of times a student revisited posts that he/she had made previously in the discussion
Listening Reflectivity
# of reviews (others’ posts)
# of times a student revisited others’ posts that he/she had viewed previously in the discussion
RELATED ANALYTICS - EXTRACTED
LEVERAGING CSCL RESEARCH | DCLA’13 | ALYSSA WISE | SIMON FRASER UNIVERSITY
CONNECT PURPOSE OF THE LEARNING ACTIVITY WITH THE
INSTRUCTOR’S EXPECTATIONS FOR ENGAGEMENT
Discussion Participation Guidelines
Attending to Others Posts
Broad Listening: Try to read as many posts as possible to consider everyone’s ideas in the discussion. This can help you examine and support your own ideas more deeply. However, when time is limited it is better to view a portion in depth, then everything superficially.
*The visual interface shows posts that you have viewed in blue and new ones in red to help you track this.
DISCOURSE GUIDELINES
CONNECT PURPOSE OF LEARNING ACTIVITY + INSTRUCTOR’S
EXPECTATIONS WITH LEARNING ANALYTICS INDICATORSLearning Analytics Guidelines
Attending to Others’ Posts
% of posts read
The proportion of posts you read (not scanned) at least once.
It is good to read as many posts as possible to consider everyone’s ideas in the discussion However, when time is limited it is better to view a portion in depth, then everything superficially.
ANALYTICS GUIDELINES
LEVERAGING CSCL RESEARCH | DCLA’13 | ALYSSA WISE | SIMON FRASER UNIVERSITY
GOAL SETTING, DATA & REFLECTIONSUPPORTING INTENTIONAL ATTENTION TO DISCOURSE
GUIDELINES PRESENT (DIVERSE) METRICS AS A STARTING POINT FOR CONSIDERATION, NOT AS ABSOLUTE ARBITERS OF
ENGAGEMENT IN THE ACTIVITY
USE OF CLASS AVERAGE TO PROVIDE CONTEXT FOR #S
STUDENTS SET PERSONAL GOALS FOR PARTICIPATION AND USE THE ANALYTICS TO HELP MONITOR THESE REAL-TIME AND WEEKLY
REFLECTIVE DIALOGUE BETWEEN EACH STUDENT AND INSTRUCTOR ABOUT THEIR PARTICIPATION, GROUNDED IN
ANALYTICS
LEVERAGING CSCL RESEARCH | DCLA’13 | ALYSSA WISE | SIMON FRASER UNIVERSITY
INITIAL FINDINGSANALYTICS PILOTED WITH SEMESTER-LONG DOCTORAL
COURSE
STRONG STUDENT BUY-IN TO USEFULNESS OF ANALYTICS TO SUPPORT PRODUCTIVE DISCUSSION, BUT DIFFERENT STUDENTS FOUND DIFFERENT ANALYTICS USEFUL FOR
SETTING GOALS AND MONITORING PROGRESS TOWARDS THEM
NO “BIG BROTHER” ISSUES
STUDENT RECOGNITION OF WHEN THEY WERE AND WEREN’T MAKING PROGRESS ON THEIR GOALS
LEVERAGING CSCL RESEARCH | DCLA’13 | ALYSSA WISE | SIMON FRASER UNIVERSITY
ONE CONCERNTHEORETICAL MODELS OF ONLINE
DISCOURSE
TWO KEY POINTSDCLA > LAK + CSCL
TRANSPARENCY & INTEGRATION