Leveraging Marketing Analytics to Improve Customer Experience
Learning Analytics & Educational Research - Leveraging Big Data In Powerful Ways
Transcript of Learning Analytics & Educational Research - Leveraging Big Data In Powerful Ways
Learning Analytics & Educational Research
Leveraging Big Data in Powerful WaysAlyssa Friend Wise
Image Credit: Graham Cook via Flickr (CC BY 2.0), adapted
Associate Professor, Simon Fraser University Educational Technology & Learning Design
A F W 3 @ S F U .C A A L Y W I S E
Image Credit: Dakotilla via Flickr (CC BY 2.0)
Impassive clock! Terrifying, sinister god, Whose finger threatens us and says: Remember! …Three thousand six hundred times an hour, Second whispers: Remember!
(Baudelaire, 1857)
The clock is a powerful machinery that creates the product of “seconds” and “minutes” (Mumford, 1934)
Image Credit: Dakotilla via Flickr (CC BY 2.0)
The clock is a powerful machinery that creates the product of “seconds” and “minutes” and thus changes our relationship to time
Was a critical tool for navigation originally in calculating ship’s longitude, today for GPS
Is important in communication, scheduling connections of people within and across geographic space
Creates an awareness of allocation that can lead to greater efficiencies or value-based decision-making
(Mumford, 1934)
Time’s Quantification & Standardization
IF BEING ABLE TO TRACK (AND KEEP TRACK OF)
SOMETHING CHANGES OUR RELATIONSHIP
TO IT, WHAT IS IT THAT BIG DATA IS CHANGING OUR
RELATIONSHIP TO IN THE REALM OF EDUCATION?
Image Credit: UGA College of Agriculture and Environmental Sciences via Flickr (CC BY 2.0), adapted
Image Credit: UGA College of Agriculture and Environmental Sciences via Flickr (CC BY 2.0), adapted
BIG DATA IS CHANGING OUR RELATIONSHIP TO
THE PROCESS OF LEARNING
EPHEMERAL FLOW OF EXPERIENCE
GLOBAL “PROCESS” VARIABLES
ONLY RECENTLY HAVE TRUE PROCESS
DATA & ANALYSES BECOME
AVAILABLE
INNOVATION / CHARACTERISTIC OUTCOMESLEARNING
PROCESS
EMERGING FORMS OF PROCESS DATA IN
EDUCATION LOG-FILE DATA FROM ONLINE COURSES (INC. MOOCS)
CLICKSTREAM DATA FROM LMSS USED IN F2F COURSES
USAGE DATA FROM DIGITAL TEXTBOOKS AND NEW SPECIALIZED CONTENT BROWSERS (E.G. NSTUDY)
DATA FROM PHYSICAL SPACE (MOVEMENT, EYETRACKS…)
SELF-CONTRIBUTED DATA (POLLING, HABITS, GOALTRACKS)
VISIONS OF WHAT WE CAN DO WITH THIS
DATACONTRIBUTE TO BROAD BASED UNDERSTANDING OF LEARNING IDENTIFY GLOBAL PATTERNS ACROSS STUDENTS FIND MEANINGFUL DISTINCTIONS -> SUBSETS OF STUDENTS
INFORM US AT THE “FRONT LINES” OF HIGHER EDUCATION CREATE ACTIONABLE LOCAL INTELLIGENCE
FOR INSTRUCTORS TO PRACTICE RESPONSIVE TEACHING FOR STUDENTS TO BECOME ACTIVE AGENTS OF THEIR OWN LEARNING - “N = ME” (WINNE, IN PRESS)
L E A R N I N G A N A LY T I C S
LEARNING ANALYTICS
THE COLLECTION AND ANALYS IS OF DATA TRACES RELATED TO LEARNING IN ORDER TO INFORM AND IMPROVE THE PROCESS AND/OR ITS OUTCOMES
(S I EMENS ET AL . , 20 11)
00101110100101110101000110011010110
0101001010110100101100110010110
11010100011001010101001010011010
1011110110011011010010101001101011
001011101001011101010001100101101
0101001010110100101100110010
11010100011001010101001010
1011110110011011010010101100101010101
0010111010010111010100011
0101001010110100101100110
1101010001100101010100100
10111101100110110100101011001
001011101001011101010001100
01010010101101001011001101010
11010100011001010101001011
1011110110011011010010110
00101110100101110101000110101
0101001010110100101100110
110101000110010101010010111
1011110110011011010010101
1101010001100101010100101
001011101001011101010001100
01010010101101001011001101
11010100011001010101001010
1011110110011011010010101
0010111010010111010100011
010100101011010010110011001
1101010001100101010100101
10111101100110110100101011
10111101100110110100101011010100
00101110100101110101000110101101101
010100101011010010110011001101
1101010001100101010100101
0010111010010111010100010
1011110110011011010010101
01010010101101001011001101
0010111010010111010100011010
0010111010010111010100011
0010111010010111010100011010011010
001011101001011101010001101001001101
0101001010110100101100111
1101010001100101010100100
001011101001011101010001100
0010111010010111010100011
0101001010110100101101110001
00101110100101110101000110
0101001010110100101100110111001
0101001010110100101011001110110
11010100011001010101001010110010
110101000110010101010010110110101010
00101110100101110101000110011010110
010100101011010010110011001011
11010100011001010101001010011010
1011110110011011010010101001101011
001011101001011101010001100101101
0101001010110100101100110010
110101000110010101010010101
10111101100110110100101011010100
00101110100101110101000110101101101
010100101011010010110011001101
0010111010010111010100011010011010
001011101001011101010001101001001101
0101001010110100101100110111001
0101001010110100101011001110110
11010100011001010101001010110010
110101000110010101010010110110101010
10111101100110110100101001
001011101001011101010001100
01010010101101001011001101
11010100011001010101001010
1101010001100101010100101
10111101100110110100101011
Sub UniquePostsRead()
For k = 1 To MaxUser Step 1RowCount = Range("A1").CurrentRegion.Rows.CountFor w = 1 to MaxWeek Step 1
StartTime = Sheets("Week").Cells(w + 1, 2)EndTime = Sheets("Week").Cells(w + 1, 3) PostNum = 0PostsIndex = 0
Do While Cells(i, datestamp) <= EndTime And i <= RowCountIf Cells(i, Source) = “Read" Then
If Cells(i, Message_Author) <> Val(ActiveSheet.Name) And Cells(i, Scan) <> "X" Then
flag = 0For j = 1 To PostsIndex Step 1
If Posts(j) = Cells(i, Message_Id) Then flag = 1 j = PostsIndexEnd If
Next jIf flag = 0 Then PostsIndex = PostsIndex + 1
Posts(PostsIndex) = Cells(i, Message_Id)End If
End IfEnd If
Sheets(“Stats").Cells(Line, 22) = PostsIndexNext w
Next k
End Sub
PercentPostsRead =SUniquePostsRead TotalPostNumber
Image Credit: Joshua Rothass via Flickr (CC BY 2.0), adapted
Image Credit: US Department of Education via Flickr (CC BY 2.0), adapted
LEARNING ANALYTICS
USES COMPUTATIONAL METHODS TO GENERATE INS IGHT INTO LEARNI NG PROCESSES THAT CAN BE USED TO INFORM HUMAN-DECIS ION MAKING WHILE LEARNING EVENTS ARE ST ILL IN PROCESS
(WISE , I N PREPA RATI ON)
HOW DO WE HELP LEARNING ANALYTICS
BE AN INNOVATION THAT MAKES A REAL
IMPACT ON TEACHING AND
LEARNING ?and maybe
even
revolutionizes
higher education!
WE NEED TO DESIGN FOR WAYS IN WHICH
ANALYTICS CAN USEFULLY
REFLECT & INFORM THE TEACHING AND
LEARNING PRACTICES OF INSTRUCTORS AND
STUDENTS
PART 1: WORKING WITH INDICATORS
THAT MEANINGFUL REFLECT LEARNING
PROCESSES
HOW DO WE DEVELOP
RICH INDICATORS THAT CAN BE MEANINGFUL
TO TEACHERS AND STUDENTS AS
REFLECTIONS OF THEIR PARTICULAR
PRACTICES OF TEACHING AND
LEARNING?
WHAT’S THE LEARNING MODEL?
Image Credit: Torley via Flickr (CC BY 2.0)
MORE IS
BETTER
Image Credit: Torley via Flickr (CC BY 2.0)
WE CAN DO
BETTER!
Image Credit: Torley via Flickr (CC BY 2.0)
ONLINE DISCUSSION LEARNING MODEL
Externalizing one’s ideas by contributing
posts to an online discussion
Taking in the externalizations of others by accessing
existing posts
• Social constructivist perspective - online discussions as a forum for learning through conversation
• Students learn as they articulate their ideas, are exposed to the ideas of others, and negotiate differences in perspective
• Focus on how students contribute comments (“speak”) and attend to other’s messages (“listen”)
Speaking Mechanism for sharing ideas
Value in speaking that is Recurring, responsive , rationaled Distributed temporally and
conversationally Moderately portioned
While “speaking” is visible, not all qualities are salient in the system (esp. as related to time)
Post quality info valuable, but complex to assess
Listening Attending to the ideas of others is critical,
but “invisible”
Value in listening that is Broad yet Deep (to consider multiple
ideas; predicts posts’ content quality) Integrated (so comments are informed by
others’ views) Recurrent (to provide context for
discussion flow; predicts responsiveness)
Early research suggested universally poor behaviors, but recent work shows students listen in very distinct ways E.g. Disregardful, Coverage, Focused,
Thorough
ONLINE DISCUSSIONLEARNING MODEL
ONLINE DISCUSSIONLEARNING MODEL ANALYTICS
Criteria Metric Definition
Temporal Distribution
Percent of sessions with posts
Number of sessions in which a student made a post, divided by their total of number sessions
Speaking Quantity
Number of posts Total number of posts a student contributed to the discussion
Average post length Total number of words posted by a student divided by the number of posts they made to the discussion
Listening Breadth
Percent of posts viewed
Number of unique posts that a student viewed divided by the total number of posts in the discussion
Percent of posts read Number of unique posts that a student read divided by the total number of posts in the discussion
Listening Recurrance
Number of reviews of own / others’ posts
Number of times a student revisited posts that they had made / viewed previously in the discussion
Conversational Distribution
Posts made / viewed throughout discussion
Dispersion or concentration of posts made / viewed by a student in the discussion space
ONLINE DISCUSSIONLEARNING MODEL ANALYTICS
PART 2: USING DATA TRACES TO INFORM OUR
TEACHING AND LEARNING ACTIVITIES
HOW DO WE CONSIDER AND DESIGN FOR WAYS
IN WHICH ANALYTICS CAN PLAY A PART IN
THE LARGER
ACTIVITY PATTERNS OF
INSTRUCTORS AND STUDENTS?
A M O D E L F O R T E A C H E R S –
C O N N E C T T H E U S E O FL E A R N I N G A N A LY T I C S T O T H E P R A C T I C E O F
L E A R N I N G D E S I G N
(LOCKYER, HEATHCOTE, & DAWSON, 20 13)
KEY CONCEPTUAL QUESTIONS
1. W H AT A R E T H E G O A L S O F T H E E D U C AT I O N A L A C T I V I T Y ? [ W H AT I S T H E P O I N T ? ]
2. W H AT D O P R O D U C T I V E & U N P R O D U C T I V E L E A R N I N G P R O C E S S E S T O M E E T T H E S E G O A L S L O O K L I K E ? [ W H AT I S T H E P R O C E S S ? ]
3. H O W C A N T H E AV A I L A B L E A N A LY T I C S S E R V E A S I N D I C AT O R S O F T H E S E ?
[ W H AT I S T H E P R O X Y ? ]
• Purpose of engaging in [online discussions]
• Expectations for a productive process of engaging in [online discussions]
• How the learning analytics provide a proxy for [this]
L INKING LEARNING ANALYTICS & LEARNING DESIGN
articulating one’s ideas, being exposed to the ideas of others, negotiating differences in perspective
attending deeply to a spectrum of others’ ideas, and contributing comments that are responsive and rationaled,
percent of posts read introduced is a metric that has clear meaning in the context of the activity
Metric Student 1(Week X)
Student 2(Week X)
Class Average (Week X)
Range of participation 2 days 6 days 5 days
# of sessions 8 3 11
Average session length 13 min 48 min 39 min
% of sessions with posts 35% 67% 49%
# of posts made 9 4 7
Average post length 126 words 386 words 216 words
% of posts read 42% 87% 75%
#of reviews of own posts 2 22 13
#of reviews of others’ posts 3 12 8
ONLINE DISCUSSIONLEARNING ANALYTICS
ONLINE DISCUSSIONLEARNING ANALYTICS
A M O D E L F O R S T U D E N T S
– C O N N E C T T H E U S E O FL E A R N I N G A N A LY T I C S T O T H E P R A C T I C E S O F
S E L F - R E G U L AT E D L E A R N I N G (WISE , 2 014 )
WHY FOCUS ON STUDENTS AS USERS OF LEARNI NG ANALYTICS?
E N G A G E T H E M A S A C T I V E PA R T N E R S I N L E A R N I N G
A B I L I T Y TO M A K E I M M E D I AT E L O C A L C H A N G E S
A C T I VAT E M E TA C O G N I T I V E P R O C E S S E S
E M P O W E R M E N T N O T E N S L AV E M E N T
D E M O C R AT I Z E A C C E S S TO D ATA
O N E - TO - O N E R AT I O AT A N Y S C A L E
STUDENT TUNING MODEL
ALIGN DESIGN FRAMEWORK
Integration (technological and pedagogical) made analytics a coherent part of the learning process
Students embraced agency in setting (often recurring) personal goals and evaluating their progress, no “big brother” issues
Individual, peer, and instructor reference frames were important for making sense of the data; reactions were both cognitive and emotional
Reflection on data a powerful starting placeConcrete and proximal goal-setting is harderChange happens slowly, isn’t always intentional, requires support!
STUDENT ANALYTICS USE INIT IAL F INDINGS ( ! )
DIST INC T CHANGE PROFILES
HOW CAN YOU USE ANALYTIC DATA TO
USEFULLY REFLECT & INFORM
YOUR TEACHING AND THE
LEARNING PRACTICES OF YOUR STUDENTS
?
Image Credit: enjosmith via Flickr (CC BY 2.0), adapted
Learning Analytics & Educational Research
Leveraging Big Data in Powerful Ways
A LY S S A F R I E N D W I S ES I M O N F R A S E R U N I V E R S I T Y
A F W 3 @ S F U .C A A L Y W I S E