Post on 11-Apr-2018
Measuring what
counts: Data-driven
decision making through
learning analytics to
enhance life skillsApril 2016
Mohamad Ridwan Othman
Vice President, AGILE
Taylors Education Group
Not everything that counts can be
counted, and not everything that
can be counted counts.
William Bruce Cameron, 1963
Taylor’s Education Group
2016
Garden International
School
Taylor’s International
School KL
Australian International
School Malaysia
NexusInternational
School Putrajaya & Singapore
Taylor’s International
School Puchong
Taylor’s University
Taylor’s College Subang
Jaya
Taylor’s College
Sri Hartamas
British University Vietnam
Schools Division 65 years in Schools
Higher Education Division 47 years in Higher Education
Est. 1969
Est. 2000Est. 1951 Est. 1991Est. 2008
& 2007
Est. 2014
AGILE was set up in 2015 to elevate and make consistent, the learning and teaching quality across all these
institutions and also for the learning community across the region
Our vision is to “To
Educate the Youth of the
World to Take Their
Productive Place as
Leaders in the Global
Community.”
Coined about 5 years
ago, one of the key
outcome was for Taylors
to educate for the
Head, Heart and Hand
i.e. a Holistic Education
THES2016.taylors.edu.my
Gen Z Connected Locally and Globally
• As global consumers, Gen Z’s have interconnectedness that make them demographically
homogenous. Universally, they are exposed to the same brands and marketing, as
geographical location has become irrelevant
• As they come of age, businesses, employers and even governments will want to steel
themselves for this demanding generation: They want things fast, flexible and in tune with their
beliefs on a mobile device.
General Response:
21st Century skill-sets and attributes. MOE Malaysia's 2015 Blueprint for HE
Many schools are exposing their students to overseas study trips, mission trips, internships and community projects
Social/Collaborative learning (Problem based Learning, Project-based Learning) paradigms
Students trying their hand at being entrepreneurs or taking on part-time jobs
Parents can help them cope with harsh realities by not shielding them too much
Connect with personal touch via mobile
TeacherDidactic
Communicator
Knowledge transfer
ContentTextbook
Courseware
LearnerReceptor
Past/Current
ModeContent-basedTeacher-centric
Education 1.0 Learners• Receiving• Responding• Regurgitating
Reference: https://usergeneratededucation.wordpress.com/tag/education-3-0/
TeacherFacilitator
Curator
ContentTextbook
Courseware
Learner
eContentInternet
Web
OER
Video Lectures
Emerging ModelLearning-based
Learner-centric
L2L
Learner
Education 2.0 Learners• Communicating• Connecting• CollaboratingEducation 3.0 Learners• Connectors• Creators• Constructivists
Reference: https://usergeneratededucation.wordpress.com/tag/education-3-0/
• Personal Learning
Networks
• Community of
Practice
5 Key Trends in the Work-place
New behaviors entering the workplace
being more public, sharing, collaborating, etc;
Mobility
being able to get your job done anywhere, anytime, and on any device;
Millennials
a new generation of digitally savvy and enabled workers;
Globalization
living and working in a world without boundaries;
Technology
big data, the cloud, internet of things, wearables, and the like
Source: Jacob Morgan: 5 Ways People Over 50 Can Stay Relevant in the Future of Work
Training Net-Gen Students for Jobs Yet to be Invented
Education 1.0
Education 2.0/3.0
It’s not the content but the pedagogical method
Source: Dancing with Robots: Human Skills for Computerized Workhttp://content.thirdway.org/publications/714/Dancing-With-Robots.pdf
Collaboration U: Business and University Partnerships To Secure Talent Pipelines
Employers are struggling to find college graduates with the necessary skills for their jobs
Employers are encountering a "significant and constantly growing skills gap in today's workforce" and
"there appears to be a
disconnect between higher education institutions that are preparing the next generation of workers and the employers who expect to hire them."
Author: Jenna Filipkowski, PhD
(Jenna.Filipkowski@HCI.org)
Publication date: June 30, 2015 v.2
Source: Human Capital Institute (2015), Collaboration U: Business and University Partnerships To Secure Talent Pipelines. http://www.hci.org/hr-research/collaboration-u-business-and-university-partnerships-secure-talent-pipelines
Changes in/of Learning
Innovation
Agile & responsive
Risk taking
Fail early, fail fast
Problem finding
Beyond problem solving
Sense-making
Learner via the Teacher
Lifelong Learner
Learning Organization
Learning Society
13% OF PEOPLE ARE ENGAGED IN
THEIR WORK
Source: Charlene Li, Altimeter, TED Talk 2014, Giving up control: leading in the digital era
TEG Holistic Education Framework
Attributes
Ou
tco
me
Educational Experience
(Value Proposition)
Life SkillsEmotional
Wellbeing
Academic
Excellence
Dim
en
sio
ns
Pro
ce
ss
Dimension
1. Assumed
2. Obvious
Source : Team Analysis
Co
re
Pu
rpo
se
To Educate Youth of the World to Take Their Productive Place as
Leaders in the Global Community
Dim
en
sio
ns
Academic Excellence Life Skills Emotional Wellbeing
Att
rib
ute
1. Discipline Specific Knowledge
2. Thinking & Problem Solving
3. Lifelong Learning
4. Communication Skills
5. Personal competencies (Self-awareness + Self-management – Goleman)
6. Social competencies (Social awareness + r’ship management – Goleman)
7. Entrepreneurialism (Create, Capture and Deliver Value)
8. Global perspectives
Three (3) dimensions have been identified
with eight (8) attributes supporting the
delivery of holistic education
Integration approach
Requires a holistic way to also measure the Life
Skills and Emotional Well Being portion.
Academic Excellence is easy
How do we measure, easily, to ensure accuracy,
validity and reliability?
Rubrics aiding academics and allowing for qualitative analysis would be helpful.
PROBLEM: TIME
CONSUMING and
RESOURCE
INTENSIVE
The ‘Promise’ of Learning Analytics
and Data-Driven Decision Making
“Why is it that the output of powerful learning analytics reporting
processes, acknowledged by institutional leaders as giving new insight
into organizational patterns and practices, fail to influence institutional
planning and strategic decision-making processes? ….
….this may be the result of lack of attention to institutional culture within
higher education, lack of understanding of the degree to which
individuals and cultures resist innovation and change, and lack of
understanding of approaches to motivating social and cultural change.”
Macfadyen, L. P., & Dawson, S. (2012). Numbers Are Not Enough. Why e-Learning Analytics Failed to Inform an Institutional Strategic
Plan. Educational Technology & Society, 15 (3), 149–163.
• Institutional Culture,
• Innovation and Change Resistant
• Approaches to motivate social and
cultural change
Biggest ‘Problem’ with Analytics in Education
• Great Engine but no or very little mapping
• BI force-fit into Edu Model
• Cumbersome & Difficult to administer
• Time consuming to map to:• Academic’s concerns,
• Departments concerns
• Dean’s and Edu Leaders concerns
• Not easy to integrate with other Data sources
Choice for this pilot in Taylor’s
• Great Engine but no or very little mapping
• BI force-fit into Edu Model
• Cumbersome & Difficult to administer
• Time consuming to map to:• Academic’s concerns,
• Departments concerns
• Dean’s and Edu Leaders concerns
• Not easy to integrate with other Data sources
Edu Mapping
Easily Integrated
Pedagogy Centric
Professor Peter Mortimore
Professor Knut Roald
Professor Kjell B. Hjertø
Professor Louise Stoll
Professor Andy Hargreaves
Professor Viviane Robinson
Professor Michael Fullan
Professor Lorna Earl
Professor Jan Merok Paulsen
Edu Mapping done with Academic Rigour:
Yngve Lindvig, Louise Stoll, Michael Fullan, Andy
Hargreaves, Jarl Inge Wærness, Lorna Earl and
Anders Fosnes. Toronto, 2015
Input
Parental Support
Parental Expectations
Parental socioeconomic background
Input grades
Professional qualityStudent-Teacher Relations
Formative assessment
Student Involvement
Teacher Feedback
Order in Class
Teacher Encouragement of Creativity
Career counselling
Learning environment
Social well-being
Bullying
Trust
Safety
Emotional intelligence
EQ – SEA (Self Emotion Appraisal)
EQ - OEA (Other Emotion Appraisal)
EQ – UOE (Use of Emotion)
EQ – ROE (Regulation of Emotion)
Mastery-approach goal items -MAP
Mastery-avoidance goal items – MAV
Performance-approach goal items - PAP
Performance-avoidance goal items - PAV
Collectivism
Individualism
Win-loose
Commitment
Student behaviour
Self-Assessment
Digital skills
Communication skills
Decision-making skills
Citizenship
Key capacities
Output
Intrinsic Motivation
Self-efficacy
Satisfaction
Tests and/or grades
School Attainment
ConflictsConflicts – (Emotional Relationship Conflict)
Conflicts – (Emotional Task Conflict)
Conflicts – CT (Cognitive Task Conflict)
Conflicts - (Cognitive Relationship Conflict)
Scientifically tested student survey
Why student survey?
Recent research has
shown that student
perception data are
predictive of student
achievement gains and
in many cases are more
reliable than classroom
observations and student
growth measures
(Schulz, Sud and Crowe 2014)
Students are at a good vantage point
to make judgements about classrooms
because they have encountered
many different learning environments
and have enough time in a class to
form accurate impressions. Also, even if
teachers are inconsistent in their day-to
day behaviour, they usually project a
consistent image of the long-standing
attributes of a classroom environment(Fraser 1998)
Unlike value-added data,
student survey data provide
actionable feedback that
teachers and supervisors
can use to target specific
areas for improvement
Example of Survey Questions
Self-emotion appraisal (SEA)
• I have a good sense of why I have certain feelings most of the time.
• I have good understanding of my own emotions.
• I really understand what I feel.
• I always know whether or not I am happy.
Others’ emotion appraisal (OEA)
• I always know my friends’ emotions from their behavior.
• I am a good observer of others’ emotions.
• I am sensitive to the feelings and emotions of others.
• I have good understanding of the emotions of people around me.
Use of emotion (UOE)
• I always set goals for myself and then try my best to achieve them.
• I always tell myself I am a competent person.
• I am a self-motivated person.
• I would always encourage myself to try my best.
Regulation of emotion (ROE)
• I am able to control my temper and handle difficulties rationally.
• I am quite capable of controlling my own emotions.
• I can always calm down quickly when I am very angry.
• I have good control of my own emotions.
Emotional
intelligence
Pilot Approach and Specifics
3000+ students in Taylor’s College Subang Jaya and Taylor’s College Sri Hartamas
4 key programmes
A Levels
South Australian Matriculation (SACE)
Canadian Pre-University
International Baccalaureate
Feb to April 2016 – Administer anonymous surveys via the system
April to May 2016 – Analysis of Results
Sharing with all teachers in TC
Train the teachers in slicing, filtering and managing results in the system
Presentation of Detailed Findings in THES2016, end of May 2016
Review and Fine-Tuning for roll-out in TC
Non-anonymous and Aggregrated at Programme level
Integrated with other systems such as the LMS (Student actual grades)
Possible Analysis and Data
For every factor, academics
in TC can easily plot two-
factor graphs like this, at
both the Institution Level and
even at their own
programme level. (Also by
Gender etc.)
Ability to see the different
student groups that maybe
require the different and
necessary interventions.
Possible Analysis and Data (Future)
Its also possible to
drill down to the
individual students
and give a detailed
map such as this for
every one.
Correlations
between these
data and other
data such as
Student Grades in
the SMS or LMS can
also be shown.
Using evidence and customized educational technology systematically for releasing the potential in every learner: Findings presented at ICSEI Jan 2016
1. Assemble and present data from multiple sources leads to better professional learning
2. In-depth processes with a broad evidence perspective leads to better professional learning
3. Use of digital reflection tools leads to better professional learning
4. This type of professional learning leads to better student outcome, measured with a broad set of indicators
Paper presented at the annual conference of the International Congress for School Effectiveness and Improvement, January 2016, Glasgow.
Evidence
“The dialog maps is an engaging way to inquire into our class-
climate. We are challenged to expand the way we think about learning
and well-being.”
Student
“For the first time the management was inquiring into our
professional learning activities, instead of just evaluating our
results.”
School leader
“Using the process designer with relevant prototypes led to better
prepared teachers that discuss teaching quality instead of data
quality.”
School Leader
“This is probably… No, this is for sure the most sophisticated and
elegant educational solution I’ve ever seen. And the thinking behind it is
very much in line with ours.”
North American, Deputy Education Minister
“After seeing the products and mission, we need to further develop our
strategy and change the focus of our agenda for buying tools for
schools.”
Group of Educational Directors, European Country
“These products is an excellent way to broaden the focus in education
to include well-being, 21 century skills and holistic learning.”
Asian group of Educational Directors
Delivering Holistic Edu, Acquiring of LifeSkills is
key for students now
Choosing the right tools helps us measure and fine-
tune
Conexus is one possible approach. Happy
Exploring! Do come to THES2016 for further
results
Mohamad Ridwan Othman
mohamadridwan.othman@taylors.edu.my
THES2016.taylors.edu.my