Learning Dispositions and Transferable Competencies: Pedagogy, Modelling and Learning Analytics

1

Simon Buckingham Shum and Ruth Deakin Crick The Open University & University of Bristol, UK @sbskmi @ruthdeakincrick / LearningEmergence.net

The story in 1 slide…

Learning dispositions matter

They can be modelled as “Learning Power”

A platform hosts apps, pools data, generates real time analytics

and manages permissions

Criteria for validating Learning Analytics — context

Ongoing/future directions

2

learning dispositions

matter

3

Why do dispositions matter?

“Knowledge of methods alone will not suffice: there must be the desire, the will, to employ them. This desire is an affair of personal disposition.”

John Dewey

4

Dewey, J. How We Think: A Restatement of the Relation of Reflective Thinking to the Educative Process. Heath and Co, Boston, 1933

Why do dispositions matter?

“educational philosophy and theory face the unfamiliar and challenging task of theorising a formative process which is not guided form the start by the target form designed in advance”

Zygmunt Bauman

5 Zygmunt Bauman, 2001: The Individualised Society, Cambridge, Polity Press.

Why do dispositions matter?

§  Widening disconnect between what engages many young people, and their experience of schooling:

§  Canadian Education Association: intellectual engagement falls during the middle school years and remains at a low level throughout secondary school

§  English Department for Education: 10% of students “hate” school, with disproportionate levels amongst less privileged learners

§  US study across 27 states: 49% students felt bored every day, 17% in every class

6

C21 competencies (we’ve all got a list) Reflects the urgent need to build learners’ capacity to cope with unprecedented uncertainty and challenge

Sensemaking Authentic purpose

Learning for teaching as designing Curriculum as narration rather than script

Learning how to learn Creativity

Entrepreneurialism Team work

Problem solving Citizenship

7

What do we mean by “disposition”?

a relatively enduring tendency to behave in a certain way

there are varying conceptions as to how fixed or malleable dispositions are

Our focus is on malleable dispositions that are important for developing intentional learners, and which,

critically, learners can recognise and develop in themselves

8

Expert  led  enquiry    

   

 Student  led-­‐enquiry      

                   

Expert  led  teaching  

                                                                                         

Student  led  revision  

know

ledg

e

co-g

ener

atio

n

and

use

Pr

e-sc

ribed

K

now

ledg

e

Teaching  as  learning  design  

Teacher  agency   Student  agency  

From transmission to learning design The Knowledge-Agency Window

Repetition, abstraction, acquisition

Authenticity, agency, identity

Learning is complex: moving between personal+ public, via different ways of knowing

Competence in the world

Knowledge skills &

understanding

Learning power

values attitudes & dispositions

Identity

story, desire, motivation,

relationships

Practical

Propositional

Presentational

Experiential

DEAKIN CRICK, R. 2012. Student Engagement: Identity, Learning Power and Enquiry - a complex systems approach. In: CHRISTENSON, S., RESCHLY, A. & WYLIE, C. (eds.) The Handbook of Research on Student Engagement New York: Springer HERON, J. & REASON, P. 1997. A Participatory Inquiry Paradigm. Qualitative Inquiry, 3, 274-294.

Public Personal

dispositions can be modelled:

“learning power”

11

Personal Development:

Values, Attitudes, Dispositions,

Identity, Story

Learning Power

Competencies and skills for employability

Knowledge, Skills and Understanding

Where did this construct come from? 3 year project to identify the most important qualities shown by effective learners, and then devise a valid assessment tool

Experts & Practitioners consulted on overall process

Meta-analysis of the literature (empirical + theoretical)

Expert Workshops (policymakers + scholars)

Leading Practitioner input to survey questions

Survey design iterations and refinement

Factor analysis on survey data (N=2000)

Seven factors identified

13

In your experience, what are the qualities shown by the most effective learners?

14

Think about the most effective learners you’ve met/mentored/taught

Not necessarily the highest grade scorers, but the ones

who went on to do well in real world learning

What qualities/dispositions/attitudes did they bring?

Tweet them now on #lak12 #dispositions

Learning to Learn: 7 Dimensions of “Learning Power”

Changing & Learning

Meaning Making

Critical Curiosity

Creativity

Learning Relationships

Strategic Awareness

Resilience

Being Stuck & Static

Data Accumulation

Passivity

Being Rule Bound

Isolation & Dependence

Being Robotic

Fragility & Dependence

Learning to Learn: 7 Dimensions of Learning Power Factor analysis of the literature plus expert interviews: identified seven dimensions of effective “learning power”, since validated empirically with learners at many levels. (Deakin Crick, Broadfoot and Claxton, 2004)

Learning to Learn: 7 Dimensions of Learning Power Factor analysis of the literature plus expert interviews: identified seven dimensions of effective “learning power”, since validated empirically with learners at many levels. (Deakin Crick, Broadfoot and Claxton, 2004)

Learning Warehouse:

analytics platform

18

Datasets: >40,000 ELLI profiles

(data from other hosted apps)

Learning Warehouse 2.0

Analytics: Real time ELLI Analytics reports

Bespoke research reports

User experience: Research-validated assessment tools

Researcher interface Learning Communities

ELLI: Effective Lifelong Learning Inventory

20 See the paper for examples of questions loading onto each dimension of Learning Power

Immediate feedback enabling timely reflection and interventions, via a mentored discussion ELLI profile showing pre/post change

21

Cohort analytics for educators and organizational leaders

22

The Learning Warehouse manages permissions for different stakeholders

§  Learners sign in to complete the right version of the ELLI questionnaire (e.g. Child or Adult) and receive their personal ELLI visual analytic

§  Administrators can upload additional learner metadata or datasets

§  Educators/organisational leaders access individual and cohort analytics, scaling to the organisation as a whole if required

§  Researchers can see all of the above, together with other datasets and institutions (depending on permissions)

23

Additional Analytics services currently provided through Vital Partnerships (social enterprise spinout from U. Bristol)

§  Bespoke organisational analyses to inform leadership §  a gender cohort in a school; a marketing department in a bank;

underachieving students; or measures of change over time in a school

§  Analysis of data across a cohort of organisations §  the impact of Learning Futures pedagogies on student engagement in

learning

§  Collaborative research data analysis service §  researchers who wish to use the instruments in their research

projects or to avail themselves of secondary datasets

§  Secondary analysis of large-scale datasets §  see paper for details of correlations found between ELLI and other

cross-dataset measures of attainment, teacher attitudes and student engagement

24

www.vitalpartnerships.com

validating learning analytics

context context context

25

System Wide (Empirical/Analytical)

When we use the same data for different purposes, we need different ‘truth paradigms’/forms of rationality (Habermas)

Group/Community (Hermeneutical)

Individual (Emancipatory) Learners

Teachers Administrators Leaders

Researchers Policymakers

…so Learning Analytics MUST be interdisciplinary + methodologically plural = painful + challenging!

Quality Criteria – how can we be confident in our measurement model?

Deakin Crick, R., Broadfoot, P. and Claxton, G. Developing an Effective Lifelong Learning Inventory: The ELLI Project. Lifelong Learning Foundation, Bristol, 2002. Deakin Crick, R. and Yu, G. The Effective Lifelong Learning Inventory (ELLI): is it valid and reliable as a measurement tool? Education Research, 50, 4, (2008), 387–402.

Empirical analytical criteria §  Reliability – are the results repeatable? §  Validity – does it measure what it says it does? §  Internal validity – do the research results mean what

they appear to? §  External validity – can the results be generalised to

other settings (ecological validity) and to other populations (population validity)?

Interpretive/Hermeneutic Criteria

§  Reliability and validity are replaced with trustworthiness.

§  Extrinsic trustworthiness: credibility, transferability, contextual transparency, verifiability.

§  Intrinsic trustworthiness: fairness, authenticity, internal ethics.

Ren, K. Could Do Better! So Why Not? An Exploration of the Learning Dispositions of Underachieving Students Doctoral Dissertation, Graduate School of Education, University of Bristol, Bristol, UK, 2010. Deakin Crick, R., Jelfs, H., Ren, K. and Symonds, J. Learning Futures. Paul Hamlyn Foundation, London, 2010.

Students, W. I. Taronga Zoo Break Out. Singleton High School Ka Wul Centre, Singleton, 2010.

Emancipatory Criteria §  Positionality – researcher declares a standpoint §  Attention to voice – who speaks for whom? §  Critical reflexivity – researcher self-awareness §  Reciprocity – trust and mutuality - dialogue §  Potential for emancipation and action §  Undistorted communication §  Substantive contribution §  Persuasiveness of discursive critique §  Participatory ethics §  Narrative evidence – aesthetic merit / impacts on

reader / communicated ‘lived experience’ Millner, N., Small, T. and Deakin Crick, R. Learning by Accident. Report No. 1, ViTaL Development and Research Programme, University of Bristol, 2006. Deakin Crick, R. and Bond, T. 'It's like a Gift: How to get a long life easier': Narratives of Personalisation, 2012. Submitted to Education Other

ongoing/future trajectories

30

Data Sets + Data Streams

Learning Warehouse 2.0 (3.0)

Learning Analytics System Simulations

Recommendation Engines

Learning App Store Learning Communities Collective Intelligence

EnquiryBlogger: ELLI Wordpress plugins

32

Ferguson, R., Buckingham Shum, S. and Deakin Crick, R.(2011). EnquiryBlogger: using widgets to support awareness and reflection in a PLE Setting. In: 1st Workshop on Awareness and Reflection in Personal Learning Environments. PLE Conference 2011, 11-13 July 2011, Southampton, UK. Eprint: http://oro.open.ac.uk/30598

ELLIment: scaling up ELLI mentoring

33 Thomas Ullman, KMi: http://people.kmi.open.ac.uk/ullmann/projects/ELLIMent

http://www.open.ac.uk/blogs/SocialLearnResearch

ELLI-based analytics for a social learning platform

Using ELLI dimensions to classify online activity

§  Time-spent in different parts of applications §  Might repeated attempts to pass an online test load onto

Resilience? §  Might the sharing of good resources from eclectic

domains load onto Meaning Making? §  Discourse analytics to detect ‘exploratory talk’ and

argumentation in online spaces §  Might the presence of questioning and challenging in

discourse load onto Critical Curiosity? §  Social network analytics

§  Might different SNA patterns in different contexts load onto Learning Relationships?

35

ELLI-based Recommendation Engine?

Could we connect learners to each other, and to educational resources, based on similar or complementary strengths on different dimensions?

However, there are many reasons why a profile may be as it is,

so beware simplistic interpretation and intervention

— ethical criteria paramount

36

To join the global community… LearningEmergence.net

37