UTSLTF2014 keynote - Learning.Analytics for Learning.Futures?

Learning.Analytics for Learning.Futures? Simon Buckingham Shum Director, Connected Intelligence Centre @sbuckshum #LearningAnalytics Ruth Deakin Crick UTS:CIC Visiting Professor & University of Bristol UK @ruthdeakincrick

cic.uts.edu.au

Google Docs for interactive exercises:

bit.ly/utsltf2014

cic.uts.edu.au

Learning.Futures @UTS

5 http://learningemergence.net/2014/04/07/learning-dispositions-authentic-inquiry-in-a-primary-school

cic.uts.edu.au cic.uts.edu.au

Iden%ty Forma%on

Personal Quali%es

Knowledge & Skills

Competence

6


we should be talking first about where we want to go

then we can discuss engines, performance and dashboards

7

VUCA

Volatile • Uncertain Complex • Ambiguous

9

well-defined routes mapped by others

routine performance success depends on following the rules

http

://w

ehea

rtit.c

om/e

ntry

/143

5734

8

10

well-defined routes mapped by others

routine performance success depends on following the rules

unknown territory naturally complex

each challenge unique success depends on improvisation

http

://aw

e-in

spire

d.de

vian

tart.

com

/art/

Whi

te-W

ater

-Rap

ids-

4594

7012

http

://w

ehea

rtit.c

om/e

ntry

/143

5734

8

From the known to the unknown

Unknown Strange Uncomfortable

What we know Familiar

Comfortable

From the known to the unknown



Comfortable

“liminal space”

“Liminal Space… when you have left the tried and true but have not yet been able to replace it with anything else.

Limina is the Latin word for threshold, the space betwixt and between http://sojo.net/magazine/2002/01/grieving-sacred-space

…when you are between your old comfort zone and any possible new

answer… If you are not trained in how to hold anxiety, how to live with ambiguity, how to entrust and wait, you will run…

anything to flee this terrible cloud of unknowing.”

Richard Rohr O.F.M. — on the spirituality of liminal space

So while we initially strive to make our students feel comfortable […] we then must help them balance their desire for

security with the need to take risks and explore new ideas and possibilities.

Rather than attempting to resolve the tension, a college should help

students find their place on this precarious threshold, in the liminal space between the familiar and strange, the old and the new.

Johansson & Felten, 2014

Our learners face this. So do we as staff.



Comfortable

“liminal space”

Can we equip students for liminal space?

Do we know how to teach for

liminality?

The Knowledge – Agency Window

Pre-‐scribed curriculum

Open-‐ended Enquiry

Teacher Directed Learning

Self-‐directed Learning

Teacher-‐directed enquiry

Student led-‐enquiry

Content/Expert led teaching

Student-‐led revision

cic.uts.edu.au

Pre-‐scribed curriculum

Open-‐ended Enquiry

Teacher Directed Learning

Self-‐directed Learning

Teacher-‐directed enquiry

Student led-‐enquiry

Content/Expert led teaching

Student-‐led revision

Analytics depends on what space you’re in Deakin Crick et al (2011), Learning Futures Final Report. Paul Hamlyn Foundation. http://learningemergence.net/library/reports/learning-futures-evaluation-2011

Teaching as Design

Teaching as Script

Authenticity Agency Identity


2 different worldviews (500 school teachers)

-1 -0.8 -0.6 -0.4 -0.2

0 0.2 0.4 0.6 0.8

1

Develop democratic relationships

Build a community of learners

Negotiate learning

Challenge and support learners

Teach students how to learn

Foster deep understanding

Explore the construction of

knowledge

Promote dialogue as a means of learning

Build on learners understanding

Connect learning to lives and aspirations

Apply and assess in authentic contexts

Communicate in multiple modes

Dynamic Design Script

Ruth Deakin Crick, Chris Goldspink & Margot Foster (2013). Telling Identities: Learning as Script or Design? Learning Emergence Discussion Paper (June, 2013). PDF http://learningemergence.net/events/lasi-dla-wkshp


Knowledge Agency Window exercise

24


bit.ly/utsltf2014


scaffolding open-ended enquiry

25


The big challenge for 21st century educators

“educational philosophy and theory face the unfamiliar and challenging task of theorising a formative process which is not guided from the start by the target form designed in advance”

Zygmunt Bauman

26 Zygmunt Bauman, 2001: The Individualised Society, Cambridge, Polity Press.

cic.uts.edu.au 27

Choosing carefully

Collecting thoroughly

Connecting deeply

Constructing creatively

Conveying new knowledge

28

Choosing and Deciding

Observing and Describing

Generating Questions

Uncovering Stories

Knowledge Mapping

Connecting with existing knowledge

Interface with success criteria

Assessing and Validating

Applying in the World

Using Genuine Interest to

Understand, Explore, Create and Improve BREAK-THROUGH

INNOVATION LEARNING

and PERFORMANCE

Dynamics of Open-Ended Enquiry

cic.uts.edu.au 29

Jess’s Choice

30

What will be there in 15 years?

Why do relationships matter?

What was there before?

How many people have been there?

How was the gorge made?

Have any famous people been there?

What kind of people used to be there?

‘It’s made me not so scared to learn other things,’ ‘It was a

tiny little project and it spiraled into all these other things that

were connected.’

cic.uts.edu.au

So my choice was made….. the gypsy caravan, and my friend…. I looked at the questions generated from my initial mind-map and I began to see a way of grouping them.

Anchored in personal interest…Starting with experience …..

Is there a thin line between the acceptance and rejection of gypsy travellers as a minority group within a wider

community? 34

The outcome is derived from the process

35

…I was digging deep, maybe I was experiencing a little of what we had talked

about, “theory U”. An inward bound journey leading to an outward, letting go of

preconceived ideas and experiencing change through the process.

cic.uts.edu.au

anchored in personal interest (rather than didactic intent) starts with experience (rather than opinion or theory) creates its own dynamic (rather than following taught routines) relates and unites disciplines and domains (rather than fragmenting them) Integrates values with learning (rather than separating them) is reflexive on its process (rather than concerned only with outcome) allows the outcome to derive from the process (rather than pre-determine it)

36


assessing open-ended enquiry?

37


bit.ly/utsltf2014

Equipping learners to tackle complexity

38

Competence in the world

Knowledge skills & understanding

Personal qualities, values

attitudes & dispositions

Identity

story, desire, motivation,

relationships, purpose

Deakin Crick, R. (2012). Student Engagement: Identity, Learning Power and Enquiry - a complex systems approach. in: Christenson, S., Reschly, A. & Wylie, C. (eds.) The Handbook of Research on Student Engagement. New York: Springer

Practical Propositional Presentational Experiential

Public

Personal

39

Royal Academy of Engineering, (2014) Thinking Like an Engineer, London, RAE.

….if the UK wants to produce more engineers, it needs to redesign the education system so that these ENGINEERING HABITS OF MIND become the desired outcomes of engineering education…p3


This material is based upon work supported, in whole or in part, by the U.S. Department of Defense through the Systems Engineering Research Center (SERC) under Contracts H98230-08-D-0171 and HQ0034-13-D-0004 . The

SERC is a federally funded University Affiliated Research Center (UARC) managed by Stevens Institute of Technology consisting of a collaborative network of over 20 universities. See www.SERCuarc.org.

40

Systems Engineering Research Center (SERC) Stevens Institute of Technology, New Jersey, USA.

.characteristics of effective engineers…

cic.uts.edu.au

1.  Paradoxical Mindset ― Big Picture Thinking and Attention to

Detail ―  Strategic and Tactical ―  Analytic and Synthetic ―  Courageous and Humble ―  Methodical and Creative

2.  Effective Communication ―  Modes (oral and written; good

speakers and listeners) ―  Audience (bridge between problem

domain and solution domain) ―  Content (social, managerial, technical) ―  Purpose (understanding needs,

negotiation, information brokering, technical arbitration, driving consensus)

41

3.  Flexible Comfort Zone ―  Open Minded ―  Rational Risk Taking ―  Multidisciplinary ―  Enjoys Challenges

4.  Smart Leadership ―  Quick Learning and Abstraction ―  Knowing when to stop ―  Focused on ‘Vision’ for System ―  Ability to Connect the Dots ―  Patience

5.  Self Starter ―  Curiosity _ Passionate and Motivated ―  Eager to Learn


where do learning analytics fit?

42


Academic Analytics: focus on data for senior educational leaders

Meso: institution-wide

Macro: region/state/national/international

43


Learning Analytics: Data explosion at the micro level enriching, and enriched by, the meso+macro levels

Learning Analytics UNESCO Policy Briefing — Simon Buckingham Shum

Micro: individual student progress

Meso: institution-wide

Macro: region/state/national/international

44


analytics for open-ended enquiry?

45


dispositional analytics

46


Survey-based analytics for learning dispositions

Deakin Crick et al (In Press). Developing Resilient Agency in Learning: the Crick Learning for Resilient Agency Profile.

47


Mindful Agency

Sense making

Creativity

Curiosity Belonging

Collaboration

Hope and optimism

Survey-based analytics for learning dispositions

Deakin Crick et al (In Press). Developing Resilient Agency in Learning: the Crick Learning for Resilient Agency Profile.

Rapid Visual Feedback to SImulate Self-‐Directed Change

A framework for a coaching conversaIon

48


Mindful Agency

Sense making

Creativity

Curiosity Belonging

Collaboration

Hope and optimism

Taking responsibility for my own learning over time through defining my purposes, understanding and managing my feelings, knowing how I go about learning & planning my learning journey carefully.

Making connections between what I already know & new information & experience. Making meaning by linking my story, my new learning & my purpose.

Wanting to get beneath the surface & find out more. Always wondering why and how.

1

Dependent Closed Openness to learning

An emotional orientation of being open & ready to invest in learning, having flexible self-belief, willing to persist & manage any self-doubt. A necessary pre-requisite for developing resilience in learning

Using my intuition & imagination to generate new ideas & knowledge. Taking risks & playing with ideas and artefacts to arrive at new solutions.

Being part of a learning community at work, at home, in education & in my social networks. Knowing I have social resources to draw on when I need them

Having the optimism & hope that I can learn & achieve over time. Having a growth mindset; believing I can generate my own new knowledge for what I need to achieve

Being able to work with others, to collaborate and co-generate new ideas and artefacts. Being able to listen and contribute productively to a team.


blogging + dispositional

analytics 50


Blogging for learning

Blogs offer learners opportunities to incorporate many perspectives, develop carefully crafted contributions, reflect and make considered responses to others (Ferguson et al., 2007). This medium provides an environment in which people can observe, articulate and refine practices (Efimova et al., 2004). At the same time, by making use of the comment facility, bloggers are able to share thoughts, ideas and opinions (Du and Wagner, 2005). In order for students to engage effectively with this emerging genre, they need to be able to experiment and take ownership of their writing, learning to develop a blog as a space for personal learning, reflection and interaction (Bryman and Burgess, 1994).


EnquiryBlogger (Wordpress Multisite plug-ins)


Composing and categorising a blog post

Standard blog editor, including option to embed

multimedia

Categories relating to authentic enquiry, which are visualized by

the plugins


Enquiry Spiral widget


ELLI Spider widget


Mood View widget

cic.uts.edu.au

Dashboard view


Teacher’s dashboard for EnquiryBlogger


Masters level EnquiryBloggers Graduate School of Education, University of Bristol

cic.uts.edu.au cic.uts.edu.au 64

Primary School EnquiryBloggers Bushfield School, Milton Keynes


Primary School EnquiryBloggers Bushfield School, Milton Keynes


writing analytics

66


Towards Academic Writing Analytics Relevance Understanding & Knowledge Structure & Organisation Linguistic Accuracy Illustrations Referencing

Argumentation

67


Rhetorical functions of metadiscourse identified by the Xerox Incremental Parser (XIP)

BACKGROUND KNOWLEDGE

Recent studies indicate …

… the previously proposed …

… is universally accepted ...

NOVELTY

... new insights provide direct evidence ...

... we suggest a new ... approach ...

... results define a novel role ...

OPEN QUESTION … little is known … … role … has been elusive

Current data is insufficient …

GENERALIZING ... emerging as a promising approach Our understanding ... has grown exponentially ... ... growing recognition of the importance ...

CONTRASTING IDEAS … unorthodox view resolves … paradoxes …

In contrast with previous hypotheses ...

... inconsistent with past findings ...

SIGNIFICANCE studies ... have provided important advances

Knowledge ... is crucial for ... understanding valuable information ... from studies

SURPRISE We have recently observed ... surprisingly We have identified ... unusual

The recent discovery ... suggests intriguing roles

SUMMARIZING The goal of this study ... Here, we show ...

Altogether, our results ... indicate

68


XIP rhetorical parser applied to student writing

Simsek, D., S. Buckingham Shum, Á. Sándor, A. De Liddo and R. Ferguson (2013). XIP Dashboard: Visual Analytics from Automated Rhetorical Parsing of Scientific Metadiscourse. 1st International Workshop on Discourse-Centric Learning Analytics, at 3rd International Conference on Learning Analytics & Knowledge. Leuven, BE (Apr. 8-12, 2013). http://oro.open.ac.uk/37391

69


XIP rhetorical parser applied to student writing

Simsek, D., S. Buckingham Shum, Á. Sándor, A. De Liddo and R. Ferguson (2013). XIP Dashboard: Visual Analytics from Automated Rhetorical Parsing of Scientific Metadiscourse. 1st International Workshop on Discourse-Centric Learning Analytics, at 3rd International Conference on Learning Analytics & Knowledge. Leuven, BE (Apr. 8-12, 2013). http://oro.open.ac.uk/37391

CONTRAST

SUMMARY & CONTRIBUTION

70

cic.uts.edu.au cic.uts.edu.au 71

Detailed video analysis of user experience

Buckingham Shum S.J., Uren V., Li G., Sereno B. and Mancini C. (2007). Modelling Naturalistic Argumentation in Research Literatures: Representation and Interaction Design Issues. Int. Jnl. Intelligent Systems, Special Issue on Computational Models of Natural Argument, Eds: C. Reed and F. Grasso, 22: 17-47. Sereno B., Buckingham Shum S. and Motta E. (2005). ClaimSpotter: an environment to support sensemaking with knowledge triples. Proceedings of the 10th international conference on Intelligent user interfaces. San Diego, California, USA: ACM, 199-206.

•  how users engage with a text annotated by a rhetorical parser

•  whether users can annotate a text semantically


Mindful Agency

Sense making

Creativity

Curiosity Belonging

Collaboration

Hope and optimism

Behavioural analytics for learning dispositions?

Social network patterns, teamwork effectiveness and

initiation of relationships?

Questioning, arguing and search behaviours reveal

intrinsic curiosity and epistemic commitments?

Tagging/sharing/blogging/social patterns reveal how you see connections between ideas?

72


dispositional analytics exercise

73


bit.ly/utsltf2014


mapping our learning@UTS

analytics for organisational learning

74


1. Build a picture of the target problem and ‘what works’ — what’s known about solutions

Coordinating Maps

(of the system, problem, and

strategies)

75


Examples of coordinating Maps to guide a systemic improvement community

http://www.carnegiefoundation.org/elibrary/getting-ideas-action-building-networked-improvement-communities-in-education

A map of the “Challenge Space” for student mathematics

http://www.carnegiealphalabs.org/persistence/

A “Driver Diagram” summarising what is known about Productive Persistence in mathematics

76


2. Coordinate to test solutions in different contexts in UTS

Coordinating Maps


strategies)

UTS learning context UTS learning

context UTS learning context

77


3. Pool the Data (quant. + qual.)

Data & Evidence

Hub Coordinating Maps


strategies)



78


An evidence hub shows who is working on which parts of the problem

People / Organizations / Projects / Claims / Evidence

Evidence Hub for Research by Children & Young People: http://rcyp.evidence-hub.net

evidence-hub.net

79


4. Analyse the data

Data & Evidence

Hub

Human + Machine Analytics

Coordinating Maps


strategies)



80


5. interpret, narrate, debate the meaning

Data & Evidence

Hub

Collective Sensemaking

Coordinating Maps


strategies)




81


6. Share insights in Hub, update your maps, and repeat…

Data & Evidence

Hub

Collective Sensemaking

Coordinating Maps


strategies)




82


Evidence Hub: indexing stories + evidence

Systems Learning & Leadership Evidence Hub: http://sysll.evidence-hub.net

A wizard guides the user through the submission of a structured story: •  What’s the Issue? •  What claim are you making/

addressing? •  What kind of evidence

supports/challenges this? •  Link it to papers/data •  Index it against the Hub’s

core themes

83


…which generates a knowledge tree http://learningemergence.net/2013/07/17/deed-elli-ai-ci-systemic-school-learning

Issue

Potential Solution

Supporting Evidence (practitioner story)

84


Impact Map: how much evidence is there to support each hypothesis?

http://oermap.org/hypothesis/578/hypothesis-a-performance/ 85


exercise on improvement cycle

86


bit.ly/utsltf2014


summary use of analytics is shaped by

pedagogy, epistemology, assessment regime

87


analytics for the assessment of authentic enquiry / C21 skills

are a frontier challenge ! formative, personalised

feedback to learners

88


to learn more…

89


Designing holistic analytics for 21st century learners?

http://reinventors.net/roundtables/new-metrics

90


LearningEmergence.net http://learningemergence.net

91

cic.uts.edu.au

ALASI 2014 Australian Learning Analytics Summer Institute Thurs 20th–Fri 21st Nov, 2014, Sydney alasi2014.uts.edu.au twitter: #alasi2014

In association with Hosted by


cic.uts.edu.au CIC events coming up… Nov 14, 10am-2pm: Assessing Learning Dispositions using CLARA (Ruth Deakin Crick) Nov 17, 9.30-10.30am: Open Digital Badges (Dan Hickey, Univ. Indiana) Nov 20-21: Australian Learning Analytics Summer Institute

UTSLTF2014 keynote - Learning.Analytics for Learning.Futures?

Education

Transcript of UTSLTF2014 keynote - Learning.Analytics for Learning.Futures?