Student engagement in teaching, learning and assessment

@mikehamlyn

Image: world.edu

OutlineIndividual engagement

retention and successa sense of belonging

Benefits of engagementfor the individualand society

Engagement OutcomesModule and award levelPublic informatiionLeague Tables

What can we do with the dataportfolio performancehow to get “smart”how to improve outcomes from better engagement

The futureHow will we carry on?

Is this engagement?How do you know?

Does this reflect reality?

What will engaged students look like in the future?

Individual Engagement

• Student Retention and Success Project• 1. At the heart of student retention and success is a strong sense

of belonging in HE for students. This is most effectively nurtured through mainstream activities that all students participate in.

• 2. The academic sphere is the most important site for nurturing belonging.

• 3. Specific interventions cannot be recommended over and above each other. Rather the institution, department and programme should all nurture a culture of belonging.

• 4. Student belonging is an outcome of: supportive peer relations; meaningful interaction between staff and students; developing knowledge, confidence and identity as successful HE learners; and an HE experience which is relevant to interests and future goals.

Benefits of engagement

Engaged students get better outcomes

Celebrating individual successMaximising individual rewardsMaximising contribution to societyMarket and non-market benefits

https://www.gov.uk/government/uploads/system/uploads/attachment_data/file/254101/bis-13-1268-benefits-of-higher-education-participation-the-quadrants.pdf

How do we do this in a region of low aspiration?

Engagement Outcomes

employability

Student surveysaward level

outcomesModule outcomes

Engagement Outcomes

• Students act increasingly as engaged consumers– Access price

comparisons– Access to performance

comparisons– Demand better

outcomes

• University needs to optimise– Module and award level

outcomes– Student survey results– Public information– League Tables

Outcomes

Number of “good” degreesNational Student Survey resultsRetention ratesEmployability

Inputs

Spend per studentStaff student ratiosEntry standardsResearch ratingsCost of livingSpend on servicesFaculty spend

An engaged student reads….. An engaged student contributes to…

• National Student Survey– Measure of final year students– Questions on satisfaction with course– Satisfaction with teaching and learning– Satisfaction with assessment and feedback

• Degree results• Retention• Employability

Used in 3 areas:– League tables/KIS (external)– Portfolio performance review (internal)– Award annual monitoring (internal)

Student Input to Information

League Tables – a mirror.

League Tables2008 2009 2010 2011 2012 2013 2014

0

20

40

60

80

100

120

Overall Position in Guardian Guide

Ovw

eral

l Pos

ition

2010 2011 2012 2013 201477.0

78.0

79.0

80.0

81.0

82.0

83.0

84.0

85.0

86.0

87.0

NSS Teaching (%)

67/119

year 2010 2011 2012 2013 201474.0

76.0

78.0

80.0

82.0

84.0

86.0

NSS Overall (%)

62/119

2010 2011 2012 2013 201458.0

60.0

62.0

64.0

66.0

68.0

70.0

72.0

74.0

NSS Feedback (%)

38/119

2010 2011 2012 2013 201417.5

18.0

18.5

19.0

19.5

20.0

20.5

21.0

21.5

22.0

Student: staff ratio

83/119

2010 2011 2012 2013 2014225.0

230.0

235.0

240.0

245.0

250.0

255.0

260.0

Entry Tariff

107/119

2010 2011 2012 2013 20140.0

10.0

20.0

30.0

40.0

50.0

60.0

70.0

Career prospects (%)

113/119

2010 2011 2012 2013 20140.0

1.0

2.0

3.0

4.0

5.0

6.0

Value added score/10

108/119

Are we trying to improving position or performance?• Clearly, we can try to play the game of moving

our league table position• What we really want to do is improve our

performance in each of the key areas to make sure there is a sustainable and genuine change

Guardian criteria Suggested Action

Entry standards Review all current standard offers to pitch ourselves properly against competitors

Student/staff ratio Reviewed more thoroughly the data we submit to HESADeveloping better models of SSR to identify where investment is most needed

Spend per student Reviewed more thoroughly the data we submit to HESAReviewed classification of spendIncreased recent spend on libraries and IT will have an impact

Guardian criteria Action

Value added Increased number of “good” degrees awarded. Reviewing all level 6 modules with low pass rates and average marks. Identifying through portfolio review awards with consistently poor progression and attainmentL&T conference on attainmentBME student performance

NSS teaching, assessment and feedback and overall satisfaction

Faculty action plans, and award level plansIncreased student engagement with surveySeven principles of feedbackOnline assessment and feedback projectReview through portfolio performance tool

employment Staffordshire Graduate – improving our students’ chances of successOn-campus graduate internships

Used in 3 areas:– League tables/KIS (external)– Portfolio performance review (internal)– Award annual monitoring (internal)

Student Input to Information

Portfolio performance review

• Measures:– Market– Academic outcomes inc

good degrees– Student satisfaction– Employability

• Uses– Comparison against

targets– Comparison between

awards– Annual monitoring– Portfolio decisions

SNC/ABB 2nd

stage enrol per

offer

total apps/2

nd stage enrol

retention 2011

retention 2012

progress 2011

progress 2012

% good degrees 2011

% good degrees 2012

total all levels enrol 2010

total all levels enrol 2011

total all levels enrol 2012

total all levels enrol 2013

dlhe 2010-11

dlhe 2011-12

NSS 2012 teaching

NSS 2012 assess and feedback

NSS 2012 overall

NSS 2013 teaching

NSS 2013 assess and feedback

NSS 2013 overall

score 1 market led (enter target below)

score 2 - quality led

0.5 5 90.0 90.0 90.0 90.0 56.0 60.0 80 85 86 70 85 86 72 85 80.00

0.22 9.50 100.0 100.0 100.0 100.0 66.7 50.0 10 10 8 9 100.0 80.0 78.0 65.0 75.0 89.0 74.0 90.0 80.87 100.690.12 16.00 93.3 86.4 80.0 72.7 25.0 66.7 27 28 21 14 50.0 0.0 97.0 94.0 100.0 83.0 85.0 87.0 60.71 81.830.42 3.40 100.0 77.8 100.0 77.8 50.0 25.0 9 7 9 9 0.0 50.0 78.0 65.0 75.0 89.0 74.0 90.0 94.68 79.92

Used in 3 areas:– League tables/KIS (external)– Portfolio performance review (internal)– Award annual monitoring (internal)

Student Input to Information

Annual Monitoring• Key part of quality process

– Reflection on award performance– Reflection on student engagement and outcomes

• Statistics – progression, achievement, retention• Response to external examiner comments• Reflection on employer input• Staffordshire Graduate• Reports are made available to students, to demonstrate our

engagement

• Critically important for partner input, and to recognise joint responsibility for improvements

How it fits together

Student satisfaction

Student engagement and outcomes

Student surveys League tables

reputation

Market demand

Course improvements

Action Plans Annual Monitoring

Improved engagement leads to improved outcomes

• improved outcomes for individual students– Grades and classifications– Employability –getting into the right jobs– Satisfaction– Satisfying aspirations

• improved overall outcomes– Institutional success and reputation– Changing the aspirations of our communities

How institutional data and analysis can help?• Identifying performance metrics at module

level• Identifying performance metrics for awards• Developing a culture where this analysis

becomes embedded– Being prepared to deal with the issue!

TLA practiceRelevant TL and assessment practiceResponsive curricula – employer and technology ledIncreased student engagement in course design and monitoringStaff engaged in reflective practice

Management InterventionQuestioning low average marks, poor attainment ratesManaging the performance of the portfolioIncreased emphasis on EEE (staff and students)Continuous staff developmentImproving quality of teaching

Uses of technologyElectronic assessment and feedbackOnline communitiesStudent portalMobile technologies

The future…..How we teach in 2013?How to learn in 2020?

How could we improve

engagement and outcomes for

institution?

Learning Analytics –

can work if all systems joined upA way to encourage or check on student engagement?

Every log inEvery use of VLE – pages read, scores on formative testsUse of library – frequency, downloads, loansAttendance at class or online

A way to audit teaching as well…….

Or is this another example of technological solutionism?

Improving individual engagement Using “big data”?

LEADERSHIP AND LEARNING ANALYTICS, http://www.educause.edu/library/resources/leadership-and-learning-analytics“To Save Everything, Click Here: Technology, Solutionism, and the Urge to Fix Problems that Don’t Exist” Evgeney Morozov, pb Allen Lane 2103, ISBN-10: 1846145481”

How will we engage students in learning when:

Half of what they learn in first year is out of date by the time they graduate?They will have maybe 10 different jobs in their career?The jobs they will do don’t even exist yet using technology that isn't invented yet,to solve problems we don’t know are problems yet.100 Billion queries a month on Google – who did we ask before?90% of the data in the world was created in the last 2 yearsThere are more students in the top 5% in China than all the students in the UK

@mikehamlyn blogs.staffs.ac.uk/mgh1/

http://vimeo.com/58839986