The role of institutional data in Learning Analytics

The role of institutional data in Learning Analytics

Dr Abelardo Pardo (@abelardopardo)Faculty of Engineering and IT slideshare.net/abelardo_pardo

Australian Association for Institutional Research SIG Forum 17 June 2016, Adelaide, Australia

Adam Jones flickr.com

https://slideshare.net/abelardo_pardo

https://www.flickr.com/photos/adam_jones/11231668364

Abelardo Pardo The role of institutional data in Learning Analytics 2

Analytics in

Education

Potential for

ImpactLearning Analytics

Strategic Elements

Nico H

ogg flickr.com

https://www.flickr.com/photos/nicohogg/14753871788


Analyticsin

Education

Nico H

ogg flickr.com



Analytics instead of intuition do drive actionsSteinberg, L. (2015). Changing the Game: The Rise of Sports Analytics. Forbes. Retrieved from http://www.forbes.com/sites/leighsteinberg/2015/08/18/changing-the-game-the-rise-of-sports-analytics

http://www.forbes.com/sites/leighsteinberg/2015/08/18/changing-the-game-the-rise-of-sports-analytics


diffusor flickr.com

Big Data has landed in Higher Education…

… it was there all along!

http://www.flickr.com/photos/15691748@N00/4525359622


http://www.theguardian.com/guardian-professional/2015/jun/30/how-universities-are-using-data-to-stop-students-dropping-out

http://www.theguardian.com/guardian-professional/2015/jun/30/how-universities-are-using-data-to-stop-students-dropping-out


deepwarren flickr.com

Observe in and out of the classroom

http://www.flickr.com/photos/fuzzhead/7739629140




Dm

itry Grigoriev flickr.com

Learning Analytics: measure, collect, analyse data about learners to understand and improve their learning and the environment in which it occurs

https://www.flickr.com/photos/at8eqeq3/16026887670


Eustaquio Santimano flickr.com

Collect

Report

Analyse

Act

Refine

https://www.flickr.com/photos/eustaquio/6065064375


Australian Learning Analytics Summer Institute

https://www.itl.usyd.edu.au/analytics/alasi.htm


Long, P., & Siemens, G. (2011). Penetrating the Fog: Analytics in Learning and Education. EDUCAUSE Review, 48(5), 31-40.


Analytics in

Education

Potentialfor

Impact

Nico H

ogg flickr.com



Bichsel, J. (2012). Analytics in Higher Education Benefits, Barriers, Progress, and Recommendations (Research Report). Louisville, CO, USA: Educause Center for Applied Research.

• Collecting more data than ever

• Used to satisfy credentialing or reporting requirements

• Not used to address strategic questions

• High potential in resource optimisation, administrative functions

Carol Von C

anon flickr.com

https://www.flickr.com/photos/biggreymare/2667770265


Norris, D. M., & Baer, L. L. (2013). Building Organizational Capacity for Analytics. Louisville, CO, USA: Educause Library.

• Ease to capture value: bottom quartile

• IT Intensity: bottom quartile

• Data-driven mind set: bottom quartile

• Data availability: bottom quartile

Shallom Johnson flickr.com

https://www.flickr.com/photos/shallom/2228327975


Macfadyen, L. P., Dawson, S., Pardo, A., & Gašević, D. (2014). Embracing Big Data in Complex Educational Systems: The Learning Analytics Imperative and the Policy Challenge. Research & Practice in Assessment, 9, 17-28.

shifts in the culture, technological infrastructure, and teaching practices are needed!

Potential for improving learning, development of self-regulated learning skills, student success


Shacklock, X. (2016). From bricks to clicks. The potential of data and analytics in higher education. London, UK: Higher Education Commission.

• Essential building blocks are in place

• Opportunities offered by data

• Planning for the future is poor


Siemens, George, Shane Dawson, and Grace Lynch. “Improving the Quality and Productivity of the Higher Education Sector. Policy and Strategy for Systems-Level Deployment of Learning Analytics.” Society for Learning Analytics Research for the Australian Office for Learning and Teaching, 2013.


Analytics in

Education

Potential for

ImpactLearningAnalytics

Nico H

ogg flickr.com



Long, P., & Siemens, G. (2011). Penetrating the Fog: Analytics in Learning and Education. EDUCAUSE Review, 48(5), 31-40.

Abelardo Pardo The role of institutional data in Learning Analytics

Active Learning

anything course-related that all students in a class session are called upon to do other than simply watching, listening and taking notes

21Felder, Richard M., and Rebeca Brent. “Active Learning: An Introduction.” ASQ Higher Education 2, no. August (2009).


Sam Abraham

flickr.com

Blended Learning

Frontier between physical and virtual spaces is blurring

https://www.flickr.com/photos/live-by-the-ocean/889324885


2 Hours

2 Hours

3 Hours

Pardo, A. & Mirriahi N. (in press). Design, Deployment and Evaluation of a Flipped Learning First Year Engineering Course. In C. Reidsema, L. Kavanagh, R. Hadgraft & N. Smith (Eds.), Flipping the Classroom: Practice and Practices. Singapore: Springer.


Tracking


No statistically significant difference in the rating of feedback (2013 edition, M=3.25, SD=0.97; 2014 edition, M=3.35, SD=1.03); t(389.78) = -0.97, p <0.17


Eustaquio Santimano flickr.com

Collect

Report

Analyse

Act

Refine

https://www.flickr.com/photos/eustaquio/6065064375


You should take a more careful look at how symbols are encoded in the video. Would you be able to encode/decode UAL symbols without looking at the video?

Good initial work. However, did you understand the trick to handle encoding with a variable number of bits? Would you be able to provide an example?

Good work. Would you be able to come up with your own machine language and your encoding scheme? Remember that it has to be unambiguous.

Thorough work with the task about machine language encoding. Give it a quick review before the midterm.

Q1 Q2 Q3 Q4

Instructor


Algorithm


AutomaticEmail


Significant difference in the rating of feedback (2014 edition, M=3.35, SD=1.03; 2015 edition, M=3.82, SD=0.90); t(389.78) = -4.88, p <10-6 Effect size (Cohen’s d) = 0.49. Medium positive effect


Significant difference in the midterm score (2014 edition, M=12.80, SD=4.79; 2015 edition, M=13.83, SD=4.89); t(684.5) = -2.86, p < 0.002 Effect size (Cohen’s d) = 0.21. Small positive effect


Jovanović, J., Pardo, A., Gasevic, D., Dawson, S., Mirriahi, N. (2016). Dynamic Analytics of Learning Strategies in Flipped Classrooms. Manuscript in preparation.

Learning Strategies


Predictive models• Week 10

• Predicted score at leaves (out of 40)

• Conditions at nodes

• If (EXC.in >=22) and (VID.PL < 8.5) then score = 6

Pardo, A., Mirriahi, N., Martínez-Maldonado, R., Jovanović, J., Dawson, S., & Gasevic, D. (2016). Generating Actionable Predictive Models of Academic Performance. Paper presented at the International Conference on Learning Analytics and Knowledge, Edinburgh.


Refine Learning Design


TrackingKrugazor flickr.com

https://www.flickr.com/photos/36453784@N00/470295396


Liu, D. Y.-T., Bartimote-Aufflick, K., Pardo, A., & Bridgeman, A. J. (2016). Data-driven Personalization of Student Learning Support in Higher Education. In A. Peña-Ayala (Ed.), Learning analytics: Fundaments, applications, and trends: A view of the current state of the art. In preparation: Springer.

• Upload student listing

• Upload indicators per student

• Write filters or rules

• Contact selected students with personalised email


What if I tell you…


Analytics in

Education

Potential for

ImpactInstitutional Adoption

StrategicElements

Nico H

ogg flickr.com



Learning analytics and Education SystemsJ Brew

flickr.com

Built on networks and relationships

Relationships span beyond organisational silos

https://www.flickr.com/photos/brewbooks/7780990192


Pardo, A., & Dawson, S. (2014). Learning Analytics: What capacities do IT departments need in order to cope with complexities and uncertainties and how can the university leadership team best support them Invited talk at EduTECH Brisbane, Australia.

Cross-disciplinary approach




• Student Engagement

• Assessment

• Feedback

• Learning Design

• etc.




• Student Engagement

• Assessment

• Feedback

• Learning Design

• etc.

• Data warehousing

• Single source of truth

• Data cleaning

• Interfaces

• etc.


Powell, S., & MacNeil, S. (2012). Analytics Series Institutional Readiness for Analytics. CETIS Analytics Series, 1(8), 1-11.


Siemens, George, Shane Dawson, and Grace Lynch. “Improving the Quality and Productivity of the Higher Education Sector. Policy and Strategy for Systems-Level Deployment of Learning Analytics.” Society for Learning Analytics Research for the Australian Office for Learning and Teaching, 2013.


Chris Yunker flickr.com

BI ICT Learning and Teaching

Break the silos

Step 0

https://www.flickr.com/photos/chris-yunker/3732340472

Abelardo Pardo The role of institutional data in Learning Analytics

• What are the questions that need to be answered?

• What are the available data sources?

• What is needed to maintain them?

• What tools are required?

• Agility to adapt to new scenarios, data sources.

• What skills and expertise?

47

Further Steps


Jayneandd flickr.com

• Institutions already collect massive amounts of data

• Huge potential for widespread improvements

• Challenging to articulate

• Needs leadership vision and

• Cross-disciplinary approach

https://www.flickr.com/photos/jayneandd/4450623309

The role of institutional data in Learning Analytics

Dr Abelardo Pardo (@abelardopardo)Faculty of Engineering and IT slideshare.net/abelardo_pardo

Australian Association for Institutional Research SIG Forum 17 June 2016, Adelaide, Australia

Adam Jones flickr.com

https://slideshare.net/abelardo_pardo

https://www.flickr.com/photos/adam_jones/11231668364

The role of institutional data in Learning Analytics

Education

Transcript of The role of institutional data in Learning Analytics