Presentation at the Milpark Business School (MBS)

Research Colloquium 25 June 2016Image credit: Image compiled and adapted from an image retrieved from- https://pixabay.com/en/binary-code-man-display-dummy-face-1327498/

Using student data to inform support, pedagogy & curricula: ethical issues & dilemmas

By Paul Prinsloo (University of South Africa, Unisa)

Acknowledgements

I do not own the copyright of any of the images in this presentation. I therefore acknowledge the original

copyright and licensing regime of every image used.

This presentation (excluding the images) is licensed under a

Creative Commons Attribution-NonCommercial 4.0 International

License

Overview of the presentation

• The broader context of research on students in higher education

• Student data as Medusa – mapping the field, the tools, the actors, the dimensions and the implications

• Looking away: Pointers for consideration• (In)conclusions

Higher education should…

• Do more with less• Expect funding to follow performance rather than precede it• Realise it costs too much, spends carelessly, teaches poorly, plans

myopically, and when questioned, acts defensively(Hartley, 1995, p. 412, 861)

We also cannot & shouldn’t underestimate the impact of the dominant models of neoliberalism and its not-so-humble servant – managerialism – on higher education (Diefenbach, 2007)

The broader higher education context

Image credit: http://commons.wikimedia.org/wiki/File:Mcdonalds_logo.png

• Changes in funding and audit regimes –evidence-based policy versus research-led…

• Increasing concerns regarding student retention and dropout

• International ranking systems, increased competition in higher education

The broader higher education context (2)

(See: Murphie, A. (2014). Auditland. PORTAL Journal of Multidisciplinary International Studies, 11(2), 1-41.)

The broader higher education context (3)• How do national,

institutional, disciplinary contexts support or frustrate efforts to remove barriers?

• What are the issues of costs and scalability in erasing inter-generational inequality?

• What data do we need in order to move towards more just, caring and compassionate access, teaching and learning?Image sources: https://twitter.com/urbandata/status/695261718344290304

https://za.pinterest.com/barbaralley/fair-is-not-equal/

• Looking for sustainable business models #FeesMustFall

• The algorithmic turn and quantification fetish in higher education

• The increasing digitisation of learning and teaching, and access to students’ digital shadows

• The gospel of technosolutionism in higher education

• The lure of Big(ger) data

The broader higher education context (4)

(Student) data as Medusa

Higher education is mesmerized and seduced by the potential of the collection, analysis and use of student data. If only we know more…

Image credit: http://en.wikipedia.org/wiki/Medusa

There is an increasing need for data/evidence

We have access to increasing amounts and granularity of student data

We have increased capacity & technologies for analysis and visualisation

The impact of impotent, static, & obsolete legislation, policies and guidelines

And a lack of oversight and enforcement

Image credit: Retrieved from https://www.flickr.com/photos/timrich26/3308513067

We need to ensure the sustainability of higher education in the light of• funding constraints• increased competition• the socioeconomic

downturn• student needs• increased need for

efficiency/effectiveness• audit & quality

assurance regimes• #FeesMustFall

The fiduciary duty of higher education to• care• create supportive,

appropriate and effective teaching and learning environments

• ethical collection, analysis and use of student data

• transparency

A balancing act

See: Prinsloo, P., & Slade, S. (2014). Educational triage in open distance learning: Walking a moral tightrope. The International Review of Research in Open and Distributed Learning, 15(4), 306-331. Retrieved from http://www.irrodl.org/index.php/irrodl/article/view/1881/3060

So…, who has access to and use student data to inform student

support, pedagogy & curricula; and under what conditions and who

provides oversight

See: Willis, J. E., Slade, S., & Prinsloo, P. (in review). Ethical oversight of student data in learning analytics: A typology derived from a cross-continental, cross-institutional perspective. Submitted to special issue of Educational Technology Research and Development (Exploring the Relationship of Ethics in Design and Learning Analytics: Implications for the Field of Instructional Design and Technology), guest edited by M. Tracey and D. Ifenthaler.

When the collection, analysis and use of student data have an internal focus• Departmental/institutional reports

& planning• Scholarship of teaching and

learning• Provide appropriate and effective

student support• Allocation of staff/resources

When the collection, analysis and use of student data have an external focus• Reporting to a range of

stakeholders, e.g. government, industry, etc., and for a range of purposes, e.g., funding

• Conference presentations• Journal articles• Monographs & edited volumes• Popular press• Marketing

Institutional Research• Often located in a

designated department• Staffed by data

scientists, analysts• Inform strategy and

policy• Use student data

already ‘gifted’ during application/ registration process and from Learning Management System (LMS)

• Specific data collection• Often blanket ethical

clearance

Research (capital ‘R’)• Mostly faculty, but

increasingly support and professional staff• Varying skills and

understanding• Chasing outputs, h-

index, citations• Results mostly not used

to inform teaching and learning

• Use primary and secondary student data

• Oversight provided by Institutional Review

Boards (IRBs)

Emerging forms of research• Mostly faculty, but increasingly

support and professional staff• Varying skills and understanding• Not produced for formal outputs

eg publication, but to inform pedagogy, assessment, personalisation, departmental reports

• Often use student data already ‘gifted’ during application/ registration process and from Learning Management System (LMS) or personal synchronous or asynchronous communication

• No ethical review/oversight

Academic & learning analytics

Type of analytics

Level or object of analysis

Who benefits?

Learning analytics

Course level: social networks, conceptual development, discourse analysis, “intelligent curriculum”

Learners, faculty

Departmental: predictive modelling, patterns of success/failure

Learners, faculty

Academic analytics

Institutional: learner profiles, performance of academics, knowledge flow

Administrators, funders, marketing

Regional (state/provincial): comparisons between systems

Funders, administrators

National and International National governments, education authorities

Siemens, G., & Long, P. (2011). Penetrating the Fog: Analytics in Learning and Education. EDUCAUSE review, 46(5), 30-40. Retrieved from http://er.educause.edu/articles/2011/9/penetrating-the-fog-analytics-in-learning-and-education

“learning analytics is the measurement, collection, analysis and reporting of data about learners and their contexts, for purposes of understanding and optimising learning and the environments in which it occurs.”

1st International Conference on Learning Analytics and Knowledge, Banff, Alberta, February 27–March 1, 2011. In Siemens and Long (2011)

(1)Humans perform

the task

(2)Task is shared

with algorithms

(3)Algorithms

perform task: human

supervision

(4)Algorithms

perform task: no human input

Seeing Yes or No? Yes or No? Yes or No? Yes or No?

Processing Yes or No? Yes or No? Yes or No? Yes or No?

Acting Yes or No? Yes or No? Yes or No? Yes or No?

Learning Yes or No? Yes or No? Yes or No? Yes or No?

Human-algorithm interaction in the collection, analysis and use of student data

Danaher, J. (2015). How might algorithms rule our lives? Mapping the logical space of algocracy. [Web log post]. Retrieved from http://philosophicaldisquisitions.blogspot.com/2015/06/how-might-algorithms-rule-our-lives.html

We know where you are. We know where you’ve been. We can

more or less know what you're thinking

about

(@FrankPasquale, 2016)

Image credit: https://en.wikipedia.org/wiki/Surveillance

Imagine what we could learn if we put a tracker on everyone and everything (Jurdak, 2016)

Image credit: https://www.flickr.com/photos/jeepersmedia/13966485507

Page credit: http://insider.foxnews.com/2016/01/31/oklahoma-college-forcing-students-wear-fitbits

Page credit: https://dzone.com/articles/are-university-campuses-turning-into-big-brother

Page credit: http://www.theguardian.com/higher-education-network/2015/nov/27/our-obsession-with-metrics-turns-academics-into-data-drones

Page credits: http://www.ft.com/cms/s/2/634624c6-312b-11e5-91ac-a5e17d9b4cff.html#slide0

‘how much is enough data to solve my problem?’

(Adryan, 2015)

Image credit: https://www.flickr.com/photos/uncle-leo/1341913549

How much (more) student data do we need?

… has become saturated with data – ranging from automatically collected, analysed and used, purposefully collected, analysed and used and volunteered on social media and in exchange of (perceived) benefits despite concerns about privacy, the uncertainty of how the data will be used (and combined with other sources of data) downstream and in the context where our trust in the collectors of data is often misplaced, irrational or wishful thinking (See Kitchen, 2013, pp. 262-263)

How do we think of the collection, analysis and use of student data in a world that…

Image credit: https://commons.wikimedia.org/wiki/File:Big_Hand_-_geograph.org.uk_-_644552.jpg

• Knowing• Not knowing• Knowing what we don’t know• Knowing what we may never know• Knowing more

The solution is not only (or necessarily?) in knowing more, but ensuring that once we know, we respond in ethical, caring,

discipline and context-appropriate ways

We need to critically consider the ethical implications of …

Pointers for a way forward• Students’ digital lives are but a minute part of a bigger whole – so we

should not pretend as if our data represent the whole• The data we collect are never ‘raw’, ‘uncontaminated’, or just ‘scraped’…

Our samples, choices, timing and tools change and impact on data. “Data are in fact framed technically, economically, ethically, temporally, spatially and philosophically. Data do not exist independently of the ideas, instruments, practices, contexts and knowledges used to generate, process and analyse them” (Kitchen, 2014, p. 2)

• Data have contexts. To re-use data outside of the original context and purpose for which it was collected impacts on the contextual integrity.

• Knowing ‘what’ is happening, does not necessarily tell us the ‘why’…• Education is an open, recursive system (Biesta 2007, 2010) where multiple

variables not only intersect but often also constitute one another. Let us therefore tread carefully between correlation and causation…

Caught between correlation and causation

Image credit: http://www.tylervigen.com/spurious-correlations

Caught between correlation and causation (cont.)

Image credit: http://www.tylervigen.com/spurious-correlations

Using student data and student vulnerability: between the devil and the deep blue sea?

Students (some more

vulnerable than others)

Generation, harvesting and

analysis of data

Our assumptions, selection of data and algorithms

may be ill-defined

Turning ‘pathogenic’ – “a response intended to

ameliorate vulnerability has the paradoxical effect of exacerbating existing

vulnerabilities or generating new ones”

(Mackenzie et al, 2014, p. 9)

Adapted from Prinsloo, P., & Slade, S. (2015). Student vulnerability, agency and learning analytics: an exploration. Presentation at LAK15, Poughkeepsie, NY, 16 March 2015 http://www.slideshare.net/prinsp/lak15-workshop-vulnerability-final

• boyd and Crawford (2011) point to the fact that just because we have access to increasing amounts and granularity of personal data, does not mean that we have to and need to collect, analyse and use this data

• While research participant involvement in Research (with a capital ‘R’) is governed by institutional review boards and policies, the (automatic) collection, analysis and use of individuals’ digital data in emerging forms of research (small ‘r’) often fall and take place outside of these policies and review boards (Willis, Slade & Prinsloo, in review)

Just because we can, does not mean we have to, and if we do, who will provide oversight?

Collecting, analysing and using student data: towards an ethics of care

1. Do no harm. Repeat after me. Do no harm2. They have a right to know. If not, then this research

resembles surveillance and spying, and not research3. Make it clear what data are collected, when, for what

purpose, for how long it will be kept and who will have access and under what circumstances

4. Provide students access to information and data held about them, to verify and/or question the conclusions drawn, and where necessary, provide context

5. Provide access to a neutral ombudsperson(See Prinsloo & Slade, 2015)

Collecting, analysing and using student data: towards an ethics of care (2)

6. Context matters. Downstream use for purposes other than the original purpose for the collection of data compromises the contextual integrity of data

7. Involve students in the meaning-making. They are not data points on a PowerPoint at a conference. They have contexts, histories. They are infinitely more than their data.

8. Who will we hold accountable for algorithms? 9. What are the benefits for students? For you? For the

institution? Be transparent.(See Prinsloo & Slade, 2015)

(In)conclusionsI am not your data, nor am I your vote bank,I am not your project, or any exotic museum object,I am not the soul waiting to be harvested,Nor am I the lab where your theories are tested,I am not your cannon fodder, or the invisible worker,or your entertainment at India habitat centre,I am not your field, your crowd, your history,your help, your guilt, medallions of your victory,I refuse, reject, resist your labels,your judgments, documents, definitions,your models, leaders and patrons,because they deny me my existence, my vision, my space,your words, maps, figures, indicators,they all create illusions and put you on pedestal,from where you look down upon me,So I draw my own picture, and invent my own grammar,I make my own tools to fight my own battle,For me, my people, my world, and my Adivasi self! ~Abhay Xaxa

Source: http://www.adivasiresurgence.com/i-am-not-your-data/

THANK YOUPaul Prinsloo Research Professor in Open Distance Learning (ODL)College of Economic and Management Sciences, Office number 3-15, Club 1, Hazelwood, P O Box 392Unisa, 0003, Republic of South Africa

T: +27 (0) 12 433 4719 (office)T: +27 (0) 82 3954 113 (mobile)

prinsp@unisa.ac.za Skype: paul.prinsloo59

Personal blog: http://opendistanceteachingandlearning.wordpress.com

Twitter profile: @14prinsp

Bibliography and additional reading

Biesta, G. (2007) Why “what works” won’t work: evidence-based practice and the democratic deficit in educational research, Educational Theory, 57(1),1–22. DOI: 10.1111/j.1741-5446.2006.00241.x.

Biesta, G. (2010) Why ‘what works’ still won’t work: from evidence-based education to value-based education, Studies in Philosophy of Education, 29, 491–503. DOI 10.1007/s11217-010-9191-x.

Booth, M. (2012, July 18). Learning analytics: the new black. EDUCAUSEreview, [online]. Retrieved from http://www.educause.edu/ero/article/learning-analytics-new-black

Boyd, D., & Crawford, K. (2013). Six provocations for Big Data. Retrieved from http://papers.ssrn.com/sol3/papers.cfm?abstract_id=1926431

Chamayou, G. (n.d.). Every move will be recorded. [Web log post]. Retrieved from https://www.mpiwg-berlin.mpg.de/en/news/features/feature14

Ball, N. (2013, November 11). Big Data follows and buries us in equal measure. [Web log post]. Retrieved from http://www.popmatters.com/feature/175640-this-so-called-metadata/

Bergstein, B. (2013, February 20). The problem with our data obsession. MIT Technology Review. Retrieved from https://www.technologyreview.com/s/511176/the-problem-with-our-data-obsession/

Bertolucci, J. (2014, July 28). Deep data trumps Big Data. Information Week. Retrieved from http://www.informationweek.com/big-data/big-data-analytics/deep-data-trumps-big-data/d/d-id/1297588

Booth, M. (2012, July 18). Learning analytics: the new black. EDUCAUSEreview, [online]. Retrieved from http://www.educause.edu/ero/article/learning-analytics-new-black

Boyd, D., & Crawford, K. (2013). Six provocations for Big Data. Retrieved from http://papers.ssrn.com/sol3/papers.cfm?abstract_id=1926431

Brock, A. (2015). Deeper data: a response to boyd and Crawford. Media, Culture & Society, 37(7), 1084-1088.Chamayou, G. (n.d.). Every move will be recorded. [Web log post]. Retrieved from

https://www.mpiwg-berlin.mpg.de/en/news/features/feature14 Citron, D.K., & Pasquale, F. (2014). The scored society: Due process for automated predictions.

http://ssrn.com/abstract=2376209 Crawford, K. (2013, April 1). The hidden biases in Big Data. Harvard Business Review. Retrieved from

https://hbr.org/2013/04/the-hidden-biases-in-big-data/ Crawford, K. (2014, May 30). The anxieties of Big Data. The New Inquiry. Retrieved from

http://thenewinquiry.com/essays/the-anxieties-of-big-data Danaher, J. (2014, January 6). Rule by algorithm? Big Data and the threat of algocracy.[Web log post]. Retrieved

from http://philosophicaldisquisitions.blogspot.com/2014/01/rule-by-algorithm-big-data-and-threat.html Danaher, J. (2015, June 15). How might algorithms rule our lives? Mapping the logical space of algocracy. [Web

log post]. Retrieved from http://philosophicaldisquisitions.blogspot.co.za/2015/06/how-might-algorithms-rule-our-lives.html

Deleuze. G. (1992). Postscript on the societies of control. October, 59 pp. 3-7. Diakopoulos, N. (2014). Algorithmic accountability. Digital Journalism. DOI: 10.1080/21670811.2014.976411 Diefenbach, T, 2007, The managerialistic ideology of organisational change management, Journal of

Organisational Change Management, 20(1), 126 — 144. Eubanks, V. (2014, January 15). Want to predict the future of surveillance? Ask poor communities. The American

Prospect. Retrieved from http://prospect.org/article/want-predict-future-surveillance-ask-poor-communities

Floridi, L. (2012). Big data and their epistemological challenge. Philosophy & Technology, 1-3.

.

Bibliography and additional reading (cont.)

Gitelman, L. (Ed.). (2013). “Raw data” is an oxymoron. London, UK: MIT Press.Hartfield, T. (2015, May 12 ). Next generation learning analytics: Or, how learning analytics is passé. [Web log post]. Retrieved from http://timothyharfield.com/blog/2015/05/12/next-generation-learning-analytics- or-how-learning-analytics-is-passe/ Hartley, D. 1995. The ‘McDonaldisation’of higher education: food for thought? Oxford Review of Education, 21(4), 409-423.Henman, P. (2004). Targeted!: Population segmentation, electronic surveillance and governing the unemployed in Australia. International Sociology, 19, 173-191Johnson, J.A. (2015, October 7). How data does political things: The processes of encoding and decoding data are never neutral. [Web log post]. Retrieved from http://blogs.lse.ac.uk/impactofsocialsciences/2015/10/07/how-data-does-political-things/ Kitchen, R. (2013). Big data and human geography: opportunities, challenges and risks. Dialogues in Human Geography, 3, 262-267. SOI: 10.1177/2043820613513388 Kitchen, R. (2014). The data revolution. London, UK: SAGE. Kitchen, R., & McArdle, G. (2016). What makes Big Data, Big Data? Exploring the ontological characteristics of 26 datasets. Big Data & Society, January-June, 1-10. DOI: 10.1177/2053951716631130 Knox, D. (2010). Spies in the house of learning: a typology of surveillance in online learning environments. Paper presented at Edge, Memorial University of Newfoundland, Canada, 12-15 October.Lagoze, C. (2014). Big Data, data integrity, and the fracturing of the control zone. Big Data & Society (July-

December), 1-11.

Bibliography and additional reading (cont.)

Leonhard, G. (2014, February 25). How tech is creating data "cravability," to make us digitally obese. Retrieved from http://www.fastcoexist.com/3026862/how-tech-is-creating-data-cravability-to-make-us-digitally-obese?utm_content=buffer643a8&utm_medium=social&utm_source=twitter.com&utm_campaign=buffer

Lupton, D. (2015). The thirteen Ps of Big Data. This Sociological Life. Retrieved from https://www.researchgate.net/profile/Deborah_Lupton/publication/276207564_The_Thirteen_Ps_of_Big_Data/links/5552c2d808ae6fd2d81d5f20.pdf

Mayer-Schönberger, V. (2009). Delete. The virtue of forgetting in the digital age. Princeton, NJ: Princeton University Press.

Mayer-Schönberger, V., Cukier, K. (2013). Big data. London, UK: Hachette.Morozov, E. (2013a, October 23). The real privacy problem. MIT Technology Review. Retrieved from

http://www.technologyreview.com/featuredstory/520426/the-real-privacy-problem/ Morozov, E. (2013b). To save everything, click here. London, UK: Penguin Books. Morozov, E. (2013). To save everything, click here. London, UK: Penguin Books. Napoli, P. (2013). The algorithm as institution: Toward a theoretical framework for automated media production

and consumption. In Media in Transition Conference (pp. 1–36). DOI: 10.2139/ssrn.2260923Nissenbaum, H. 2015. Respecting context to protect privacy: Why meaning matters. Science and engineering

ethics. Retrieved from http://link.springer.com/article/10.1007/s11948-015-9674-9 Pasquale, F. (2015, October 14). Scores of scores: how companies are reducing consumers to single numbers. The Atlantic. Retrieved from http://www.theatlantic.com/business/archive/2015/10/credit- scores/410350/

Bibliography and additional reading (cont.)

Bibliography and additional reading (cont.)

Pasquale, F. [FrankPasquale]. (2016, February 19). "We know where you are. We know where you’ve been. We can more or less know what you're thinking about.” http://www.theatlantic.com/technology/archive/2016/02/google-cute-evil/463464/ … #Jigsaw [Tweet]. Retrieved from https://twitter.com/FrankPasquale/status/700473628605947904 Pasquale, F. (2015). The black box society. Harvard Publishing, US.Prinsloo, P. (2015, September 3). The ethics of (not) knowing our students. Presentation at the University of

South Africa, Pretoria. Retrieved from http://www.slideshare.net/prinsp/the-ethics-of-not-knowing-our-students-52373670

Prinsloo, P., & Slade, S. (2014). Educational triage in open distance learning: Walking a moral tightrope. The International Review of Research in Open and Distributed Learning, 15(4), 306-331. Retrieved from http://www.irrodl.org/index.php/irrodl/article/view/1881/3060

Prinsloo, P., & Slade, S. (2015, March). Student privacy self-management: implications for learning analytics. In Proceedings of the Fifth International Conference on Learning Analytics And Knowledge (pp. 83-92). ACM. Retrieved from http://dl.acm.org/citation.cfm?id=2723585

Prinsloo, P., Archer, E., Barnes, G., Chetty, Y., & Van Zyl, D. (2015). Big (ger) data as better data in open distance learning. The International Review of Research in Open and Distributed Learning, 16(1).

Rosen, J. (2010, July 21). The web means the end of forgetting. New York Times [Online].Selwyn, N. (2014). Distrusting educational technology. Critical questions for changing times. New York, NY:

RoutledgeScharmer, O. (2014, July 18). From Big Data to deep data. Huffington Post. Retrieved from

http://www.huffingtonpost.com/otto-scharmer/from-big-data-to-deep-dat_b_5599267.html

Bibliography and additional reading (cont.)

Shacklett, M. (2015, January 6). Thick data closes the gaps in big data analytics. TechRepublic. Retrieved from http://www.techrepublic.com/article/thick-data-closes-the-gaps-in-big-data-analytics/

Slade, S., & Prinsloo, P. (2013). Learning Analytics: Ethical Issues and Dilemmas. American Behavioral Scientist 57(1) ,1509–1528.Slade, S., & Prinsloo, P. (2015). Student perspectives on the use of their data: between intrusion, surveillance and care. European Journal of Open, Distance and Elearning. (pp.16- 28).Special Issue. http://www.eurodl.org/materials/special/2015/Slade_Prinsloo.pdfTene, O. & Polonetsky, J. (2013). Judged by the Tin Man: Individual rights in the age of Big Data. J. on Telecomm.

& High Tech. L., 11, 351.Totaro, P., & Ninno, D. (2014). The concept of algorithm as an interpretive key of modern rationality. Theory

Culture Society 31, pp. 29—49. DOI: 10.1177/0263276413510051 Uprichard, E. (2013). Big data, little questions. Discover Society, 1 October. Retrieved from

http://discoversociety.org/2013/10/01/focus-big-data-little-questions/ Wagner, D., & Ice, P. (2012, July 18). Data changes everything: delivering on the promise of learning analytics in

higher education. EDUCAUSEreview, [online]. Retrieved from http://www.educause.edu/ero/article/data-changes-everything-delivering-promise-learning-analytics-higher-education

Wang, T. (2013, January 20). Why Big Data needs thick data. Medium. Retrieved from https://medium.com/ethnography-matters/why-big-data-needs-thick-data-b4b3e75e3d7#.4jbatgurh

Watters, A. (2013, October 13). Student data is the new oil: MOOCs, metaphor, and money. [Web log post]. Retrieved from http://www.hackeducation.com/2013/10/17/student-data-is-the-new-oil/

41

Watters, A. (2014). Social justice. [Web log post]. Retrieved from http://hackeducation.com/2014/12/18/top-ed-tech-trends-2014-justice

Wigan, M.R., & Clarke, R. (2013). Big data’s big unintended consequences. Computer,(June), 46-53. Willis, J. E., Slade, S., & Prinsloo, P. (in review). Ethical oversight of student data in learning analytics: A typology

derived from a cross-continental, cross-institutional perspective. Submitted to special issue of Educational Technology Research and Development (Exploring the Relationship of Ethics in Design and Learning Analytics: Implications for the Field of Instructional Design and Technology), guest edited by M. Tracey and D. Ifenthaler.

Bibliography and additional reading (cont.)