Current State and Future Trends:A citation network analysis of the Learning Analytics Field
description
Transcript of Current State and Future Trends:A citation network analysis of the Learning Analytics Field
S. DawsonD. Gasevic
G. SiemensS. Joksimovic
Current State and Future Trends:A citation network analysis of the
Learning Analytics Field
Goal
• Citation analysis and structural mapping to gain insight into the influence and impact within LA – a snapshot of LA through analysis of articles and
citations (LAK conferences and special issues)
Context
• Although much potential and excitement:– to date LA has served to identify a condition, but
has not advanced to deal with the learning challenges in a more nuanced and integrated manner
Aim
• Identify emergence of influential trends and hierarchies in the field
• Commencement point (Leah):– a foundation for future work– identify promising areas of research– Identify under represented disciplines– Improve integration across disciplines and theory
and practice
Context• Learning analytics:
– has emerged as a field (maturation)– multi-disciplinary– often mis-represented and poorly understood
• (Academic analytics; business intelligence; assessment analytics; social analytics; web analytics; educational data mining)
Approach
• Bibliometrics measure the impact/influence of an author or article using various citation analyses
• Garfield 1955 (Impact)
Ding, Y. (2011). Scientific collaboration and endorsement: Network analysis of coauthorship and citation networks, Journal of Informetrics, 5(1), 187-203
Data
• LAK11, 12, 13• Special issues – ETS, JALN, ABS• Data analysis
– Citation counts– Author/citation network analysis– Contribution type– Research methods– Author disciplinary background
Approach
• Citation and author networks:– Identify the prominent research– Identify linkages between disciplines and authors– Identify diversity of research genres
Citation analysis
• The use of citations long been used to measure impact– Core output of research is publications– As research grows, output (papers) further build
on other associated works (citations)– “quality” as quantity of citations
• Identify areas of prominent research activity
Citation analysis
• Criticisms– Cronyism– Self citations– “Rich get richer”
By the way some great refs
Gašević, D., Zouaq, A., & Janzen, R. (2013). “Choose Your Classmates, Your GPA Is at Stake!” The Association of Cross-Class Social Ties and Academic Performance. American Behavioral Scientist 57 (10), 1460-1479
Siemens, G. (2013). Learning Analytics The Emergence of a Discipline. American Behavioral Scientist 57 (10), 1380-1400
Dawson, S., Tan, J., & McWilliam, E. (2011). Measuring creative potential: Using social network analysis to monitor a learners' creative capacity. Australasian Journal of Educational Technology 27 (6), 924-942
Citation analysis
Citation analysis
• Ok Some faults – broadly accepted that quality is linked to the number of citations
Citation analysis
• Number of citations (Highly cited articles)– Predominately conceptual and opinion papers
(e.g. Educause)– Methods (Wasserman and Faust)– Few empirical studies
• Citation and author networks:– Illustrate linkages and disciplines
Example author networks
Paper 1: Liz, Julie, Sara
Sara
JulieLiz
Example author networks
Paper 1: Liz, Julie, Sara
Sara
JulieLiz
2 Degrees
Example citation network
Paper 1: Liz, Julie, SaraCites: Liam & Luka
Sara
JulieLiz
Liam
Luka
Citation network
LAK conferenceJournals
Citation networks
• Citation network moderate level of clustering– Consistent across LAK proceedings– Few strong connections? – Degrees low – indication of diverse and
inconsistent literature sources– Degrees (increasing) from LAK11 to 13
Author networks (LAK)
Author networks (Journal)
Author networks
• Author networks – small cliques with few highly connected nodes
• For an interdisciplinary field still largely disciplinary clustered
Paper classification
• Schema from Info Systems (6 categories)
1. Evaluation research – (e.g. case study empirical)
2. Validation research – (e.g. testing theory/ method/ solution empirical)
Glass, R.L., et.al, 2002. Research in software engineering: an analysis of the literature. Information and Software Technology 44, 8, 491-506
Paper classification
3. Solution proposal (solution/ technique to address an issue)
4. Conceptual proposal (e.g. frameworks)
5. Opinion(well its my opinion/argument)
6. Experience (Let me tell you a story)
7. Panel/workshop
Author Background
Extensive search – if home dept not listed in author details
Paper classification (LAK)
• Dominated by evaluations of research (journals)
• Proposal of solution dominates conference
Paper classification (Journal)
• Dominated by evaluations of research (journals)
• Proposal of solution dominates conference
Paper classification
• Dominated by computer science (LAK)• Greater number of education researchers in
journals– Reflection of special issues– Reflection of priority sites for publications
• Largely conceptual and opinion publications
Methods classification
• Qualitative• Quantitative• Mixed methods• Other
Methods classification (LAK)
• Other dominates – reflects high number of opinion and conceptual papers
Methods classification (Journals)
Methods classification
• Other dominates – reflects high number of opinion, experience and conceptual papers
Conclusions
• The field is in its infancy– Citations still predominately opinion and
definitional pieces– Clustering and degrees– Few number of empirical studies cited but this is
growing• Mature fields greater examples of validation
research and importantly critiques of studies
Conclusions
• Computer scientists dominate LAK proceedings– Need to look at how other voices are heard
• Education research dominates Journals– Reflection of broader priorities?
Conclusions
• Early work need to extend• Structural mapping and citation analyses more
common and more sophisticated.• Raise awareness
– Inform practice– Build connections– Foster further empirical work
Conclusions
• Understanding our field we can better advance our field.
• Question: To what extent can we use these analyses to architect the development of the field?
Questions
• Next steps:–Broader scope (extend network
analyses)–Keyword clustering–Citation location–Incorporate multiple citations/ paper
Questions
• To what extent can we use these analyses to architect the development of the field?
• [email protected]• [email protected]• [email protected]• [email protected]