1

Education Data SciencesFraming Emergent Practices for Analytics of Learning, Organizations, and Systems

Philip J. PietyEd Info Connections

ppiety@edinfoconnections.com

Daniel T. HickeyLearning Sciences,

School of EducationIndiana University

dthickey@indiana.edu

MJ BishopCenter for Innovation and

Excellence in Learning & Teaching University System of Maryland

mjbishop@usmd.edu

2

Acknowledgements

3

Four Big Ideas

1. Sociotechnical paradigm shift2. Notion of Education Data Sciences (EDS)– Academic/Institutional Analysis– Learning Analytics/Educational Data Mining– Learning Analytics/Personalization– Systemic Instructional Improvement

3. Common features across these communities4. Framework for EDS

4

SOCIOTECHNICAL PARADIGM SHIFT IN CONCEPTION OF DATA

From External/Distant/Artificial to Internal/Current/Contextual

5

Paradigm Shifts

4. Disruption in traditional evidentiary practice

3. Expansion of academic knowledge

(ex: CCSS, NGSS)

2. Qualitative shift from

institution to individual

1. Digital tools create vast quantities, categories of data

6

The Educational Data Movement

Understanding how the organizational model of education is similar to/different from other fields is key to understanding the educational data movement.

1980 – 1990 - 2000 - 2010

Finance

Manufacturing

Retail

Health Care

Education

7

The Educational Data Movement

Understanding how the organizational model of education is similar to/different from other fields is key to understanding the educational data movement.

1980 – 1990 - 2000 - 2010

Finance

Manufacturing

Retail

Health Care

Big Data,Analytics,

Informatics, Data

Science

Education

8

The Educational Data Movement

9

Early Childhood

K-12

Post Secondary

Continuing/Career

Individuals Cohorts Organizations Systems

Scale of Educational Context

Educ

ation

al L

evel

(Age

)The EDS Landscape

10

Early Childhood

K-12

Post Secondary

Continuing/Career

Individuals Cohorts Organizations Systems

Scale of Educational Context

Educ

ation

al L

evel

(Age

)

Academic/Institutional

Analytics

Academic/Institutional Analytics

11

Academic/Institutional Analytics

12

Systemic/Instructional Improvement

Early Childhood

K-12

Post Secondary

Continuing/Career

Individuals Cohorts Organizations Systems

Scale of Educational Context

Educ

ation

al L

evel

(Age

)

Academic/Institutional

Analytics

Systemic/Instructional Improvement

13

Systemic/Instructional Improvement

“In many ways, the practice of data use is out ahead of research. Policy and interventions to promote data use far outstrip research studying the process, context, and consequences of these efforts. But the fact that there is so much energy promoting data use and so many districts and schools that are embarking on data use initiatives means that conditions are ripe for systematic, empirical study.”

Coburn, Cynthia E., and Erica O. Turner. "Research on data use: A framework and analysis." Measurement: Interdisciplinary Research & Perspective 9.4 (2011): 173-206.

14

Systemic/Instructional Improvement

Early Childhood

K-12

Post Secondary

Continuing/Career

Individuals Cohorts Organizations Systems

Lear

ning Analy

tics/

Ed. D

ata M

ining

Scale of Educational Context

Educ

ation

al L

evel

(Age

)

Academic/Institutional

Analytics

EDM/Learning Analytics

15

EDM/Learning Analytics

16

Systemic/Instructional Improvement

Early Childhood

K-12

Post Secondary

Continuing/Career

Individuals Cohorts Organizations Systems

Lear

ner A

naly

tics/

Pe

rson

aliz

ation Lear

ning Analy

tics/

Ed. D

ata M

ining

Scale of Educational Context

Educ

ation

al L

evel

(Age

)

Academic/Institutional

Analytics

LearnER Analytics/Personalization

17

LearnER Analytics/Personalization

18

Systemic/Instructional Improvement

Early Childhood

K-12

Post Secondary

Continuing/Career

Individuals Cohorts Organizations Systems

Lear

ner A

naly

tics/

Pe

rson

aliz

ation Lear

ning Analy

tics/

Ed. D

ata M

ining

Scale of Educational Context

Educ

ation

al L

evel

(Age

)

Academic/Institutional

Analytics

D. Flipped Classrooms

C. Early Warning Systems

A. School to College Analyses

B. Teacher Preparation

Efficacy Evaluation

Boundary Conditions

19

COMMON FEATURES & FACTORS IN EDUCATIONAL DATA SCIENCES

A unified perspective for Educational Data Science

20

Five Common Features in EDS

1. Rapidly changing- Indicative of sociotechnical movement

2. Boundary issues- All communities touch on other communities

3. Disruption in evidentiary practices- Big data is disrupting all the sectors

4. Visualization, interpretation, and culture- Dashboards, representations, APIs, open data

5. Ethics, privacy and governance - FERPA & COPPA

21

Four Factors that Make All Educational Data Unique

• Human/social creation–Most requires human manipulation

• Measurement imprecision–Reliability issues are huge

• Comparability challenges –Validity creates “wicked problems”

• Fragmentation–Systems can’t talk to each other

22

SOME COMMON PRINCIPLES

A unified perspective for Educational Data Science

23

Computer Science

Education Data Sciences

Statistical Data Analysis

Organization & Mgmt Sciences

Classroom/ Learning Technology

Learning Sciences

Decision Sciences

Machine Learning

Data Mining

Hum-Comp. Interaction &Visualization

Natural Language Processing

Computational Statistics

Interdisciplinary Perspectives

Information Sciences

24

Recognize Social/Temporal LevelsTimescale

Context

Targeted Educational

ContentTime

Frame

Format of Educational

EvidenceAppropriate Formative Function for Students

Ideal FormativeFunctions for Others

Immediate 

Curricular Activity(lesson)

Minutes Event-oriented observations (Informal observations of the enactment of the activity)

Discourse during the enactment of a particular activity.

Teacher: Refining discourse during the enactment of a particular activity. 

Close Curricular Routines (chapet/unit)

Days  Activity-oriented quizzes (semi-formal classroom assessments)

Discourse following the enactment of chapter, quiz.

Teacher: Refining the specific curricular routines and providing informal remediation to students.

Proximal Entire Curricula

Weeks  Curriculum-oriented exams (Formal classroom assessments)

Understanding of primary concepts targeted in curriculum. 

Teacher/curriculum developer: providing formal remediation and formally refining curricula. 

Distal Regional/National Content Standards

Months Criterion-referenced tests (external tests aligned to content standards)

  Administrators: Selection of curricula that have the largest impact on achievement in broad content domains. 

Remote National Achieve-ment

Years  Norm-referenced external tests standardized across years (ex: ITBS, NAEP)

  Policy makers:  Long-term impact of policies on broad achievement targets.

25

Digital Fluidity

State Longitudinal Data Systems

District Data Warehouses and

Teacher Evaluation Systems

Learning Tools-Driven

Analytics

School Teams

School Leaders

District Curriculum

District Leaders

Teacher Planning

Individual Students

State Analysis

26

Values in Design

Infrastructure and Tools Context

Organizational and Political Context

•routines•access to data•leadership•time•norms•power relations

Processes of data use

•noticing•interpreting•constructing implications

•data components•linkages•time span covered•Infrastructure boundaries•data quality•technology features

27

Flashlights, Imperfect Lenses

28

Four Big Ideas

1. Sociotechnical paradigm shift2. Notion of Education Data Sciences (EDS)– Academic/Institutional Analysis– Learning Analytics/Educational Data Mining– Learning Analytics/Personalization– Systemic Instructional Improvement

3. Common features across these communities4. Framework for EDS

29

Education Data SciencesFraming Emergent Practices for Analytics of Learning, Organizations, and Systems

Philip J. PietyEd Info Connections

ppiety@edinfoconnections.com

Daniel T. HickeyLearning Sciences,

School of EducationIndiana University

dthickey@indiana.edu

MJ BishopCenter for Innovation and

Excellence in Learning & Teaching University System of Maryland

mjbishop@usmd.edu