EVALUATING NETWORK GROWTH: A CASE STUDY

USING SOCIAL NETWORK ANALYSIS

Goodier, S.;

;

© 2018, GOODIER, S.

This work is licensed under the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/legalcode), which permits unrestricted use, distribution, and reproduction, provided the original work is properly credited.

Cette œuvre est mise à disposition selon les termes de la licence Creative Commons

Attribution (https://creativecommons.org/licenses/by/4.0/legalcode), qui permet

l’utilisation, la distribution et la reproduction sans restriction, pourvu que le mérite de la

création originale soit adéquatement reconnu.

IDRC Grant/ Subvention du CRDI: 107311-001-Research into Open Educational Resources for

Development

Sarah GoodierROER4D Evaluation Advisor

EES 2016

29 September 2016

Evaluating network growth:A case study using Social Network Analysis

9/29/20161

",-'I-'t-.'ROER4uesearcn an Open amcauonal uesources lorDevalopmen1

216 mu xonnmnzz

IDRC 38 CRDI E35.’-.E:;Fi9uE5:}pMg‘ V)

r‘ L‘

Outline

• The project being evaluated: ROER4D

• Evaluation framework: what & how

• The Tools workflow

• Findings: Twitter network growth & online interactions

• Discussion & conclusion

THE ROER4D PROJECT

in the Global South

• In what ways, and under what circumstances can the adoption of OER address the increasing demand for accessible, relevant, high-quality and affordable education and what is its impact in the Global South?

Research on Open Educational Resources for Development

http://roer4d.org/

ROER4D has 86 researchers across 18 sub-projects in 26 countries across 16 time zones

h on Open Educational Resources for Development in the Global Sou

GENERAL OBJECTIVE:

IMPROVE EDUCATIONAL POLICY, PRACTICE and RESEARCH in developing Countries

by beuei understanding me use and impad of om

Kongullavnojecr CLUSTERS Augusl 201 3, Febnury zon

MghmkunvvPmsvan

adn anoioan ' n -Ie‘“”“ ~‘( v0 a k""=*'ia ' Philippines

Tamania

Brazil

Colombia

‘’ V 5viLanka

Teacher educate ‘ ' Malaysiaado of OER Rvnndl

Kenya

Fem

npunumensiwneseemioemiiaeueiepinenx ,,_,,,M,__,,,,_,,__,_,,,,‘,_,,__

Imm) Wllxvwyvvvxrll e.i...e.;.n.n,e.,..i.

mineuum/mend (‘A2 in m/ >1oA><i=r—uEL mm. IHIVVIIQVV

'°“‘"‘ “‘“‘ *"““”‘ ” ‘ ”‘ “ “““““

“ L ROER4D w-79-n iuncagcmi

1. Build an empirical knowledge base on the use and impact of OER in education

2. Develop the capacity of OER researchers

3. Build a network of OER scholars

4. Communicate research to inform education policy and practice

5. Curate output as open content

ROER4D Objectives

EVALUATING ROER4D

1. Build an empirical knowledge base on the use and impact of OER in education

2. Develop the capacity of OER researchers

3. Build a network of OER scholars

4. Communicate research to inform education policy and practice

5. Curate output as open content

ROER4D Key Evaluation Areas

EVA

LUA

TIO

N

Using the utilization focused evaluation (UFE) framework

What is utilization focused evaluation (UFE)?

UFE is centered around intended use by intended users

- What do the users (key stakeholders) want to know?

- How will answers to the evaluation questions potentially help to improve the project?

UFE in 12 steps:Steps are iterative,

not linear

Utilization Focused Evaluation Framework (Adapted from Ramirez & Brodhead, 2013)

step 1

Assessmg pmyam .mm.;s

¢Subl

Asxesxmg evahnamt madlnesx

sun 2

Vdznumns Dnmzlv mnendzd

51:9 5;

menufitatmnvi pnmarv

mended uses

m s:

rmm my gnuuwnn

¢may 1:

ivuvumnn awn

Sllp 3:

Smmmfnn plus!

sun 9:

mi: «mum

3:20 In:

um: anllvfls

sxevFicmlzfiun :11 la:

Sign A

5-muons! anawfls

step 12

Meta zvamanon

1. Build an empirical knowledge base on the use and impact of OER in education

2. Develop the capacity of OER researchers

3. Build a network of OER scholars

4. Communicate research to inform education policy and practice

5. Curate output as open content

ROER4D Key Evaluation Areas

Social networking

KEQ:To what extent has ROER4D built a network of OER scholars?

Twitter in communications

mm wmmu

ammnrgzm

M H . .4

>h:4t{|YY ‘ HV m

r~‘.w>«.:u vnhw K‘

|W'fiI®ayflMHHIMmi mama amnrre

flfl

The Tools workflowData collected monthly or

when the PIU requires an update

Account activity

\Interactions - 7-day w

/'

Interactions - over time

FINDINGS

Twitter Analytics325

am

275

250

§

Number:11

lnzilelnentsE

E

5

§

15

0;

u man exnands

I an clicks

I usar pmme clicks

- lzvontes

- mnlles

- retweets

TAGSom ov-.--;«»_3a...m.

V7

Or-mu°"“"“"‘”“

o«w==_gO~>m_5a..w...°”"‘”"“ ”“‘“""“ Owmo 3--=-5

°"‘’"M 0“M ‘W

‘mmommm

mu. o..aa..;_')u ..... (cm, /9 CW“ on-ma

weom.“ 0,‘.

/,.o \��o��o��q�W _..W

\��o��o^�q� .

um“ 0 Oon-um

MMM°u_U,,g_......A. o..m,.;§)'=-'~=~~

Oow»

Oo,...,mo~u.-...

5"

cm... 0 .,..

O

one-mnu.

0‘“‘“"L.u= u-4 ._..q....m3

0,...°“"'“"“"‘° O

O

ob-‘UBuA°Dl£lElA“9om,“

O>"“=*“.‘,“""m-«...— —— Rm...“ :1

‘HEW:M

ow ommm

° ‘3~~. lweets repnes mennons

».fi..&,_,_,_m4 own Qmnwe-u - scveen name ROERAU °**"“* °‘“‘

\ Owfg (mm. .., M May

0....‘ o‘‘‘'‘‘“!0“

,

1417 m Orr xmm

Om». omen.“ 5.2.2.4

owm qM,...,....‘ N omo_..........‘Dl,,h¢¢,£,,r;,‘.«..g

o.........g.

fig. 9a....-.1o...~,,.g.;,,mMz

V

\��o��o��q� O=='=~'«--=:~ om... omM..._.,§.‘,nm‘,,H___Kf§j;-mu:"d..4I4foml;‘@‘R~

“

M13“... ... emu.“

o,..,, 5”.‘ ,om... o,.:..m.‘

oc..x..m...‘

“Hum”

_W_ o...-...._m_

onC’ 0

wow“, :. o...m,O

.\��o��o��q� cm on 15117}

ow. “EBA”!

°"“‘"“ OM" °’"“‘“'"W

ow w-,.,,. ...

'”

°"“"""°0... IL: °,__M__

Owm mo......m0

o-«MuO’‘‘’‘‘‘ 0 '

0.4..

OW \ own 0......“ 0.).-Am.7

ommn nm

677 nudes 324 edges! keY / mermuns—-re1wee1

own I\ m

omsie;-;...,~om-w om.

nu». nun,

O-=4-«A»-M»°'fid'°'3wM

..

aw--W om.»

nu

NodeXL

Kev

. Indired tweetexs

. Isolated network

. Isolated tweeters

5 Direct and central tweelels

£7/an

(7. T19

1775

DISCUSSION & CONCLUSION

Key points

SNA focuses on relationships – good for networking

Tools workflow for ROER4D networking data

Tracks network growth and interactions over time

Evidence of success in the ROER4D project

Informed project & fueled improvement

Can be applied to other projects

This work is licensed under a Creative Commons Attribution 4.0 International License.

Website: http://roer4d.org

Contact author: Sarah Goodier sarah.goodier@uct.ac.za@SarahGoodier

Excluding images,

screenshots and

logos and/or

unless otherwise

indicated on

content

Thank you!

Further reading:

About ROER4D:• Hodgkinson-Williams, C. (2013). Research on Open

Educational Resources for Development in Post-secondary Education in the Global South (ROER4D) - Scoping Document. Available online: http://hdl.handle.net/11427/8430 [Last accessed 10 February 2015].

• Hodgkinson-Williams, C. and Cartmill, T. (2014). Research on Open Educational Resources for Development in the Global South: 1st Technical Report 23 June 2013 to 27 August 2014. Available online: http://hdl.handle.net/11427/9695 [Last accessed 11 February 2015].

About evaluation:• Patton, M. Q. (2008). Utilization-focused evaluation.

California: Sage Publications Inc.

• Ramirez, R. and Brodhead, D. (2013). Utilization Focused Evaluation: A Primer for Evaluators. Penang: Southbound.